Image processing method and device, equipment, medium and program product

Abstract

The invention provides an image processing method and device, equipment, a medium and a program product, and relates to the technical field of computers, and the method comprises the steps: obtaining a to-be-processed image, inputting the to-be-processed image into an interest value extraction network, obtaining an interest value feature map, determining the interest value distribution of the to-be-processed image according to the interest value feature map, and determining the interest value of the to-be-processed image according to the interest value distribution. And performing hierarchical compression processing on the to-be-processed image to obtain a compressed image, encoding and decoding the compressed image, and inputting the encoded and decoded image into the intelligent task network to execute the to-be-processed task. According to the method, the image is subjected to one-time grading compression processing before being coded, so that the information amount of some areas in the to-be-processed image can be reduced and the compression ratio can be improved in a pre-compression mode, and the contrast among different areas of the to-be-processed image can be improved, and the performance during task execution can be improved.

Description

Background Art

[0002] In order to reduce the amount of data transmission, it is necessary to encode the image or video, then restore the image at the decoding end, and then perform subsequent intelligent tasks.

[0003] In related technologies, image encoding and decoding usually directly encodes and decodes the original image without considering the performance of the decoded image in the subsequent intelligent task network.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides an image processing method, apparatus, device, medium and program product, which can reduce the amount of information in some areas of the image to be processed to a certain extent, improve the compression ratio, and improve the contrast between different areas of the image to be processed, so as to improve the performance when executing the task.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] In a first aspect, an embodiment of the present disclosure provides an image processing method, the method comprising:

[0008] Get the image to be processed;

[0009] Input the image to be processed into the interest value extraction network to obtain the interest value feature map;

[0010] Determine the interest value distribution of the image to be processed according to the interest value feature map;

[0011] According to the distribution of interest values, the image to be processed is subjected to hierarchical compression processing to obtain a compressed image, wherein different levels of compression processing correspond to different quantization parameters;

[0012] The compressed image is encoded and decoded, and input into the intelligent task network to execute the task to be processed.

[0013] In a possible embodiment, the interest value extraction network includes: a residual network and a feature pyramid network; wherein, starting from the second layer of the residual network, each layer is connected to each layer of the feature pyramid network;

[0014] The image to be processed is input into the interest value extraction network model to obtain the interest value feature map, including:

[0015] Input the image to be processed into the residual network to obtain the feature map output by the second convolutional layer of the residual network;

[0016] The feature map output by the second convolutional layer of the residual network is input into the second layer of the feature pyramid network to obtain the interest value feature map.

[0017] In a possible embodiment, determining the interest value distribution of the image to be processed according to the interest value feature map includes:

[0018] According to the interest value feature graph, according to the preset object importance weighting, determine the interest value distribution graph after the interest value feature graph is quantized;

[0019] The distribution map is upsampled according to the interest value distribution map to determine the interest value distribution of the image to be processed.

[0020] In a possible embodiment, performing hierarchical compression processing on the image to be processed according to the distribution of interest values ​​to obtain a compressed image includes:

[0021] According to the preset partition value and the distribution of the interest value, a plurality of regions at different levels in the image to be processed are determined;

[0022] Hierarchical compression processing is performed on multiple regions at different levels to obtain compressed images.

[0023] In a possible embodiment, determining a plurality of regions at different levels in the image to be processed according to the preset partition value and the interest value distribution includes:

[0024] Determine a first region of the image to be processed; the first region is a region where the interest value is greater than a first preset value;

[0025] Determine a second region of the image to be processed; the first region is a region where the interest value is less than a second preset value;

[0026] Determine a third area of ​​the image to be processed; the third area is an area where the interest value is less than or equal to the first preset value and greater than or equal to the second preset value.

[0027] In a possible embodiment, hierarchical compression processing is performed on multiple regions at different levels to obtain a compressed image, including:

[0028] Performing a first level compression process on a first region of the image to be processed, performing a second level compression process on a third region, and performing a third level compression process on the second region to obtain a compressed image;

[0029] Among them, the quantization parameters corresponding to the first level compression processing, the second level compression processing and the third level compression processing are sorted from small to large.

[0030] In a possible embodiment, the compressed image is encoded and decoded, and input into the intelligent task network to perform the task to be processed, including:

[0031] Perform image encoding processing on the compressed image to obtain an encoded code stream file;

[0032] The code stream file is transmitted and image decoding is performed to obtain a decoded image;

[0033] The decoded image is input into the intelligent task network to perform the task to be processed.

[0034] In a second aspect, an embodiment of the present disclosure provides an image processing device, including:

[0035] An acquisition unit, used for acquiring an image to be processed;

[0036] An extraction unit, used for inputting the image to be processed into the interest value extraction network to obtain an interest value feature map;

[0037] A determination unit, used to determine the interest value distribution of the image to be processed according to the interest value feature map;

[0038] A compression unit, used for performing hierarchical compression processing on the image to be processed according to the distribution of the interest values ​​to obtain a compressed image, wherein different levels of compression processing correspond to different quantization parameters;

[0039] The task execution unit is used to encode and decode the compressed image and input it into the intelligent task network to execute the task to be processed.

[0040] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the method in the first aspect above by executing the executable instructions.

[0041] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method in the first aspect described above is implemented.

[0042] In a fifth aspect, according to another aspect of the present disclosure, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above methods.

[0043] The embodiments of the present disclosure provide an image processing method, apparatus, device, medium and program product, the method comprising: obtaining an image to be processed, inputting the image to be processed into an interest value extraction network, obtaining an interest value feature map, determining the interest value distribution of the image to be processed according to the interest value feature map, performing hierarchical compression processing on the image to be processed according to the interest value distribution, obtaining a compressed image, encoding and decoding the compressed image, and inputting it into an intelligent task network to execute the task to be processed. By performing a hierarchical compression processing on the image before encoding in the above manner, not only can the compression ratio be improved by pre-compression, the compression efficiency is improved, and the amount of data transmission is reduced, but also the amount of information in some areas of the image to be processed can be reduced, and the contrast between different areas of the image to be processed can be improved.

[0044] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0046] Figure 1 A flowchart of an image processing method according to an embodiment of the present disclosure is shown;

[0047] Figure 2 A schematic diagram showing the structure of a residual network and a feature pyramid network in an embodiment of the present disclosure is shown;

[0048] Figure 3 A flow chart of obtaining an interest value feature map in an embodiment of the present disclosure is shown;

[0049] Figure 4 A schematic diagram showing the structure of another residual network and feature pyramid network in an embodiment of the present disclosure is shown;

[0050] Figure 5 A flow chart of determining the distribution of interest values ​​of an image to be processed according to an embodiment of the present disclosure is shown;

[0051] Figure 6 A flow chart of obtaining a compressed image in an embodiment of the present disclosure is shown;

[0052] Figure 7 A flowchart showing another image processing method in an embodiment of the present disclosure is shown;

[0053] Figure 8A schematic diagram showing the structure of an image processing device according to an embodiment of the present disclosure is shown;

[0054] Fig. 9 A schematic structural diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0055] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0056] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0057] Because in related technologies, image encoding and decoding usually directly encodes and decodes the original image without considering the performance of the decoded image in the subsequent intelligent task network, some specific intelligent task networks process the image in the above way and have poor performance in executing the task.

[0058] The embodiments of the present disclosure provide an image processing method, apparatus, device, medium and program product, the method comprising: obtaining an image to be processed, inputting the image to be processed into an interest value extraction network, obtaining an interest value feature map, determining the interest value distribution of the image to be processed according to the interest value feature map, performing hierarchical compression processing on the image to be processed according to the interest value distribution, obtaining a compressed image, encoding and decoding the compressed image, and inputting it into an intelligent task network to execute the task to be processed. By performing a hierarchical compression processing on the image before encoding in the above manner, not only can the amount of information in some areas of the image to be processed be reduced by pre-compression, the compression efficiency can be improved, the amount of data transmission can be reduced, and the compression ratio can be improved, but also the contrast between different areas of the image to be processed can be improved, and the accuracy loss of subsequent intelligent tasks is small.

[0059] The present exemplary implementation is described in detail below with reference to the accompanying drawings and embodiments.

[0060] First, an image processing method is provided in an embodiment of the present disclosure. The method can be executed by any electronic device with computing and processing capabilities.

[0061] Figure 1 A flowchart of an image processing method in an embodiment of the present disclosure is shown. Figure 1 As shown, the method provided in the embodiment of the present disclosure includes the following steps:

[0062] S102: Acquire an image to be processed.

[0063] Among them, the image in the present disclosure can be a single image or an image captured from a video. Based on this, it can be understood that the image processing method in the present disclosure can also be used to compress the video.

[0064] Exemplarily, the input image to be processed is used to perform a task, the input image to be processed can be used in a visual recognition algorithm, sent to a visual algorithm model for visual recognition, and output a processing result.

[0065] S104: Input the image to be processed into the interest value extraction network to obtain an interest value feature map.

[0066] In a possible embodiment, the image to be processed is extracted to obtain a feature map that can be used to analyze the interest value. In order to ensure the accuracy of subsequent interest value analysis, the interest value feature map can be a high-resolution feature map.

[0067] S106: Determine the interest value distribution of the image to be processed according to the interest value feature map.

[0068] The interest value is a quantitative value. In the field of image processing and computer vision, the quantified interest value is used to represent the different meanings and contents of different points or regions in the image.

[0069] S108: Perform hierarchical compression processing on the image to be processed according to the distribution of the interest values ​​to obtain a compressed image, wherein different levels of compression processing correspond to different quantization parameters.

[0070] S110: Encode and decode the compressed image and input it into the intelligent task network to execute the task to be processed.

[0071] In a possible embodiment, the image to be processed is compressed by hierarchical compression processing according to the distribution of interest values, which can reduce the amount of information in some areas of the image to be processed and improve the contrast between different areas of the image to be processed. The compressed image can be used in subsequent intelligent tasks, and the compression ratio in the subsequent encoding and decoding process can be improved by pre-compression.

[0072] In a possible embodiment, the way of compressing the image in the hierarchical compression process may use an end-to-end compression method, or may use a traditional image compression method such as VTM, without limitation.

[0073] It should be noted that the hierarchical compression in the present disclosure refers to different levels of compression processing corresponding to different quantization parameters (QP). The highest level of compression processing is no compression, corresponding to the highest image quality, and the quantization parameter is 0.

[0074] In a possible embodiment, the interest value extraction network in the embodiment of the present disclosure may be various networks capable of extracting high-resolution feature maps. In the following embodiment, the interest value extraction network is described by taking the interest value extraction network including: Residual Network (ResNet) and Feature Pyramid Network (FPN) as an example. The residual network may be a 50-layer residual network (ResNet50).

[0075] in, Figure 2 A schematic diagram of the structure of a residual network and a feature pyramid network is shown; Figure 2 As shown, starting from the second layer of the residual network, each layer is connected to each layer of the feature pyramid network, that is, the second layer of the residual network is connected to the first layer of the feature pyramid network, the third layer of the residual network is connected to the second layer of the feature pyramid network, and so on.

[0076] Through the structure of residual network and feature pyramid network, we can get the feature map of interest value. Figure 3 A flow chart of obtaining a feature map of interest value is shown, such as Figure 3 As shown, the following steps are included:

[0077] S302: Input the image to be processed into the deep residual network to obtain a feature map output by the second convolutional layer of the deep residual network.

[0078] S304: Input the feature map output by the second convolutional layer of the deep residual network into the second layer of the feature pyramid network to obtain an interest value feature map.

[0079] In a possible embodiment, the feature map processed by the residual network and the feature pyramid network can obtain better and richer contextual semantic information as the number of layers increases, but the resolution will become lower. In order to accurately identify each area in the interest value feature map and determine the interest value in subsequent processing, the present disclosure needs to use a high-resolution feature map and select the output features of the second layer, namely the P2 layer, as the interest value feature map.

[0080] In a possible embodiment, if there are some tasks to be performed in the intelligent task network that pay more attention to semantic information, deep feature maps can be used.

[0081] In one possible embodiment, Figure 4 The schematic diagram of the structure of the residual network and the feature pyramid network is shown; Figure 4 As shown, Figure 4 In the example, the structure of ResNet50+FPN is used. Figure 4 The figure shows the first 5 layers of the ResNet50 network, the first 4 layers of the FPN network, and the specific connection relationship between ResNet50 and FPN.

[0082] like Figure 4 As shown, the input image to be processed is of size 640×640×3 and is first input into ResNet50 for processing. For the specific processing process, refer to Figure 4 , no more details are given.

[0083] The feature map output by Stage 1 is input into the first layer of the FPN network to obtain a 160×160×256 interest value feature map.

[0084] In one possible embodiment, Figure 5 A flow chart of determining the distribution of interest values ​​of an image to be processed is shown. Figure 5 As shown, the following steps are included:

[0085] S502: According to the interest value feature map, determine an interest value distribution map after quantizing the interest value feature map according to a preset object importance weighting.

[0086] In a possible embodiment, the preset object importance weighting may be set for different objects.

[0087] According to different pending tasks to be executed in the intelligent task network, object importance weights are pre-set for different objects.

[0088] Among them, the pre-set method may include: using automatic image recognition and semantic information analysis to identify objects that are associated with the task to be processed, and automatically setting object importance weights for the objects. For example, if the task to be processed is visual recognition of autonomous driving or intelligent traffic control tasks, the objects associated with this type of task are determined through semantic information analysis, and the objects are identified in the image through automatic image recognition, and the object importance weights are automatically set for the objects.

[0089] Exemplarily, the tasks to be processed are visual recognition for autonomous driving and intelligent traffic control tasks, and importance weights can be set for objects such as people and cars.

[0090] Exemplarily, if the task to be processed is a recognition task of a guide robot, the target area is roads and distinctive road signs, and importance weighting is set for the roads and / or road signs.

[0091] Exemplarily, if the task to be processed is a task of medical image analysis, the target area is a lesion part that needs to be identified, and an importance weighting is set for the lesion part.

[0092] The interest value distribution map after the interest value feature map is quantized is determined by calculating the preset object importance weights.

[0093] S504: Perform distribution map upsampling according to the interest value distribution map to determine the interest value distribution of the image to be processed.

[0094] In a possible embodiment, after obtaining the interest value distribution map, since it is the interest value distribution of the interest value feature map, in the encoding and decoding process, it is necessary to encode and decode the original image, that is, the image to be processed, so it is necessary to upsample the distribution map to determine the interest value distribution of the image to be processed.

[0095] In one possible embodiment, Figure 6 A flow chart of obtaining a compressed image is shown. Figure 6 As shown, the following steps are included:

[0096] S602: Determine a plurality of regions at different levels in the image to be processed according to the preset partition value and the interest value distribution.

[0097] In a possible embodiment, based on the interest value distribution of the image to be processed, the interest values ​​of different points or areas in the image to be processed can be determined, and the preset partition value is used to compare with the interest values ​​of different points or areas in the image to be processed to divide the image to be processed into multiple areas of different levels.

[0098] In a possible embodiment, a preset partition value may be set according to a task to be performed for an image to be processed.

[0099] For example, if the task to be processed is visual recognition of autonomous driving or intelligent traffic control task, two preset partition values ​​can be set, and the specific values ​​of the preset partition values ​​are set to A and B.

[0100] If the task to be processed is a recognition task of a guide robot, one preset partition value may be set, and a specific value of the preset partition value is set to C.

[0101] If the task to be processed is a medical image analysis task, two preset partition values ​​may be set, and specific values ​​of the preset partition values ​​are set to D and E.

[0102] In a possible embodiment, the method for determining multiple areas of different levels may include: determining a first area of ​​the image to be processed; the first area is an area whose interest value is greater than a first preset value; determining a second area of ​​the image to be processed; the first area is an area whose interest value is less than a second preset value; determining a third area of ​​the image to be processed; the third area is an area whose interest value is less than or equal to the first preset value and greater than or equal to the second preset value.

[0103] The first area is a marginal area, the second area is a non-interest area, and the third area is a target area.

[0104] Exemplarily, the first preset value may be set to: min interest value + 4*(max interest value - min interest value) / 5. The second preset value may be set to: min interest value + 2*(max interest value - min interest value) / 5.

[0105] For different tasks to be processed, the maximum and minimum values ​​of the interest value may be different, resulting in different preset partition values. In addition, the maximum and minimum values ​​of the interest value may be the same. The coefficient in the calculation formula can be adjusted according to the type of task to be processed.

[0106] The calculation formula can be represented as: min interest value + a*(max interest value - min interest value) / b, and min interest value + c*(max interest value - min interest value) / d.

[0107] S604: Perform hierarchical compression processing on multiple regions at different levels to obtain a compressed image.

[0108] In a possible embodiment, the method of obtaining the compressed image may include: performing a first level compression process on a first region of the image to be processed, performing a second level compression process on a third region, and performing a third level compression process on the second region to obtain the compressed image. The first level compression process, the second level compression process, and the third level compression process, respectively, correspond to quantization parameters in ascending order.

[0109] In a possible embodiment, if the task to be processed is medical image analysis, the first level compression processing method can be used for the lesion part and directly retained, the second level compression processing method can be used for the edge area, and the third level compression processing method can be used for the area of ​​no interest.

[0110] In a possible embodiment, the quality of compression can be determined based on a quantization parameter. When a higher quality is used for compression, the quantization parameter is usually lower, which means a lighter degree of compression and a higher image quality. When a lower quality is used for compression, the quantization parameter is usually higher, which means a heavier degree of compression and a lower image quality.

[0111] The quantization parameter may be set to perform hierarchical compression processing on regions of different levels. For example, the quantization parameter may be set to 1 for the first region, 40 for the second region, and 20 for the third region.

[0112] In a possible embodiment, if for task A, the image to be processed is divided into 10 regions of different levels, then for the regions in the compressed image corresponding to the regions of different levels in the image to be processed (in the compressed image), 10 quantization parameters can be set to characterize different image qualities.

[0113] In a possible embodiment, for the first level compression processing of the first region, the quantization parameter may be set to 0, that is, a non-compression processing method is selected.

[0114] In a possible implementation, in the present disclosure, hierarchical compression processing is performed on the image to be processed. In the step of obtaining a compressed image, the hierarchical compression processing is a pre-compression process, and an image with image quality graded is obtained based on this process, which is then used for subsequent input into the intelligent task network to execute the task to be processed.

[0115] It may include: performing image encoding processing on the compressed image to obtain an encoded code stream file; transmitting the code stream file and performing image decoding processing to obtain a decoded image; and inputting the decoded image into the intelligent task network to execute the task.

[0116] Exemplarily, the image encoding process may use a Visual Tracking and Measurement (VTM) encoder to perform image encoding.

[0117] Exemplarily, the image decoding process may use a decoding scheme corresponding to the VTM encoder.

[0118] Exemplarily, the intelligent task network may adopt a Faster Region Convolutional Neural Network (Faster RCNN) model.

[0119] Figure 7 A flowchart of another image processing method is shown. Figure 7 As shown, the following steps are included:

[0120] S702: Acquire an image to be processed.

[0121] S704: Input the image to be processed into the residual network to obtain a feature map output by the second convolutional layer of the residual network.

[0122] S706: Input the feature map output by the second convolutional layer of the residual network into the second layer of the feature pyramid network to obtain an interest value feature map.

[0123] S708: Determine, according to the interest value feature graph and in accordance with preset object importance weighting, an interest value distribution graph after quantization of the interest value feature graph.

[0124] S710: Perform distribution map upsampling according to the interest value distribution map to determine the interest value distribution of the image to be processed.

[0125] S712: Determine a plurality of regions at different levels in the image to be processed according to the preset partition value and the interest value distribution.

[0126] S714: Perform hierarchical compression processing on multiple regions at different levels to obtain a compressed image. Different levels of compression processing correspond to different quantization parameters. For a first region whose interest value is greater than a first preset value, the quantization parameter is set to 0.

[0127] S716: Perform image encoding processing on the compressed image to obtain an encoded code stream file.

[0128] S718: Transmit the code stream file and perform image decoding processing to obtain a decoded image.

[0129] S720: Input the decoded image into the intelligent task network to execute the task to be processed.

[0130] Through the method disclosed in the present invention, the quantization parameter of the area corresponding to the important target that will definitely be used in the execution of the task can be set to 0 during the hierarchical compression process to ensure that there will be no information loss in the area corresponding to the important target. At the same time, information suppression can be performed on other areas to further compress the image information volume, reduce data transmission, improve the compression ratio of the image in the subsequent encoding process, reduce the information loss caused by the encoding and decoding process, and improve the performance in the subsequent execution of the tasks to be processed.

[0131] For specific tasks in different scenes, we can retain the important targets in the scene in a targeted manner, focus on the image semantic information in the scene, and reduce the amount of information in other areas by extracting interest values. While improving the compression ratio, the contrast between the target area and other areas is improved, with less loss in accuracy for subsequent intelligent tasks.

[0132] Based on the same inventive concept, the present disclosure also provides an image processing device in the following embodiments. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0133] Figure 8 A schematic diagram of the structure of an image processing device in an embodiment of the present disclosure is shown. Figure 8 As shown, it includes: an acquisition unit 801, used to acquire the image to be processed, an extraction unit 802, used to input the image to be processed into an interest value extraction network to obtain an interest value feature map, a determination unit 803, used to determine the interest value distribution of the image to be processed according to the interest value feature map, a compression unit 804, used to perform hierarchical compression processing on the image to be processed according to the interest value distribution to obtain a compressed image, wherein different levels of compression processing correspond to different quantization parameters, and a task execution unit 805, used to encode and decode the compressed image and input it into the intelligent task network to execute the task to be processed.

[0134] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0135] Refer to the following Fig. 9 The electronic device 900 according to this embodiment of the present disclosure is described. Fig. 9 The electronic device 900 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0136] like Fig. 9 As shown, the electronic device 900 is in the form of a general computing device. The components of the electronic device 900 may include but are not limited to: at least one processing unit 910, at least one storage unit 920, and a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910).

[0137] The storage unit stores program codes, which can be executed by the processing unit 910, so that the processing unit 910 performs the steps of various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 910 can perform any step of the above method embodiments.

[0138] The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 9201 and / or a cache storage unit 9202 , and may further include a read-only storage unit (ROM) 9203 .

[0139] The storage unit 920 may also include a program / utility 9204 having a set (at least one) of program modules 9205, such program modules 9205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0140] Bus 930 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0141] The electronic device 900 may also communicate with one or more external devices 940 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 900, and / or may communicate with any device that enables the electronic device 900 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 950. Furthermore, the electronic device 900 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 960. As shown, the network adapter 960 communicates with other modules of the electronic device 900 via a bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0142] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0143] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer program product or a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method in the above embodiment.

[0144] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the above method of the present disclosure is stored thereon. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above “Exemplary Method” section of this specification.

[0145] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0146] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0147] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0148] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0149] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0150] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0151] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0152] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. An image processing method, It is characterized in that The method comprises: Get the image to be processed; Inputting the image to be processed into an interest value extraction network to obtain an interest value feature map; Determining the interest value distribution of the image to be processed according to the interest value feature map; According to the distribution of the interest values, the image to be processed is subjected to hierarchical compression processing to obtain a compressed image; wherein different levels of compression processing correspond to different quantization parameters; The compressed image is encoded and decoded, and input into the intelligent task network to execute the task to be processed.

2. The method according to claim 1, It is characterized in that The interest value extraction network includes: a residual network and a feature pyramid network; wherein, starting from the second layer of the residual network, each layer is connected to each layer of the feature pyramid network; The step of inputting the image to be processed into an interest value extraction network model to obtain an interest value feature map includes: Inputting the image to be processed into a residual network to obtain a feature map output by the second convolutional layer of the residual network; The feature map output by the second convolutional layer of the residual network is input into the second layer of the feature pyramid network to obtain an interest value feature map.

3. The method according to claim 1, It is characterized in that The determining, according to the interest value feature map, the interest value distribution of the image to be processed includes: According to the interest value feature graph, determining an interest value distribution graph after quantization of the interest value feature graph according to a preset object importance weighting; The interest value distribution map is upsampled according to the interest value distribution map to determine the interest value distribution of the image to be processed.

4. The method according to claim 1, It is characterized in that According to the interest value distribution, the image to be processed is subjected to hierarchical compression processing to obtain a compressed image, including: Determining a plurality of regions at different levels in the image to be processed according to a preset partition value and the interest value distribution; Hierarchical compression processing is performed on multiple regions at different levels to obtain compressed images.

5. The method according to claim 4, It is characterized in that The step of determining a plurality of regions at different levels in the image to be processed according to the preset partition value and the interest value distribution includes: Determine a first region of the image to be processed; the first region is a region where the interest value is greater than a first preset value; Determine a second area of ​​the image to be processed; the first area is an area where the interest value is less than a second preset value; Determine a third area of ​​the image to be processed; the third area is an area whose interest value is less than or equal to the first preset value and greater than or equal to the second preset value.

6. The method according to claim 5, It is characterized in that The step of performing hierarchical compression processing on a plurality of regions at different levels to obtain a compressed image includes: Performing a first level compression process on a first region of the image to be processed, performing a second level compression process on the third region, and performing a third level compression process on the second region, to obtain the compressed image; Among them, the quantization parameters corresponding to the first level compression processing, the second level compression processing and the third level compression processing are sorted from small to large.

7. The method according to claim 1, It is characterized in that The encoding and decoding of the compressed image and inputting the compressed image into the intelligent task network to execute the task to be processed includes: Performing image encoding processing on the compressed image to obtain an encoded code stream file; The code stream file is transmitted, and image decoding is performed to obtain a decoded image; The decoded image is input into the intelligent task network to execute the task to be processed.

8. An image processing device, It is characterized in that include: An acquisition unit, used for acquiring an image to be processed; An extraction unit, used for inputting the image to be processed into an interest value extraction network to obtain an interest value feature map; A determining unit, configured to determine the interest value distribution of the image to be processed according to the interest value feature map; A compression unit, configured to perform hierarchical compression processing on the image to be processed according to the distribution of the interest values ​​to obtain a compressed image; wherein different levels of compression processing correspond to different quantization parameters; The task execution unit is used to encode and decode the compressed image and input it into the intelligent task network to execute the task to be processed.

9. An electronic device, It is characterized in that include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, include: A computer program or instruction, characterized in that when the computer program or instruction is executed by a processor, the method described in any one of claims 1 to 7 is implemented.

Images