Image processing method, apparatus, device, medium, and program product

Abstract

The present disclosure provides an image processing method, device, equipment, medium and program product, relating to the technical field of computer, which comprises: acquiring a to-be-processed image, inputting the to-be-processed image into an interest value extraction network to obtain an interest value feature map, determining an interest value distribution of the to-be-processed image according to the interest value feature map, performing hierarchical compression processing on the to-be-processed image according to the interest value distribution to obtain a compressed image, encoding and decoding the compressed image, and inputting the compressed image into an intelligent task network to execute a to-be-processed task. The image is subjected to hierarchical compression processing before encoding, which not only can reduce the information quantity of some regions in the to-be-processed image through pre-compression to improve the compression ratio, but also can improve the contrast between different regions of the to-be-processed image to improve the performance when executing the task.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an image processing method, apparatus, device, medium, and program product. Background Technology

[0002] To reduce the amount of data transmitted, images or videos need to be encoded, then the images are restored at the decoding end before subsequent intelligent tasks can be performed.

[0003] In related technologies, image encoding and decoding usually directly encode and decode the original image without considering the performance of the decoded image in subsequent intelligent task networks.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This disclosure provides an image processing method, apparatus, device, medium, and program product that at least to some extent reduces the amount of information in some areas of the image to be processed, increases the compression ratio, improves the contrast between different areas of the image to be processed, and enhances the performance when performing tasks.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] In a first aspect, embodiments of this disclosure provide an image processing method, the method comprising:

[0008] Obtain the image to be processed;

[0009] The image to be processed is input into the interest value extraction network to obtain the interest value feature map;

[0010] Based on the interest value feature map, determine the distribution of interest values ​​in the image to be processed;

[0011] Based on the distribution of interest values, the image to be processed is subjected to hierarchical compression processing to obtain a compressed image. The quantization parameters are different for different levels of compression processing.

[0012] The compressed image is encoded and decoded, and then input into the intelligent task network to perform the task to be processed.

[0013] In one possible embodiment, the interest value extraction network includes: a residual network and a feature pyramid network; wherein, starting from the second layer of the residual network, each layer is connected to each layer of the feature pyramid network;

[0014] The image to be processed is input into the interest value extraction network model to obtain the interest value feature map, including:

[0015] The image to be processed is input into the residual network to obtain the feature map output by the second convolutional layer of the residual network;

[0016] The feature map output from the second convolutional layer of the residual network is input into the second layer of the feature pyramid network to obtain the interest value feature map.

[0017] In one possible embodiment, determining the distribution of interest values ​​in the image to be processed based on the interest value feature map includes:

[0018] Based on the interest value feature map, the interest value distribution map after quantization is determined according to the pre-set object importance weights.

[0019] The distribution map is upsampled based on the interest value distribution map to determine the interest value distribution of the image to be processed.

[0020] In one possible embodiment, the image to be processed is subjected to hierarchical compression processing according to the interest value distribution to obtain a compressed image, including:

[0021] Based on the preset partition values ​​and interest value distribution, multiple regions at different levels in the image to be processed are determined;

[0022] Hierarchical compression processing is performed on multiple regions at different levels to obtain compressed images.

[0023] In one possible embodiment, multiple regions at different levels in the image to be processed are determined based on preset partition values ​​and the distribution of interest values, including:

[0024] Determine the first region of the image to be processed; the first region is the region where the interest value is greater than a first preset value.

[0025] Determine the second region of the image to be processed; the first region is the region where the interest value is less than the second preset value.

[0026] Determine the third region of the image to be processed; the third region is the region where the interest value is less than or equal to the first preset value and greater than or equal to the second preset value.

[0027] In one possible embodiment, hierarchical compression processing is performed on multiple regions at different levels to obtain a compressed image, including:

[0028] The first region of the image to be processed is subjected to first-level compression, the third region is subjected to second-level compression, and the second region is subjected to third-level compression to obtain a compressed image.

[0029] The quantization parameters for the first-level compression, second-level compression, and third-level compression are ordered from smallest to largest.

[0030] In one possible embodiment, the compressed image is encoded and decoded, and then input into an intelligent task network to perform a processing task, including:

[0031] The compressed image is processed by image encoding to obtain the encoded bitstream file;

[0032] The bitstream file is transmitted and image decoding is performed to obtain the decoded image;

[0033] The decoded image is input into the intelligent task network to perform the task to be processed.

[0034] Secondly, embodiments of this disclosure provide an image processing apparatus, including:

[0035] The acquisition unit is used to acquire the image to be processed;

[0036] The extraction unit is used to input the image to be processed into the interest value extraction network to obtain the interest value feature map;

[0037] The determining unit is used to determine the distribution of interest values ​​in the image to be processed based on the interest value feature map.

[0038] The compression unit is used to perform hierarchical compression processing on the image to be processed according to the distribution of interest values ​​to obtain a compressed image. The quantization parameters are different for different levels of compression processing.

[0039] The task execution unit is used to encode and decode compressed images and input them into the intelligent task network to perform the tasks to be processed.

[0040] Thirdly, embodiments of this disclosure provide an electronic device, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the method of the first aspect described above by executing the executable instructions.

[0041] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0042] Fifthly, according to another aspect of this disclosure, a computer program product or computer program is also provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform any of the methods described above.

[0043] This disclosure provides an image processing method, apparatus, device, medium, and program product. The method includes: acquiring an image to be processed; inputting the image to be processed into an interest value extraction network to obtain an interest value feature map; determining the interest value distribution of the image to be processed based on the interest value feature map; performing hierarchical compression processing on the image to be processed based on the interest value distribution to obtain a compressed image; encoding and decoding the compressed image; and inputting it into an intelligent task network to execute a processing task. By performing hierarchical compression processing on the image before encoding, this method can not only improve the compression ratio and efficiency and reduce the amount of data transmission through pre-compression, but also reduce the amount of information in some areas of the image to be processed, thereby improving the contrast between different areas of the image to be processed.

[0044] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0045] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0046] Figure 1 A flowchart illustrating an image processing method according to an embodiment of this disclosure is shown;

[0047] Figure 2 This diagram illustrates the structure of a residual network and a feature pyramid network according to an embodiment of the present disclosure.

[0048] Figure 3 This diagram illustrates a flowchart of obtaining an interest value feature map according to an embodiment of the present disclosure;

[0049] Figure 4 This illustration shows a structural diagram of another residual network and feature pyramid network in an embodiment of this disclosure;

[0050] Figure 5 This diagram illustrates a flowchart of determining the distribution of interest values ​​in an image to be processed, according to an embodiment of the present disclosure.

[0051] Figure 6 This diagram illustrates a flowchart of obtaining a compressed image according to an embodiment of the present disclosure;

[0052] Figure 7 A flowchart illustrating another image processing method in an embodiment of this disclosure is shown;

[0053] Figure 8This diagram illustrates the structure of an image processing apparatus according to an embodiment of the present disclosure.

[0054] Figure 9 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0055] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0056] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0057] Because in related technologies, image encoding and decoding usually directly encode and decode the original image without considering the performance of the decoded image in subsequent intelligent task networks, some specific intelligent task networks perform poor tasks when processing images in the above manner.

[0058] This disclosure provides an image processing method, apparatus, device, medium, and program product. The method includes: acquiring an image to be processed; inputting the image to be processed into an interest value extraction network to obtain an interest value feature map; determining the interest value distribution of the image to be processed based on the interest value feature map; performing hierarchical compression processing on the image to be processed based on the interest value distribution to obtain a compressed image; encoding and decoding the compressed image; and inputting it into an intelligent task network to execute a processing task. By performing hierarchical compression processing on the image before encoding, this method not only reduces the amount of information in some areas of the image to be processed through pre-compression, improving compression efficiency, reducing data transmission volume, and increasing the compression ratio, but also improves the contrast between different areas of the image to be processed, with minimal loss of accuracy in subsequent intelligent tasks.

[0059] The following detailed description of this exemplary implementation method is provided in conjunction with the accompanying drawings and embodiments.

[0060] First, this disclosure provides an image processing method that can be executed by any electronic device with computing capabilities.

[0061] Figure 1 A flowchart of an image processing method according to an embodiment of this disclosure is shown, such as... Figure 1 As shown, the method provided in this embodiment includes the following steps:

[0062] S102: Obtain the image to be processed.

[0063] The images in this disclosure can be single images or images captured from videos. Based on this, it can be understood that the image processing methods in this disclosure can also be used to compress videos.

[0064] For example, the input image to be processed is used to perform the task. The input image to be processed can be used for a visual recognition algorithm, fed into a visual algorithm model for visual recognition, and the processing result is output.

[0065] S104: Input the image to be processed into the interest value extraction network to obtain the interest value feature map.

[0066] In one possible embodiment, the image to be processed is extracted to obtain a feature map that can be used to analyze interest values. In order to ensure the accuracy of subsequent interest value analysis, the interest value feature map can be a high-resolution feature map.

[0067] S106: Determine the distribution of interest values ​​in the image to be processed based on the interest value feature map.

[0068] Interest value is a quantified value. In image processing and computer vision, quantified interest values ​​represent the different meanings and contents contained in different points or regions of an image.

[0069] S108: Based on the distribution of interest values, perform hierarchical compression processing on the image to be processed to obtain a compressed image. Different levels of compression processing correspond to different quantization parameters.

[0070] S110: Encodes and decodes the compressed image and inputs it into the intelligent task network to perform the task to be processed.

[0071] In one possible embodiment, the image to be processed is compressed by hierarchical compression based on the distribution of interest values. This can reduce the amount of information in some areas of the image to be processed, improve the contrast between different areas of the image to be processed, and use the compressed image in subsequent intelligent tasks. Furthermore, pre-compression can improve the compression ratio in subsequent encoding and decoding processes.

[0072] In one possible embodiment, the image compression process in the hierarchical compression process can use an end-to-end compression method or a traditional image compression method such as VTM, without limitation.

[0073] It should be noted that the hierarchical compression processing in this disclosure refers to different levels of compression processing with different quantization parameters (QP). The highest level of compression processing is no compression, which corresponds to the highest image quality, and the quantization parameter is 0.

[0074] In one possible embodiment, the interest value extraction network in this disclosure can be any network capable of extracting high-resolution feature maps. In the following embodiments, the interest value extraction network is described using examples including a Residual Network (ResNet) and a Feature Pyramid Network (FPN). The Residual Network can be a 50-layer ResNet50.

[0075] in, Figure 2 A schematic diagram of a residual network and a feature pyramid network is shown; as follows: Figure 2 As shown, starting from the second layer of the residual network, each layer is connected to each layer of the feature pyramid network. That is, the second layer of the residual network is connected to the first layer of the feature pyramid network, the third layer of the residual network is connected to the second layer of the feature pyramid network, and so on.

[0076] By utilizing the structures of residual networks and feature pyramid networks, interest value feature maps can be obtained. Figure 3 A flowchart for obtaining an interest value feature map is shown, such as... Figure 3 As shown, it includes the following steps:

[0077] S302: Input the image to be processed into the deep residual network to obtain the feature map output by the second convolutional layer of the deep residual network.

[0078] S304: Input the feature map output from the second convolutional layer of the deep residual network into the second layer of the feature pyramid network to obtain the interest value feature map.

[0079] In one possible embodiment, the feature maps processed by the residual network and feature pyramid network can obtain better and richer contextual semantic information as the number of layers increases, but the resolution will decrease. In order to accurately identify each region in the interest value feature map and determine the interest value in subsequent processing, a high-resolution feature map is required. The output feature of the second layer, namely the P2 layer, is selected as the interest value feature map.

[0080] In one possible implementation, if there are tasks in some intelligent task networks that require more attention to semantic information, then deep feature maps can be used.

[0081] In one possible embodiment, Figure 4 The diagram shows the structural schematics of the residual network and the feature pyramid network; as shown. Figure 4 As shown, Figure 4 Taking the ResNet50+FPN structure as an example, and in Figure 4 The diagram shows the first 5 layers of the ResNet50 network, the first 4 layers of the FPN network, and the specific connection relationships between ResNet50 and FPN.

[0082] like Figure 4 As shown, the input image to be processed is 640×640×3 pixels. It is first input into ResNet50 for processing. For details of the processing, please refer to [link / reference]. Figure 4 This will not be elaborated upon further.

[0083] The feature map output from Stage 1 is input into the first layer of the FPN network to obtain a 160×160×256 interest value feature map.

[0084] In one possible embodiment, Figure 5 A flowchart for determining the distribution of interest values ​​in an image to be processed is shown, such as... Figure 5 As shown, it includes the following steps:

[0085] S502: Based on the interest value feature map, determine the interest value distribution map after quantization of the interest value feature map by weighting according to the pre-set object importance.

[0086] In one possible embodiment, the pre-set object importance weighting can be set for different objects.

[0087] Based on the different tasks to be processed in the intelligent task network, different objects are pre-weighted according to their importance.

[0088] The pre-set methods may include: using automatic image recognition and semantic information analysis to identify objects that are related to the task to be processed, and automatically assigning object importance weights to these objects. For example, if the task to be processed is visual recognition for autonomous driving or intelligent traffic control, then semantic information analysis is used to determine the objects associated with this type of task, and automatic image recognition is used to identify the objects in the image, automatically assigning object importance weights to these objects.

[0089] For example, the task to be processed is a visual recognition task for autonomous driving or an intelligent traffic management task, and importance weights can be set for objects such as people and vehicles.

[0090] For example, if the task to be processed is the identification task of the guide robot, the target area is the road and distinctive road signs, and importance weights are set for the road and / or road signs.

[0091] For example, if the task to be processed is medical image analysis, the target region is the lesion to be identified, and importance weighting is set for the lesion.

[0092] The distribution map of interest values ​​after quantization is determined by calculating the weighted average of the importance of objects in a pre-defined manner.

[0093] S504: Perform upsampling of the interest value distribution map based on the interest value distribution map to determine the interest value distribution of the image to be processed.

[0094] In one possible embodiment, after obtaining the interest value distribution map, since it is the interest value distribution of the interest value feature map, the original image, i.e. the image to be processed, needs to be encoded and decoded during the encoding and decoding process. Therefore, it is necessary to perform upsampling of the distribution map to determine the interest value distribution of the image to be processed.

[0095] In one possible embodiment, Figure 6 A flowchart for obtaining a compressed image is shown, such as... Figure 6 As shown, it includes the following steps:

[0096] S602: Based on the preset partition values ​​and interest value distribution, determine multiple regions in the image to be processed that are at different levels.

[0097] In one possible embodiment, the interest values ​​of different points or regions in the image to be processed can be determined based on the distribution of interest values ​​in the image to be processed. A preset partition value is used to compare with the interest values ​​of different points or regions in the image to be processed, thereby dividing the image to be processed into multiple regions of different levels.

[0098] In one possible embodiment, preset partition values ​​can be set according to the task to be performed on the image to be processed.

[0099] For example, if the task to be processed is a visual recognition or intelligent traffic control task for autonomous driving, two preset partition values ​​can be set, and the specific values ​​of the preset partition values ​​can be set to A and B.

[0100] If the task to be processed is the recognition task of the guide robot, one preset partition value can be set, and the specific value of the preset partition value is set to C.

[0101] If the task to be processed is medical image analysis, two preset partition values ​​can be set, and the specific values ​​of the preset partition values ​​can be set to D and E.

[0102] In one possible embodiment, determining multiple regions at different levels may include: determining a first region of the image to be processed; the first region being a region with an interest value greater than a first preset value; determining a second region of the image to be processed; the first region being a region with an interest value less than a second preset value; determining a third region of the image to be processed; the third region being a region with an interest value less than or equal to the first preset value and greater than or equal to the second preset value.

[0103] The first region is the edge region, the second region is the region of no interest, and the third region is the target region.

[0104] For example, the first preset value can be set to: min interest value + 4 * (max interest value - min interest value) / 5. The second preset value can be set to: min interest value + 2 * (max interest value - min interest value) / 5.

[0105] When dealing with different tasks, the maximum and minimum values ​​of interest may differ, resulting in different preset partition values. In addition, the maximum and minimum values ​​of interest may be the same, and the coefficients in the calculation formula can be adjusted according to the type of task to be processed.

[0106] The calculation formula can be represented as: min interest value + a*(max interest value - min interest value) / b, and min interest value + c*(max interest value - min interest value) / d.

[0107] S604: Perform hierarchical compression processing on multiple regions at different levels to obtain a compressed image.

[0108] In one possible embodiment, obtaining the compressed image may include: performing a first-level compression process on a first region of the image to be processed, performing a second-level compression process on a third region, and performing a third-level compression process on a second region to obtain the compressed image. The quantization parameters corresponding to the first-level compression process, the second-level compression process, and the third-level compression process are ordered from smallest to largest.

[0109] In one possible embodiment, if the task to be processed is medical image analysis, the lesion area can be directly preserved by using the first level of compression processing, the edge area can be compressed by the second level of compression processing, and the area of ​​no interest can be compressed by the third level of compression processing.

[0110] In one possible embodiment, the quality of compression can be determined based on quantization parameters. When using higher quality compression, the quantization parameters are usually lower, meaning lighter compression and higher image quality; while when using lower quality compression, the quantization parameters are usually higher, indicating heavier compression and lower image quality.

[0111] By setting quantization parameters, hierarchical compression processing can be performed for different levels of regions. For example, the quantization parameter can be set to 1 for the first region, 40 for the second region, and 20 for the third region.

[0112] In one possible embodiment, if for task A, the image to be processed is divided into 10 different levels of regions, then for the regions in the compressed image that correspond to the different levels of regions in the image to be processed (in the compressed image), 10 quantization parameters can be set to characterize different image qualities.

[0113] In one possible embodiment, the method of performing first-level compression processing on the first region can be achieved by setting the quantization parameter to 0, i.e., selecting a non-compression processing method.

[0114] In one possible implementation, the step of performing hierarchical compression processing on the image to be processed in this disclosure to obtain a compressed image is a pre-compression process. Based on this process, an image with image quality grade is obtained, which is then used as input to the intelligent task network to perform the task to be processed.

[0115] It can include: performing image encoding processing on the compressed image to obtain an encoded bitstream file; transmitting the bitstream file and performing image decoding processing to obtain a decoded image; and inputting the decoded image into the intelligent task network to execute the task.

[0116] For example, image encoding processing can be performed using a visual tracking and measurement (VTM) encoder.

[0117] For example, image decoding processing can use the decoding scheme corresponding to the VTM encoder.

[0118] For example, the intelligent task network can employ a Faster Region Convolutional Neural Network (Faster RCNN) model.

[0119] Figure 7 A flowchart of another image processing method is shown, such as Figure 7 As shown, it includes the following steps:

[0120] S702: Obtain the image to be processed.

[0121] S704: Input the image to be processed into the residual network to obtain the feature map output by the second convolutional layer of the residual network.

[0122] S706: Input the feature map output from the second convolutional layer of the residual network into the second layer of the feature pyramid network to obtain the interest value feature map.

[0123] S708: Based on the interest value feature map, determine the interest value distribution map after quantization of the interest value feature map by weighting according to the pre-set object importance.

[0124] S710: Perform upsampling of the interest value distribution map based on the interest value distribution map to determine the interest value distribution of the image to be processed.

[0125] S712: Based on the preset partition values ​​and interest value distribution, determine multiple regions at different levels in the image to be processed.

[0126] S714: Perform hierarchical compression processing on multiple regions at different levels to obtain a compressed image. The quantization parameters are different for different levels of compression processing. The quantization parameter is set to 0 for the first region where the interest value is greater than the first preset value.

[0127] S716: Performs image encoding processing on the compressed image to obtain the encoded bitstream file.

[0128] S718: Transmits the bitstream file and performs image decoding to obtain the decoded image.

[0129] S720: Inputs the decoded image into the intelligent task network to perform the task to be processed.

[0130] The method disclosed herein allows for the setting of quantization parameters to 0 in the hierarchical compression process of regions corresponding to important targets that will inevitably be used in the execution of tasks. This ensures that there is no information loss in the regions corresponding to important targets, while also suppressing information in other regions. This further compresses the amount of image information, reduces data transmission, improves the compression ratio of the image in subsequent encoding processes, reduces information loss caused by encoding and decoding processes, and improves performance in subsequent tasks.

[0131] For specific tasks in different scenarios, important targets in that scenario can be selectively preserved, focusing on the semantic information of the image in that scenario. By extracting interest values, the amount of information in other areas can be reduced, improving the compression ratio while enhancing the contrast between the target area and other areas, with minimal loss of accuracy for subsequent intelligent tasks.

[0132] Based on the same inventive concept, this disclosure also provides an image processing apparatus, as shown in the following embodiments. Since the principle by which this apparatus solves the problem is similar to that of the method embodiments described above, the implementation of this apparatus embodiment can refer to the implementation of the method embodiments described above, and repeated details will not be described again.

[0133] Figure 8 This diagram illustrates the structure of an image processing apparatus according to an embodiment of the present disclosure, such as... Figure 8 As shown, it includes: an acquisition unit 801 for acquiring an image to be processed; an extraction unit 802 for inputting the image to be processed into an interest value extraction network to obtain an interest value feature map; a determination unit 803 for determining the interest value distribution of the image to be processed based on the interest value feature map; a compression unit 804 for performing hierarchical compression processing on the image to be processed based on the interest value distribution to obtain a compressed image, wherein different levels of compression processing correspond to different quantization parameters; and a task execution unit 805 for encoding and decoding the compressed image and inputting it into an intelligent task network to execute the task to be processed.

[0134] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0135] The following reference Figure 9 To describe an electronic device 900 according to such an embodiment of the present disclosure. Figure 9 The electronic device 900 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0136] like Figure 9 As shown, the electronic device 900 is manifested in the form of a general-purpose computing device. The components of the electronic device 900 may include, but are not limited to: at least one processing unit 910, at least one storage unit 920, and a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910).

[0137] The storage unit stores program code that can be executed by the processing unit 910, causing the processing unit 910 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 910 can perform the steps of any of the above method embodiments.

[0138] Storage unit 920 may include readable media in the form of volatile storage units, such as random access memory (RAM) 9201 and / or cache memory 9202, and may further include read-only memory (ROM) 9203.

[0139] Storage unit 920 may also include a program / utility 9204 having a set (at least one) program module 9205, such program module 9205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0140] Bus 930 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0141] Electronic device 900 can also communicate with one or more external devices 940 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 900, and / or with any device that enables electronic device 900 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 950. Furthermore, electronic device 900 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 960. As shown, network adapter 960 communicates with other modules of electronic device 900 via bus 930. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0142] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0143] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods described above.

[0144] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the methods described above is stored thereon. In some possible implementations, various aspects of this disclosure may also be implemented as a program product including program code, which, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0145] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0146] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0147] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0148] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0149] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0150] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0151] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0152] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: Obtain the image to be processed; The image to be processed is input into the interest value extraction network to obtain the interest value feature map; Based on the interest value feature map, determine the interest value distribution of the image to be processed; Based on the interest value distribution, the image to be processed is subjected to hierarchical compression processing to obtain a compressed image; wherein, different levels of compression processing correspond to different quantization parameters; the higher the level of hierarchical compression, the smaller the quantization parameter, and the higher the quality of the compressed image; The compressed image is encoded and decoded, and then input into the intelligent task network to perform the task to be processed; Based on the interest value distribution, the image to be processed is subjected to hierarchical compression processing to obtain a compressed image, including: Based on the preset partition values ​​and the interest value distribution, multiple regions at different levels in the image to be processed are determined; the preset partition values ​​are determined based on the minimum and maximum interest values ​​in the interest value distribution. Hierarchical compression processing is performed on multiple regions at different levels to obtain a compressed image; The step of determining the interest value distribution of the image to be processed based on the interest value feature map includes: determining the interest value distribution map after quantization of the interest value feature map by weighting according to a pre-set object importance based on the interest value feature map; and determining the interest value distribution of the image to be processed by upsampling the distribution map based on the interest value distribution map.

2. The method according to claim 1, characterized in that, The interest value extraction network includes a residual network and a feature pyramid network; wherein, starting from the second layer of the residual network, each layer is connected to each layer of the feature pyramid network; The step of inputting the image to be processed into the interest value extraction network model to obtain the interest value feature map includes: The image to be processed is input into the residual network to obtain the feature map output by the second convolutional layer of the residual network; The feature map output from the second convolutional layer of the residual network is input into the first layer of the feature pyramid network to obtain the interest value feature map.

3. The method according to claim 1, characterized in that, The step of determining multiple regions at different levels in the image to be processed based on preset partition values ​​and the distribution of interest values ​​includes: A first region of the image to be processed is determined; the first region is a region whose interest value is greater than a first preset value. A second region of the image to be processed is determined; the first region is a region whose interest value is less than a second preset value. A third region of the image to be processed is determined; the third region is a region whose interest value is less than or equal to the first preset value and greater than or equal to the second preset value.

4. The method according to claim 3, characterized in that, The step of performing hierarchical compression processing on multiple regions at different levels to obtain a compressed image includes: The first region of the image to be processed is subjected to a first level of compression, the third region is subjected to a second level of compression, and the second region is subjected to a third level of compression to obtain the compressed image; The quantization parameters for the first level of compression, the second level of compression, and the third level of compression are ordered from smallest to largest.

5. The method according to claim 1, characterized in that, The process of encoding and decoding the compressed image and inputting it into the intelligent task network to perform the task to be processed includes: The compressed image is subjected to image encoding processing to obtain an encoded bitstream file; The bitstream file is transmitted and image decoding is performed to obtain a decoded image; The decoded image is input into the intelligent task network to execute the task to be processed.

6. An image processing apparatus, characterized in that, include: The acquisition unit is used to acquire the image to be processed; An extraction unit is used to input the image to be processed into an interest value extraction network to obtain an interest value feature map; The determining unit is configured to determine the interest value distribution of the image to be processed based on the interest value feature map. The compression unit is used to perform hierarchical compression processing on the image to be processed according to the interest value distribution to obtain a compressed image; wherein, the quantization parameters are different for different levels of compression processing; the higher the level of hierarchical compression, the smaller the quantization parameter, and the higher the quality of the compressed image; The task execution unit is used to encode and decode the compressed image and input it into the intelligent task network to execute the task to be processed. The compression unit is further configured to determine multiple regions at different levels in the image to be processed based on preset partition values ​​and the interest value distribution; the preset partition values ​​are determined based on the minimum and maximum interest values ​​in the interest value distribution; and perform hierarchical compression processing on the multiple regions at different levels to obtain a compressed image. The determining unit is further configured to determine the interest value distribution map after quantization of the interest value feature map based on the interest value feature map and weighted according to the pre-set object importance; and to determine the interest value distribution of the image to be processed by performing distribution map upsampling based on the interest value distribution map.

7. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 5 by executing the executable instructions.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method described in any one of claims 1 to 5.

9. A computer program product, comprising: A computer program or instruction, characterized in that, when executed by a processor, the computer program or instruction implements the method described in any one of claims 1 to 5.

Images