Image processing method and device, electronic equipment and readable storage medium

Abstract

The invention relates to an image processing method and device, electronic equipment and a readable storage medium. The method comprises the following steps: acquiring a to-be-processed image; performing enhancement processing on a to-be-processed area in the to-be-processed image to obtain a first processed image; performing enhancement processing on the to-be-processed image to obtain a second processed image; determining a mask corresponding to the to-be-processed image; each mask value in the mask is used for representing a fusion weight corresponding to a pixel value in the first processing image or the second processing image; and based on the mask, fusing the first processing image and the second processing image to obtain a target image. By adopting the method, the image definition can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, electronic device and readable storage medium. Background Art

[0002] Image super-resolution refers to a technology that improves image resolution through hardware or software algorithms. Image super-resolution can effectively improve the clarity of images and is widely used in security monitoring, medical imaging, mobile phone photography and other fields.

[0003] In traditional technology, the resolution of an image is improved by using an interpolation method. For low-quality images, the interpolation method has poor deblurring ability and limited detail enhancement, resulting in low clarity of the image processed by using the interpolation method. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, device, electronic device and readable storage medium, which can improve the clarity of an image.

[0005] In a first aspect, the present application provides an image processing method, comprising:

[0006] Get the image to be processed;

[0007] Performing enhancement processing on the to-be-processed area in the to-be-processed image to obtain a first processed image;

[0008] Performing enhancement processing on the image to be processed to obtain a second processed image;

[0009] Determining a mask corresponding to the image to be processed; each mask value in the mask is used to represent a fusion weight corresponding to a pixel value in the first processed image or the second processed image;

[0010] Based on the mask, the first processed image and the second processed image are fused to obtain a target image.

[0011] In a second aspect, the present application further provides an image processing device, comprising:

[0012] An acquisition module, used for acquiring an image to be processed;

[0013] A local processing module, used for performing enhancement processing on the to-be-processed area in the to-be-processed image to obtain a first processed image;

[0014] A global processing module, used for performing enhancement processing on the image to be processed to obtain a second processed image;

[0015] A mask determination module, used to determine a mask corresponding to the image to be processed; each mask value in the mask is used to represent a fusion weight corresponding to a pixel value in the first processed image or the second processed image;

[0016] A fusion module is used to fuse the first processed image and the second processed image based on the mask to obtain a target image.

[0017] In a third aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0018] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0019] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0020] The above-mentioned image processing method, device, electronic device and readable storage medium enhance the area to be processed in the image to be processed to obtain a first processed image, enhance the image to be processed to obtain a second processed image, determine the mask corresponding to the image to be processed, that is, perform local enhancement processing on the area to be processed in the image to be processed, and the effect of enhancing the area to be processed is better. Therefore, the quality of the image data corresponding to the area to be processed in the first processed image is higher, and the global enhancement processing is performed on the image to be processed. Compared with the local enhancement processing on the area to be processed, the quality of the image data corresponding to the area to be processed in the second processed image is lower, and the mask value in the mask corresponding to the image to be processed represents whether the pixel point is located in the area to be processed. Then, according to the mask corresponding to the image to be processed, the image data corresponding to the area to be processed in the first processed image and the image data corresponding to the non-area to be processed in the second processed image are fused into a target image. The quality of the image data corresponding to the area to be processed and the image data corresponding to the non-area to be processed in the target image are both relatively high, thereby improving the clarity of the target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 is a schematic flow chart of an image processing method in one embodiment;

[0023] Figure 2A A schematic diagram of a model management system in an embodiment;

[0024] Figure 2B A schematic diagram of a model management system in another embodiment;

[0025] Figure 3 A schematic diagram of a model selection interface in an embodiment;

[0026] Figure 4 A schematic diagram of a third neural network model testing process in one embodiment;

[0027] Figure 5 A schematic diagram of a process of detecting a face area, a text area and a small structure area by a third neural network model in one embodiment;

[0028] Figure 6 A schematic diagram of an image processing flow in one embodiment;

[0029] Figure 7 is a schematic diagram of a target image and an enhanced image in one embodiment;

[0030] Figure 8 is a schematic diagram of a target image and an enhanced image in another embodiment;

[0031] Fig. 9 is a schematic diagram of a target image and an enhanced image in another embodiment;

[0032] Fig.10 is a schematic diagram of a target image and an enhanced image in another embodiment;

[0033] Fig.11 is a schematic diagram of a target image and an enhanced image in another embodiment;

[0034] Fig.12 is a schematic diagram of a target image and an enhanced image in another embodiment;

[0035] Fig.13 is a structural block diagram of an image processing device in one embodiment;

[0036] Fig.14 FIG. 4 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0038] In some exemplary embodiments, Figure 1 As shown, an image display method is provided. This embodiment takes the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes steps 102 to 110, wherein:

[0039] Step 102: Obtain an image to be processed.

[0040] The image to be processed refers to an image that needs to be enhanced. The image to be processed may be a low-resolution image or a low-quality image. The low-quality image may be a low-brightness image, a blurred image or a blurred image.

[0041] Exemplarily, the electronic device acquires an image to be processed.

[0042] In some exemplary embodiments, the electronic device obtains the image to be processed by photographing with a camera.

[0043] In some exemplary embodiments, the electronic device receives an image to be processed sent by a target device, where the target device refers to a device that sends the image to be processed, and the target device may be a camera, a computer device, or the like.

[0044] Step 104 , performing enhancement processing on the to-be-processed region in the to-be-processed image to obtain a first processed image.

[0045] Among them, the area to be processed refers to the target area in the image to be processed. The area to be processed can be a pre-set area. The area to be processed can include but is not limited to the face area, text area and small structure area, etc. The small structure area refers to the part of the image that contains fine details and small-scale features, such as texture, QR code, small pattern or complex edge, etc. The area to be processed refers to the area where the first neural network model performs enhancement processing or the area where the first neural network model performs enhancement processing with better effect, and the area where the second neural network model performs enhancement processing with poor effect. The area to be processed can be set according to the effect of the enhancement processing performed by the second neural network model. For example, the area to be processed can be the area where the second neural network model performs enhancement processing with poor effect.

[0046] The first neural network model may refer to a neural network model that performs enhancement processing on the area to be processed, or a neural network model that performs better enhancement processing on the area to be processed. The first neural network model may be an image super resolution (SR) model. The first neural network model is mainly trained on the area to be processed, and can effectively improve the detail effect and clarity of the area to be processed. For example, the first neural network model is an SR model trained by Generative Adversarial Networks (GAN) technology, and mainly performs enhancement processing on face areas, text areas, and small structure areas.

[0047] The second neural network model may refer to a neural network model that performs global enhancement processing on the processed image. The second neural network model may be a Stable Diffusion-based SuperResolution (SDSR) model, and the second neural network model may be a pre-trained SDSR model. Since the pre-trained SDSR model includes a variational autoencoder (VAE), the variational autoencoder performs three downsampling operations on the input data, destroying the information of many small structure areas in the input data, resulting in poor enhancement effect in the small structure areas, and the original training data set of the pre-trained SDSR model lacks various text scene data, resulting in poor enhancement effect in the text area, and the faces in the training samples in the original training data set of the pre-trained SDSR model have regional differences, resulting in poor enhancement effect in the face area, that is, the second neural network model is an SDSR model with poor enhancement effect on the face area, the text area and the small structure area.

[0048] Enhancement processing may refer to a process of improving the quality of the image to be processed, and the enhancement processing includes but is not limited to at least one of resolution enhancement processing, brightness enhancement processing, and color enhancement processing. The first processed image refers to an image obtained by performing enhancement processing on the area to be processed in the image to be processed, which can be understood as an image in which the area to be processed in the image to be processed is enhanced and the enhancement effect of the area to be processed is good.

[0049] Exemplarily, the electronic device determines a region to be processed in the image to be processed, performs enhancement processing on the region to be processed in the image to be processed, and obtains a first processed image. The enhancement processing on the region to be processed in the image to be processed can be implemented by hardware or software.

[0050] In some exemplary embodiments, the electronic device determines a to-be-processed area in the to-be-processed image, performs a first intensity enhancement process on the to-be-processed area in the to-be-processed image, and performs a second intensity enhancement process on the non-to-be-processed area in the to-be-processed image, to obtain a first processed image, wherein the first intensity is greater than the second intensity.

[0051] Step 106: Perform enhancement processing on the image to be processed to obtain a second processed image.

[0052] The second processed image refers to an image obtained by performing global enhancement processing on the image to be processed.

[0053] Exemplarily, the electronic device globally enhances the image to be processed to obtain a second processed image. Globally enhancing the image to be processed can be implemented by hardware or software.

[0054] Step 108, determining a mask corresponding to the image to be processed; each mask value in the mask is used to represent a fusion weight corresponding to a pixel value in the first processed image or the second processed image.

[0055] Among them, the mask refers to a set of mask values ​​corresponding to the pixels in the image to be processed, and the mask value refers to an identifier representing the fusion weight of the pixel. It can be understood that the mask value in the pixel in the mask represents whether the pixel is located in the area to be processed. For example, the mask value is 1 or 0. If the mask value of the pixel is 1, it represents that the pixel is located in the area to be processed, the fusion weight corresponding to the first pixel value of the pixel in the first processed image is large, and the fusion weight corresponding to the second pixel value of the pixel in the second processed image is small; if the mask value of the pixel is 0, it represents that the pixel is located in the non-area to be processed, the fusion weight corresponding to the first pixel value of the pixel in the first processed image is small, and the fusion weight corresponding to the second pixel value of the pixel in the second processed image is large. Fusion weight refers to the fusion coefficient.

[0056] Exemplarily, the electronic device determines the mask value of the pixel point in the to-be-processed area in the to-be-processed image as a first identifier, determines the mask value of the pixel point in the non-to-be-processed area as a second identifier, and obtains the mask corresponding to the to-be-processed image based on the mask value corresponding to each pixel point in the to-be-processed image. The first identifier indicates that the pixel point is located in the to-be-processed area, and the second identifier indicates that the pixel point is located in the non-to-be-processed area.

[0057] In some exemplary embodiments, after determining the mask corresponding to the image to be processed, the method further includes: for each pixel in the image to be processed, based on the mask value corresponding to the pixel in the mask, and based on the mask value, determining the first fusion weight and the second fusion weight corresponding to the pixel. The mask value may be a first identifier or a second identifier, the first identifier indicates that the pixel is located in the area to be processed, and the second identifier indicates that the pixel is located in the non-area to be processed. When the first identifier is 1 and the second identifier is 0, the first fusion weight is proportional to the mask value, and the second fusion weight is inversely proportional to the mask value.

[0058] In some exemplary embodiments, after determining the first fusion weight and the second fusion weight corresponding to the pixel point, it also includes: obtaining the first pixel value corresponding to the pixel point in the first processed image, obtaining the second pixel value corresponding to the pixel point in the second processed image, and adding the product of the first pixel value and the first fusion weight to the product of the second pixel value and the second fusion weight to obtain the target pixel value corresponding to the pixel point.

[0059] Step 110: Based on the mask, the first processed image and the second processed image are fused to obtain a target image.

[0060] Exemplarily, for each pixel in the image to be processed, the mask value corresponding to the pixel in the mask is obtained, the first pixel value corresponding to the pixel in the first processed image is obtained, and the second pixel value corresponding to the pixel in the second processed image is obtained. Based on the mask value, the first pixel value, and the second pixel value corresponding to the pixel, the target pixel value corresponding to the pixel is determined, and based on the target pixel value corresponding to each pixel in the image to be processed, the target image is obtained.

[0061] In some exemplary embodiments, when the mask value is the first identifier or the second identifier, the first identifier indicates that the pixel point is located in the area to be processed, the second identifier indicates that the pixel point is located in the non-area to be processed, the first identifier is 1, and the second identifier is 0, based on the mask value, the first pixel value, and the second pixel value corresponding to the pixel point, determining the target pixel value corresponding to the pixel point includes: multiplying the first pixel value by the mask value to obtain a first product; multiplying the difference between one and the mask value by the second pixel value to obtain a second product; and adding the first product to the second product to obtain the target pixel value corresponding to the pixel point. That is, the target pixel value = the first pixel value * the mask value + the second pixel value * (1-mask value).

[0062] In this embodiment, the first processed image is obtained by performing enhancement processing on the to-be-processed area in the to-be-processed image, the second processed image is obtained by performing enhancement processing on the to-be-processed image, and the mask corresponding to the to-be-processed image is determined, that is, the to-be-processed area in the to-be-processed image is locally enhanced, and the effect of enhancing the to-be-processed area is better. Therefore, the quality of the image data corresponding to the to-be-processed area in the first processed image is higher, and the to-be-processed image is globally enhanced. Compared with the local enhancement processing on the to-be-processed area, the quality of the image data corresponding to the to-be-processed area in the second processed image is lower, and the mask value in the mask corresponding to the to-be-processed image represents whether the pixel point is located in the to-be-processed area, and then according to the mask corresponding to the to-be-processed image, the image data corresponding to the to-be-processed area in the first processed image and the image data corresponding to the non-to-be-processed area in the second processed image are fused into a target image, and the quality of the image data corresponding to the to-be-processed area and the image data corresponding to the non-to-be-processed area in the target image are both relatively high, thereby improving the clarity of the image.

[0063] In some exemplary embodiments, steps 104 to 108 include:

[0064] The first neural network model is used to enhance the area to be processed in the image to be processed to obtain a first processed image; the second neural network model is used to enhance the image to be processed to obtain a second processed image; and the third neural network model is used to determine the mask corresponding to the image to be processed.

[0065] The third neural network model refers to a detection model for the area to be processed, which can be understood as the third neural network model being used to determine the area to be processed in the image to be processed. The input of the third neural network model is the image to be processed, and the output of the third neural network model is the position of the pixel point in the area to be processed, and the position of the pixel point can be represented by a two-dimensional coordinate.

[0066] Exemplarily, the electronic device inputs the image to be processed into the first neural network model, the first neural network model outputs the first processed image, and the electronic device obtains the first processed image; the electronic device inputs the image to be processed into the second neural network model, the second neural network model outputs the second processed image, and the electronic device obtains the second processed image; the electronic device inputs the image to be processed into the third neural network model, the third neural network model outputs the position of the pixel to be processed, and the electronic device obtains the position of the pixel to be processed, and based on the position of the pixel to be processed, the mask value of the pixel to be processed is determined as the first identifier, and the mask values ​​of the remaining pixels are determined as the second identifier, and the mask corresponding to the image to be processed is obtained based on the mask values ​​corresponding to each pixel in the image to be processed. The pixel to be processed refers to the pixel located in the area to be processed determined by the third neural network, and the pixel to be processed can be located in the area to be processed or in the non-area to be processed, that is, the area to be processed determined by the third neural network model and the area to be processed determined by the first neural network model can be image areas that do not completely overlap.

[0067] In some exemplary embodiments, the training process of the second neural network model includes: acquiring multiple high-resolution images, for each high-resolution image, processing the high-resolution image using a simulated degradation method to obtain a low-resolution image corresponding to the high-resolution image, and combining the high-resolution image and the low-resolution image to form a training data set; using multiple training data sets to train the initial second neural network model to obtain a second neural network model.

[0068] In this embodiment, a pre-trained second neural network model is used to enhance the image to be processed, and the powerful clear picture prior ability of the second neural network model is used to better retain and enhance the details of the image to be processed, thereby improving the clarity of the second processed image. For face areas, text areas and small structure areas where the enhancement processing effect of the second neural network model is poor, the first neural network model is used to perform image enhancement processing on the face areas, text areas and small structure areas in the image to be processed, and then the third neural network model is used to determine the face areas, text areas and small structure areas in the image to be processed, thereby providing accurate basic data for the subsequent fusion of the first processed image and the second processed image.

[0069] In some exemplary embodiments, based on the mask, the first processed image and the second processed image are fused to obtain a target image, including:

[0070] For each pixel in the image to be processed, the mask value of the pixel in the mask is obtained; when the mask value is a first identifier, the pixel value corresponding to the pixel in the first processed image is determined as the target pixel value of the pixel; when the mask value is a second identifier, the pixel value corresponding to the pixel in the second processed image is determined as the target pixel value of the pixel; based on the target pixel value corresponding to each pixel in the image to be processed, a target image is obtained.

[0071] The first mark refers to a mark indicating that the pixel point is located in the area to be processed, that is, the first mark refers to a mark indicating that the pixel point is located in the face area, the text area and the small structure area. The second mark refers to a mark indicating that the pixel point is located in the non-area to be processed, that is, the second mark refers to a mark indicating that the pixel point is located in other areas except the face area, the text area and the small structure area.

[0072] Exemplarily, for each pixel in the image to be processed, the electronic device obtains the mask value of the pixel in the mask. If the mask value is a first identifier, the pixel value corresponding to the pixel in the first processed image is determined as the target pixel value of the pixel. If the mask value is a second identifier, the pixel value corresponding to the pixel in the second processed image is determined as the target pixel value of the pixel. Based on the target pixel value corresponding to each pixel in the image to be processed, a target image is obtained.

[0073] In this embodiment, if the mask value is the first identifier, indicating that the pixel point is located in the area to be processed, the pixel value corresponding to the pixel point in the first processed image is determined as the target pixel value of the pixel point. If the mask value is the second identifier, indicating that the pixel point is located in the area not to be processed, the pixel value corresponding to the pixel point in the second processed image is determined as the target pixel value of the pixel point. That is, the target image is obtained by fusing the first processed image and the second processed image, thereby avoiding the problem of low image data quality in the face area, text area and small structure area in the second processed image, thereby improving the clarity of the target image.

[0074] In some exemplary embodiments, obtaining an image to be processed includes:

[0075] Acquire an initial image and a resolution of the initial image; compare the resolution of the initial image with a resolution threshold; and when the resolution of the initial image is less than the resolution threshold, determine the initial image as an image to be processed.

[0076] The resolution of the initial image refers to the number of pixels per unit area in the initial image, and the resolution threshold refers to the maximum resolution of the initial image as the image to be processed.

[0077] Exemplarily, the electronic device obtains an initial image and a resolution of the initial image, compares the resolution of the initial image with a resolution threshold, and if the resolution of the initial image is less than the resolution threshold, determines the initial image as an image to be processed.

[0078] In this embodiment, by comparing the resolution of the initial image with a resolution threshold, if the resolution of the initial image is less than the resolution threshold, the initial image is determined as the image to be processed, that is, the initial image with lower clarity is determined as the image to be processed that requires image processing.

[0079] In some exemplary embodiments, when the resolution of the initial image is less than a resolution threshold, determining the initial image as an image to be processed includes:

[0080] When the resolution of the initial image is less than the resolution threshold, the classification identifier of the initial image is determined; when the classification identifier is the target classification identifier, the initial image is determined as the image to be processed.

[0081] The classification identifier refers to an identifier that characterizes the quality of the initial image. The classification identifier may be a first classification identifier or a second classification identifier. The first classification identifier characterizes that the initial image is a locally blurred image or a semantically clear image. The second classification identifier characterizes that the initial image is a globally blurred image or a semantically unclear image. The target classification identifier refers to a pre-set classification identifier used to determine that the initial image is an image to be processed. The target classification identifier may be the first classification identifier, that is, the target classification identifier characterizes a locally blurred image or a semantically clear image.

[0082] Exemplarily, when the resolution of the initial image is less than a resolution threshold, the electronic device determines the classification identifier of the initial image based on the initial image, compares the classification identifier with the target classification identifier, and if the classification identifier is the same as the target classification identifier, determines the initial image as the image to be processed.

[0083] In the present embodiment, when the classification identifier is the target classification identifier, the initial image is determined as the image to be processed, that is, the initial image is determined as the image to be processed only when the initial image is a locally blurred image or a semantically clear image. This avoids the situation where the initial image is determined as the image to be processed when the initial image is a globally blurred image or a semantically unclear image, and the above-mentioned image processing process is performed on the image to be processed, resulting in a large difference between the target image and the initial image.

[0084] In some exemplary embodiments, determining a classification identification of an initial image includes:

[0085] The classification identification of the initial image is determined by the fourth neural network model.

[0086] The fourth neural network model refers to a neural network model used to determine the classification mark. The fourth neural network model can be a convolutional neural network (CNN) model for binary classification. The input of the fourth neural network model is the initial image, and the output is the classification mark.

[0087] Exemplarily, the electronic device inputs the image to be processed into the fourth neural network model, the fourth neural network model outputs a classification identifier, and the electronic device obtains the classification identifier of the initial image.

[0088] In this embodiment, the classification identification of the initial image is determined by the fourth neural network model, which provides an accurate judgment basis for the subsequent determination of the image processing method, and avoids the situation where the initial image is determined as the image to be processed and the above-mentioned image processing process is performed on the image to be processed, resulting in a large difference between the target image and the initial image when the initial image is a globally blurred image or an image with unclear semantics.

[0089] In some exemplary embodiments, when the classification identifier is a target classification identifier, determining the initial image as the image to be processed includes:

[0090] When the classification identifier is the target classification identifier, the resolution of the initial image is compared with the target resolution; the target resolution represents the standard resolution of the first neural network model and the second neural network model for enhancing the image to be processed; when the resolution of the initial image is equal to the target resolution, the initial image is determined as the image to be processed; when the resolution of the initial image is not equal to the target resolution, the initial image is sampled based on the resolution of the initial image and the target resolution to obtain the image to be processed; the resolution of the image to be processed is equal to the target resolution.

[0091] The target resolution refers to the resolution of the image suitable for enhancement processing by the first neural network model and the second neural network model. It can be understood that the first neural network model and the second neural network model are better for enhancing the image with the target resolution. The standard resolution refers to the resolution of the image suitable for enhancement processing by the first neural network model and the second neural network model.

[0092] Exemplarily, when the classification identifier is a target classification identifier, the electronic device compares the resolution of the initial image with the target resolution. If the resolution of the initial image is equal to the target resolution, the initial image is determined as the image to be processed; if the resolution of the initial image is not equal to the target resolution, the sampling magnification is determined based on the resolution of the initial image and the target resolution, and the initial image is sampled based on the sampling magnification to obtain an image to be processed whose resolution is equal to the target resolution.

[0093] In this embodiment, when the resolution of the initial image is not equal to the target resolution, the initial image is sampled and processed based on the resolution of the initial image and the target resolution to obtain an image to be processed. That is, by sampling the initial image, an image to be processed whose resolution is equal to the target resolution is obtained, which lays a foundation for the subsequent first neural network model and the second neural network model to process the image to be processed.

[0094] In some exemplary embodiments, after determining the classification identifier of the initial image, the method further includes:

[0095] When the classification identifier is not the target classification identifier, the initial image is input into the fifth neural network model to obtain the target image.

[0096] The fifth neural network model refers to a neural network model used to enhance globally blurred images or semantically unclear images. The fifth neural network model may be an image super-resolution (SR) model. The image enhancement effect of the fifth neural network model is relatively weak. It can be understood that the fifth neural network model performs slight enhancement processing on globally blurred images or semantically unclear images.

[0097] Exemplarily, the electronic device compares the classification identifier of the initial image with the target classification identifier. If the classification identifier is different from the target classification identifier, the initial image is input into the fifth neural network model, the fifth neural network model outputs the target image, and the electronic device obtains the target image.

[0098] In this embodiment, when the classification identifier is not the target classification identifier, the electronic device inputs the initial image into the fifth neural network model to obtain the target image, that is, for the globally blurred image or the semantically unclear image, the fifth neural network model is used to perform slight enhancement processing on it, thereby avoiding the situation where the target image is significantly different from the initial image, thereby improving the clarity of the target image.

[0099] In some exemplary embodiments, the enhancement processing is at least one of resolution enhancement processing, brightness enhancement processing, and semantic enhancement processing.

[0100] Among them, resolution enhancement refers to the process of improving the resolution of an image, which makes the details of the image clearer and richer. Brightness enhancement refers to the process of adjusting the brightness of an image, which makes the image brighter or meets specific visual effects. Semantic enhancement refers to the process of enhancing the visual effects of key structures or specific content in an image, which can improve the visibility and clarity of specific semantic areas in an image.

[0101] Exemplarily, an electronic device acquires an image to be processed; performs at least one of resolution enhancement, brightness enhancement, and semantic enhancement on a region to be processed in the image to be processed to obtain a first processed image; performs at least one of resolution enhancement, brightness enhancement, and semantic enhancement on the image to be processed to obtain a second processed image; determines a mask corresponding to the image to be processed; and based on the mask, fuses the first processed image and the second processed image to obtain a target image.

[0102] In this embodiment, the explicit enhancement processing may be a superposition of one or more of resolution enhancement processing, brightness enhancement processing, and semantic enhancement processing.

[0103] In some exemplary embodiments, the model management system diagram is as follows Figure 2A As shown, the model management system enables users (e.g., developers) to easily integrate artificial intelligence into applications (e.g., software). The model management system may provide a model selection interface, and the user may use the model selection interface to provide user specifications, which may include, for example, selections of the following: AI (Artificial Intelligence) tasks (e.g., object detection, scene recognition, image tagging, etc.) expected to be integrated into their applications; devices expected to deploy AI tasks; one or more performance constraints of the selected devices (e.g., memory, latency, power, etc.). Based on the user specification, the model management system may generate a customized AI model that can be integrated into the application and can operate on the device specified by the user. The model management system may generate a customized AI model by retrieving an original or advanced model from a database containing multiple AI models, and may compile an advanced model to generate the model binary. In a specific embodiment, for an AI model retrieved according to a user specification, unnecessary components may be removed and one or more components may be added to create a customized AI model. The model management system may bind one or more interface functions to the model binary. One or more interface functions may be functions associated with an AI task selected by a user, for example, if the task is image tagging, the interface function is a set of labels contained in the image. In response to binding the interface function to the model binary, the model management system may convert the model binary into a device-level model that is configured on a device selected by the user, for example, the device-level model may be configured for an electronic device for image processing. The device-level model may ultimately be provided to the user in the form of a downloadable link that the user may download to integrate the desired AI functionality into their application.

[0104] Send the user specification defined by the model selection interface to Figure 2AThe query generator server 202 of the model management system shown. The query generator server 202 receives the user specification and generates a query 204, which can be used to retrieve one or more original models 208 (or device-level models) from the database 206 of AI models, which may include multiple AI models that can be defined and stored based on previous user specifications, and the previous user specifications may be related or unrelated to the current user specifications. The original model 208 retrieved from the database 206 of AI models can be processed by the model compiler 210 to adapt it to the current user specification. For example, the model compiler 210 can compile the model to remove any unnecessary modules / components and add any required components to match the user specification. In a specific embodiment, the output of the model compiler 210 is a compact model package or model binary 212. The interface binder 214 can bind the compact model package / model binary to one or more interface functions. Once one or more interface functions are bound to the compact model package / model binary, the final software binary (a combination of the model binary and the interface functions) can be sent to a distributor 218, which creates a link 220 for the user to download the final software binary 216 on his device. Upon download, the user can incorporate the received final software binary into his application.

[0105] In some exemplary embodiments, the model management system diagram is as follows Figure 2B As shown, training of a model may include improving the performance of an existing model, adding one or more new features to an existing model, and / or creating a new model. A user may provide training data when defining a user specification using a model selection interface, and the user may provide the training data together with the user specification to a model management system, which may use the training data to train a pool of AI models on a server 222. The trained AI model 224 may be sent to a ranker 226 for ranking. The ranker 226 may rank the models based on various factors, which may include performance metrics associated with the AI ​​model, benchmark data, user feedback on the model, and the like. Based on the ranking, the highest-ranking model 228 may be determined. For example, the top five models may be selected and sent to a model converter 230 of the device, which may convert the highest-ranked AI model into a device-level model 232, which may be automatically benchmarked 234 on these models in terms of speed, memory, and power constraints, as discussed above, and finally these device-level models may be stored in a database 206 of AI models together with the benchmark data for future access and / or retrieval. In a particular embodiment, Figure 2B Steps 222-234 shown may be used with Figure 2A The steps shown are performed asynchronously.

[0106] The model selection interface diagram is as follows: Figure 3As shown, in some embodiments, the model selection interface may be presented in a browser, for example, a user may enter a website, and the model selection interface is presented in the browser of the website. The model selection interface may provide a variety of menus 302-310, options within these menus, and sub-options or categories 312-314, which the user may select in order to implement the AI ​​functions required by their application. It should be noted that the user interface shown here is for exemplary purposes and is not limited in any way to the menus, options, and sub-options shown herein.

[0107] like Figure 3 As shown, the model selection interface may include a task menu 302, a device type menu 304, and a constraint menu 306-310. The task menu 302 may include a variety of AI tasks, from which users may select desired tasks to incorporate into their applications. By way of example and not limitation, these tasks may include scene recognition, image tagging, object detection, object tracking, object segmentation, human pose estimation, image enhancement, behavior recognition, human emotion recognition, speech recognition, text recognition, natural language understanding, and / or any kind of data processing tasks. In a particular embodiment, the selected task may include one or more tasks, for example, an object detection task may include a person, a tree, a car, a bicycle, a dog, a building, etc.; for another example, a scene recognition task may include a beach, a snowy mountain, a flower, a building, an autumn leaf, a waterfall, a night view, etc. For another example, an image enhancement task may include denoising, super-resolution, etc.

[0108] The device type menu 304 may include different types of user devices for deploying a given task. By way of example and not limitation, user devices may include smartphones, tablets, smart watches, ARM (Advanced RISCMachines)-based platforms, embedded sensors, cameras, Intel-based platforms, drones, Snapdragon-based platforms, NVIDIA-based platforms, X-86 series, Ambarella platforms, etc. From the list of user devices, the user can select a device to deploy the AI ​​model, in other words, the user can select one or more target devices to deploy the AI ​​model.

[0109] The constraint menus 306-310 may include a memory constraint menu 306, a delay constraint menu 308, and a power constraint menu 310. The memory constraint menu 306 may enable a user to specify the amount of memory to be used for the task they selected, e.g., Figure 3As shown, the user may choose to allocate 100MB (Megabyte) of memory to the object detection task. The delay constraint menu 308 may enable the user to specify the refresh rate or frames per second (FPS) at which the task runs, for example, Figure 3 As shown, the user may choose to run the object detection task at 10 FPS. The power constraint menu 310 may enable the user to specify the amount of power to be utilized by the task selected, for example, Figure 3 As shown, the user can choose to allocate 1JPI (Joules Per Instruction) of power to the object detection task.

[0110] Use as Figure 3 The model selection interface shown generates user specifications and sends the user specifications to Figure 2A Or the model management system shown in 2B, generates the first neural network model, the second neural network model, the third neural network model, the fourth neural network model and the fifth neural network model in sequence, and binds the first neural network model, the second neural network model, the third neural network model, the fourth neural network model and the fifth neural network model to the electronic device. Among them, the first neural network model is an SR model, which focuses on enhancing the face area, the text area and the small structure area; the second neural network model is a pre-trained SDSR model, which is a neural network model for enhancing the global image, but the enhancement effect on the face area, the text area and the small structure area is poor; the third neural network model is used to detect the face area, the text area and the small structure area in the image; the fourth neural network model is used to determine the classification mark of the image, that is, to determine whether the image is a locally blurred image, a semantically clear image, a globally blurred image or a semantically unclear image; the fifth neural network model is an SR model, which is used to slightly enhance the globally blurred image or the semantically unclear image.

[0111] The third neural network model testing process diagram is as follows Figure 4 As shown, the third neural network model includes an encoder module and a decoder module. During the test process, the sample image and the operation parameters can be provided to the third neural network model together. The third neural network model can provide sample output data by processing the sample image using the operation parameters. The sample output data is compared with the known data of the sample image to determine whether the sample output data matches the known data of the sample image. If the sample output data matches the known data of the sample image, the parameters of the third neural network model are set as the operation parameters; if the sample output data does not match the known data of the sample image, the operation parameters are fine-tuned.

[0112] The schematic diagram of the process of the third neural network model detecting face area, text area and small structure area is as follows Figure 5 As shown, the image to be processed is input into the third neural network model, and the encoder module of the third neural network model encodes the image to be processed, and the features in the image to be processed are represented as feature vectors in the feature space. The feature vector is provided to the decoder module, and the decoder module decodes the feature vector to evaluate one or more attributes of the image to be processed, so as to determine the output data from the image to be processed, that is, the positions of the pixels located in the face area, the text area and the small structure area. Decoding the feature vector may include classifying the feature vector using the classification parameters determined during the test process, and classifying the feature vector may include operating the feature vector using one or more classifiers or networks that support classification.

[0113] The image processing flow chart is as follows: Figure 6 As shown, including:

[0114] The electronic device obtains an initial image and a resolution of the initial image, compares the resolution of the initial image with a resolution threshold, if the resolution of the initial image is less than the resolution threshold, inputs the initial image into a fourth neural network model, the fourth neural network model outputs a classification identifier, the electronic device obtains the classification identifier of the initial image, compares the classification identifier with a target classification identifier, if the classification identifier is not the target classification identifier, the electronic device inputs the initial image into a fifth neural network model to obtain a target image.

[0115] When the classification identifier is a target classification identifier, the electronic device compares the resolution of the initial image with the target resolution. If the resolution of the initial image is equal to the target resolution, the initial image is determined as the image to be processed; if the resolution of the initial image is not equal to the target resolution, the sampling magnification is determined based on the resolution of the initial image and the target resolution, and the initial image is sampled based on the sampling magnification to obtain an image to be processed whose resolution is equal to the target resolution.

[0116] The electronic device inputs the image to be processed into the first neural network model, the first neural network model outputs the first processed image, and the electronic device obtains the first processed image; the electronic device inputs the image to be processed into the second neural network model, the second neural network model outputs the second processed image, and the electronic device obtains the second processed image; the electronic device inputs the image to be processed into the third neural network model, the third neural network model outputs the position of the pixel to be processed, and the electronic device obtains the position of the pixel to be processed, and based on the position of the pixel to be processed, the mask value of the pixel to be processed is determined as the first identifier, and the mask values ​​of the remaining pixels are determined as the second identifier, and the mask corresponding to the image to be processed is obtained based on the mask values ​​corresponding to each pixel in the image to be processed. The pixel to be processed refers to the pixel located in the area to be processed determined by the third neural network, and the pixel to be processed can be located in the area to be processed or in the non-area to be processed, that is, the area to be processed determined by the third neural network model and the area to be processed determined by the first neural network model can be image areas that do not completely overlap.

[0117] For each pixel in the image to be processed, the electronic device obtains the mask value of the pixel in the mask. If the mask value is a first identifier, the pixel value corresponding to the pixel in the first processed image is determined as the target pixel value of the pixel. If the mask value is a second identifier, the pixel value corresponding to the pixel in the second processed image is determined as the target pixel value of the pixel. Based on the target pixel value corresponding to each pixel in the image to be processed, the target image is obtained.

[0118] Figure 7 The left image in the figure is the target image obtained by using the above image processing method, and the right image is the enhanced image obtained by using a separate SDSR model to enhance the initial image. The clarity of the face area of ​​the target image is higher than that of the enhanced image. Figure 8 The left image in the figure is the target image obtained by using the above image processing method, and the right image is the enhanced image obtained by using a separate SDSR model to enhance the initial image. The clarity of the text area in the target image is higher than that of the enhanced image. Fig. 9 The left image in the figure is the target image processed by the above image processing method, and the right image is the enhanced image obtained by using a separate SDSR model to enhance the initial image. The clarity of the text area of ​​the target image is higher than that of the enhanced image. It can be seen that compared with the enhancement of the initial image using the SDSR model alone, the clarity of the face area and text area of ​​the target image processed by the above image processing method is higher.

[0119] Fig.10The left picture in the figure is an enhanced image obtained by enhancing the initial image using the SR model trained by the Generative Adversarial Networks (GAN) technology. The right picture is a target image processed by the above image processing method. The clarity of the target image is higher than that of the enhanced image. Fig.11 The left picture in the figure is an enhanced image obtained by using the above SR model to enhance the initial image, and the right picture is a target image processed by the above image processing method. The clarity of the target image is higher than that of the enhanced image. Fig.12 The left picture in the figure is the enhanced image obtained by using the above SR model to enhance the initial image, and the right picture is the target image obtained by using the above image processing method. The clarity of the target image is higher than that of the enhanced image. It can be seen that compared with using the SR model alone to enhance the initial image, the clarity of the target image obtained by using the above image processing method is higher.

[0120] In this embodiment, the to-be-processed area in the to-be-processed image is enhanced to obtain a first processed image, the to-be-processed image is enhanced to obtain a second processed image, and the mask corresponding to the to-be-processed image is determined, that is, the to-be-processed area in the to-be-processed image is locally enhanced, and the effect of the enhancement processing on the to-be-processed area is better. Therefore, the quality of the image data corresponding to the to-be-processed area in the first processed image is higher, and the to-be-processed image is globally enhanced. Compared with the local enhancement processing on the to-be-processed area, the quality of the image data corresponding to the to-be-processed area in the second processed image is lower, and the mask value in the mask corresponding to the to-be-processed image represents whether the pixel point is located in the to-be-processed area, and then according to the mask corresponding to the to-be-processed image, the image data corresponding to the to-be-processed area in the first processed image and the image data corresponding to the non-to-be-processed area in the second processed image are fused into a target image, and the quality of the image data corresponding to the to-be-processed area and the image data corresponding to the non-to-be-processed area in the target image are both relatively high, thereby improving the clarity of the image.

[0121] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0122] Based on the same inventive concept, the embodiment of the present application also provides an image processing device for implementing the above-mentioned image processing method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image processing device embodiments provided below can refer to the limitations on the image processing method above, and will not be repeated here.

[0123] In an exemplary embodiment, Fig.13 As shown, an image processing device is provided, including: an acquisition module 1302, a local processing module 1304, a global processing module 1306, a mask determination module 1308 and a fusion module 1310, wherein:

[0124] An acquisition module 1302 is used to acquire an image to be processed;

[0125] A local processing module 1304 is used to perform enhancement processing on the to-be-processed area in the to-be-processed image to obtain a first processed image;

[0126] A global processing module 1306 is used to perform enhancement processing on the image to be processed to obtain a second processed image;

[0127] The mask determination module 1308 is used to determine the mask corresponding to the image to be processed; each mask value in the mask is used to represent the fusion weight corresponding to the pixel value in the first processed image or the second processed image;

[0128] The fusion module 1310 is used to fuse the first processed image and the second processed image based on the mask to obtain a target image.

[0129] In an exemplary embodiment, the local processing module 1304 is also used to: enhance the area to be processed in the image to be processed by a first neural network model to obtain a first processed image; the global processing module 1306 is also used to: enhance the image to be processed by a second neural network model to obtain a second processed image; the mask determination module 1308 is also used to: determine the mask corresponding to the image to be processed by a third neural network model.

[0130] In an exemplary embodiment, the fusion module 1310 is also used to: for each pixel in the image to be processed, obtain the mask value of the pixel in the mask; when the mask value is a first identifier, determine the pixel value corresponding to the pixel in the first processed image as the target pixel value of the pixel; when the mask value is a second identifier, determine the pixel value corresponding to the pixel in the second processed image as the target pixel value of the pixel; and obtain the target image based on the target pixel value corresponding to each pixel in the image to be processed.

[0131] In an exemplary embodiment, the acquisition module 1302 is also used to: acquire an initial image and a resolution of the initial image; compare the resolution of the initial image with a resolution threshold; and when the resolution of the initial image is less than the resolution threshold, determine the initial image as an image to be processed.

[0132] In an exemplary embodiment, the acquisition module 1302 is further used to: determine the classification identifier of the initial image when the resolution of the initial image is less than the resolution threshold; and determine the initial image as the image to be processed when the classification identifier is a target classification identifier.

[0133] In an exemplary embodiment, the acquisition module 1302 is further used to determine the classification identification of the initial image through a fourth neural network model.

[0134] In an exemplary embodiment, the acquisition module 1302 is also used to: when the classification identifier is a target classification identifier, compare the resolution of the initial image with the target resolution; the target resolution represents the standard resolution of the first neural network model and the second neural network model for enhancing the image to be processed; when the resolution of the initial image is equal to the target resolution, determine the initial image as the image to be processed; when the resolution of the initial image is not equal to the target resolution, sample the initial image based on the resolution of the initial image and the target resolution to obtain the image to be processed; the resolution of the image to be processed is equal to the target resolution.

[0135] In an exemplary embodiment, the acquisition module 1302 is further used to: when the classification identifier is not the target classification identifier, input the initial image into the fifth neural network model to obtain the target image.

[0136] In an exemplary embodiment, the enhancement processing is at least one of resolution enhancement processing, brightness enhancement processing, and semantic enhancement processing.

[0137] Each module in the above-mentioned image processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in an electronic device in the form of hardware, or can be stored in a memory in an electronic device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0138] In an exemplary embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.14As shown. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and an external device. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, an image processing method is implemented. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the electronic device casing, or an external keyboard, touchpad or mouse.

[0139] Those skilled in the art will understand that Fig.14 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] In one embodiment, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0141] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0142] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0143] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0144] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0146] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method, characterized in that: include: Get the image to be processed; Performing enhancement processing on the to-be-processed area in the to-be-processed image to obtain a first processed image; Performing enhancement processing on the image to be processed to obtain a second processed image; Determining a mask corresponding to the image to be processed; Each mask value in the mask is used to represent a fusion weight corresponding to a pixel value in the first processed image or the second processed image; Based on the mask, the first processed image and the second processed image are fused to obtain a target image.

2. The method according to claim 1, characterized in that The step of performing enhancement processing on the to-be-processed area in the to-be-processed image to obtain a first processed image includes: Performing enhancement processing on the to-be-processed region in the to-be-processed image by using a first neural network model to obtain a first processed image; The step of performing enhancement processing on the image to be processed to obtain a second processed image includes: Performing enhancement processing on the image to be processed by a second neural network model to obtain a second processed image; The step of determining a mask corresponding to the image to be processed includes: The mask corresponding to the image to be processed is determined by a third neural network model.

3. The method according to claim 1, characterized in that The step of fusing the first processed image and the second processed image based on the mask to obtain a target image includes: For each pixel in the image to be processed, obtaining a mask value of the pixel in the mask; When the mask value is the first identifier, determining the pixel value corresponding to the pixel point in the first processed image as the target pixel value of the pixel point; When the mask value is a second identifier, determining a pixel value corresponding to the pixel point in the second processed image as a target pixel value of the pixel point; A target image is obtained based on the target pixel value corresponding to each pixel point in the image to be processed.

4. The method according to claim 1, characterized in that: The step of obtaining the image to be processed comprises: Acquire an initial image and a resolution of the initial image; comparing the resolution of the initial image with a resolution threshold; When the resolution of the initial image is less than the resolution threshold, the initial image is determined as the image to be processed.

5. The method according to claim 4, characterized in that When the resolution of the initial image is less than the resolution threshold, determining the initial image as an image to be processed includes: When the resolution of the initial image is less than the resolution threshold, determining a classification identifier of the initial image; In the case where the classification identifier is a target classification identifier, the initial image is determined as an image to be processed.

6. The method according to claim 5, characterized in that The determining of the classification identification of the initial image includes: The classification identification of the initial image is determined by a fourth neural network model.

7. The method according to claim 5, characterized in that When the classification identifier is a target classification identifier, determining the initial image as an image to be processed includes: In the case where the classification identifier is a target classification identifier, the resolution of the initial image is compared with a target resolution; the target resolution represents a standard resolution of the first neural network model and the second neural network model for performing enhancement processing on the image to be processed; When the resolution of the initial image is equal to the target resolution, determining the initial image as the image to be processed; When the resolution of the initial image is not equal to the target resolution, the initial image is sampled based on the resolution of the initial image and the target resolution to obtain an image to be processed; the resolution of the image to be processed is equal to the target resolution.

8. The method according to claim 5, characterized in that After determining the classification identifier of the initial image, the method further includes: When the classification identifier is not the target classification identifier, the initial image is input into the fifth neural network model to obtain the target image.

9. The method according to claim 1, characterized in that: The enhancement processing is at least one of resolution enhancement processing, brightness enhancement processing and semantic enhancement processing.

10. An image processing device, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be processed; A local processing module, used for performing enhancement processing on the to-be-processed area in the to-be-processed image to obtain a first processed image; A global processing module, used for performing enhancement processing on the image to be processed to obtain a second processed image; A mask determination module, used to determine a mask corresponding to the image to be processed; each mask value in the mask is used to represent a fusion weight corresponding to a pixel value in the first processed image or the second processed image; A fusion module is used to fuse the first processed image and the second processed image based on the mask to obtain a target image.

11. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images