Image processing method, device, electronic device and storage medium

Abstract

The present application discloses an image processing method, comprising: obtaining a first image corresponding to an image to be processed, and obtaining a second image corresponding to the image to be processed, wherein the first image and the second image are images obtained after image quality reduction processing is performed on the image to be processed, and the image quality of the first image is different from the image quality of the second image; obtaining image structure information corresponding to the image to be processed; and performing image fusion processing on the first image and the second image based on the image structure information corresponding to the image to be processed to obtain a target image corresponding to the image to be processed. The image processing method provided by the present application can ensure the diversity of image structure information and image quality between different target images, thereby avoiding the problem of low authenticity of the target image caused by the relatively single image structure information and image quality between different target images when a globally unified generation method is adopted.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more particularly to an image processing method, an image processing device, an electronic device, and a storage medium. Background Art

[0002] Image quality enhancement technology is an image processing technology that improves image clarity, resolvability, etc. to a preset threshold by enhancing useful information in the image and removing blur and noise in the image, thereby improving image quality. Image quality enhancement technology has a wide range of application needs in various scenarios related to image transmission, image storage, image editing, and image display. Generally speaking, the implementation of image quality enhancement technology mainly needs to be based on a network model for image quality enhancement. In order to obtain a network model for image quality enhancement, a large number of high-quality sample images and low-quality sample images corresponding to these high-quality sample images are required as training samples to train the network model for image quality enhancement.

[0003] Nowadays, with the rapid development and upgrading of image acquisition equipment, the ways to obtain high-quality sample images are becoming more and more numerous and convenient. At this time, in order to obtain low-quality sample images corresponding to high-quality sample images in a large number and concisely, it is often necessary to obtain low-quality sample images corresponding to high-quality sample images by reducing the image quality of high-quality sample images.

[0004] Existing image quality reduction processing methods generally use a globally unified generation method for a large number of high-quality images to be degraded to generate low-quality images corresponding to the high-quality images. This often leads to the problem of poor authenticity of the low-quality images due to the relatively simple image structure information of different low-quality images. Summary of the Invention

[0005] The present application provides an image processing method, apparatus, electronic device, and storage medium to improve the authenticity of a target image generated after image quality reduction processing.

[0006] This application provides an image processing method, comprising:

[0007] Obtaining a first image corresponding to an image to be processed, and obtaining a second image corresponding to the image to be processed, wherein the first image and the second image are images obtained after image quality reduction processing is performed on the image to be processed, and image quality of the first image is different from image quality of the second image;

[0008] Obtaining image structure information corresponding to the image to be processed;

[0009] Image fusion processing is performed on the first image and the second image according to image structure information corresponding to the image to be processed to obtain a target image corresponding to the image to be processed.

[0010] Optionally, obtaining image structure information corresponding to the image to be processed includes:

[0011] Dividing the image to be processed into a plurality of target image areas;

[0012] Image structure information corresponding to the multiple target image regions is obtained.

[0013] Optionally, performing image fusion processing on the first image and the second image according to image structure information corresponding to the image to be processed to obtain a target image corresponding to the image to be processed includes:

[0014] Obtaining target weights for characterizing image structure information corresponding to the plurality of target image regions;

[0015] According to the target weight, image fusion processing is performed on the first image and the second image to obtain a target image corresponding to the image to be processed.

[0016] Optionally, obtaining target weights for characterizing image structure information corresponding to the multiple target image regions includes:

[0017] obtaining, based on the image edge feature information corresponding to each target image region and the image clarity feature information corresponding to each target image region, a region weight for characterizing the image structure information corresponding to each target image region;

[0018] The target weight is obtained according to the region weight.

[0019] Optionally, obtaining, based on the image edge feature information corresponding to each target image region and the image clarity feature information corresponding to each target image region, a region weight for characterizing the image structure information corresponding to each target image region among the multiple target image regions includes:

[0020] Obtaining a regional edge feature value for characterizing image edge feature information corresponding to each target image region, and obtaining a regional clarity feature value for characterizing image clarity feature information corresponding to each target image region;

[0021] The region weight is determined according to the region edge feature value and the region clarity feature value.

[0022] Optionally, obtaining the target weight according to the region weight includes:

[0023] Determining image edge feature value images corresponding to the plurality of target image regions according to the region edge feature values ​​and the region clarity feature values, and obtaining image clarity feature value images corresponding to the plurality of target image regions;

[0024] The target weight is determined according to the image edge eigenvalue image and the image clarity eigenvalue image.

[0025] Optionally, determining the target weight according to the image edge eigenvalue image and the image clarity eigenvalue image includes:

[0026] determining, based on the image edge eigenvalue image and the image clarity eigenvalue image, an initial image structure weight image for characterizing image structure information corresponding to the plurality of target image regions;

[0027] Performing a Gaussian blur operation and an image morphological operation on the initial image structure weighted image to obtain a second image structure weighted image for representing image structure information corresponding to the plurality of target image regions;

[0028] The weights in the second image structure weight image are normalized to obtain a target structure weight image for characterizing the image structure information corresponding to the multiple target image areas, and the weights in the target structure weight image are used as the target weights.

[0029] Optionally, obtaining the image structure information corresponding to the multiple target image regions includes:

[0030] Performing edge detection and blur detection on the image to be processed to obtain image edge feature information corresponding to the multiple target image regions, and obtaining image clarity feature information corresponding to the multiple target image regions;

[0031] Image structure information corresponding to the multiple target image regions is obtained based on the image edge feature information corresponding to the multiple target image regions and the image clarity feature information corresponding to the multiple target image regions.

[0032] Optionally, performing image fusion processing on the first image and the second image according to the target weight to obtain a target image corresponding to the image to be processed includes:

[0033] According to the multiple target image areas, dividing the first image into multiple first image areas corresponding to the multiple target image areas, and dividing the second image into multiple second image areas corresponding to the multiple target image areas;

[0034] According to the target weight, weighted averaging processing is performed on the image structure information corresponding to the multiple first image regions and the image structure information corresponding to the multiple second image regions to obtain the target image.

[0035] Optionally, obtaining a first image corresponding to the image to be processed and obtaining a second image corresponding to the image to be processed includes:

[0036] Obtaining the image to be processed;

[0037] determining an image quality of the first image, and determining an image quality of the second image;

[0038] According to the image quality of the first image, the image to be processed is subjected to image quality reduction processing to obtain the first image, and according to the image quality of the second image, the image to be processed is subjected to image quality reduction processing to obtain the second image.

[0039] Optionally, performing image quality reduction processing on the image to be processed according to the image quality of the first image to obtain the first image includes: adding interference information to the image to be processed according to the image quality of the first image to obtain the first image.

[0040] Optionally, performing image quality reduction processing on the image to be processed based on the image quality of the first image to obtain the first image includes: removing valid information of the image to be processed based on the degree of quality reduction of the first image to obtain the first image.

[0041] Optionally, the method further includes: obtaining a sample image to be processed for training an image quality enhancement model, wherein the image quality enhancement model is a model for performing image quality enhancement processing;

[0042] The sample image to be processed and the target image are used as inputs of the image quality enhancement model to train the image quality enhancement model.

[0043] In another aspect, the present application provides an image processing device, comprising:

[0044] a first image processing unit, configured to obtain a first image corresponding to an image to be processed, and obtain a second image corresponding to the image to be processed, wherein the first image and the second image are images obtained by performing image quality reduction processing on the image to be processed, and the image quality of the first image is different from the image quality of the second image;

[0045] A second image processing unit, configured to obtain image structure information corresponding to the image to be processed;

[0046] The third image processing unit is configured to perform image fusion processing on the first image and the second image according to image structure information corresponding to the image to be processed, so as to obtain a target image corresponding to the image to be processed.

[0047] In another aspect, the present application provides an electronic device, comprising:

[0048] processor; and

[0049] The memory is used to store a program for the image processing method. After the device is powered on and the program for the image processing method is run by the processor, the following steps are performed:

[0050] Obtaining a first image corresponding to an image to be processed, and obtaining a second image corresponding to the image to be processed, wherein the first image and the second image are images obtained after image quality reduction processing is performed on the image to be processed, and image quality of the first image is different from image quality of the second image;

[0051] Obtaining image structure information corresponding to the image to be processed;

[0052] Image fusion processing is performed on the first image and the second image according to image structure information corresponding to the image to be processed to obtain a target image corresponding to the image to be processed.

[0053] In another aspect, the present application provides a storage medium storing a program for an image processing method, wherein the program is executed by a processor to perform the following steps: obtaining a first image corresponding to an image to be processed, and obtaining a second image corresponding to the image to be processed, wherein the first image and the second image are images obtained after image quality reduction processing is performed on the image to be processed, and the image quality of the first image is different from the image quality of the second image;

[0054] Obtaining image structure information corresponding to the image to be processed;

[0055] Image fusion processing is performed on the first image and the second image according to image structure information corresponding to the image to be processed to obtain a target image corresponding to the image to be processed.

[0056] Compared with the prior art, this application has the following advantages:

[0057] The image processing method provided in the present application first obtains a first image and a second image of different image quality corresponding to the image to be processed, then obtains image structure information corresponding to the image to be processed, and further performs image fusion processing on the first image and the second image according to the image structure information corresponding to the image to be processed to obtain a target image corresponding to the image to be processed. The image processing method provided in the present application adaptively fuses the first image and the second image of different quality intensities according to its own image structure information during the target image generation process, which can ensure the diversity of image structure information and image quality between different target images, thereby avoiding the problem of low authenticity of the target image caused by the relatively single image structure information and image quality between different target images when a globally unified generation method is adopted. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a first schematic diagram of an application scenario of the image processing method provided in this application.

[0059] Figure 2 This is a second schematic diagram of an application scenario of the image processing method provided in this application.

[0060] Figure 3 This is a flowchart of an image processing method provided in the first embodiment of the present application.

[0061] Figure 4 This is a flowchart of an image fusion method provided in the first embodiment of the present application.

[0062] Figure 5 This is a schematic diagram of an image processing device provided in the second embodiment of the present application.

[0063] Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.

[0065] In order to more clearly demonstrate the image processing method provided by the present application, the application scenario of the image processing method provided by the present application is first introduced. In practical applications, the image processing method provided in the embodiment of the present application is used to reduce the image quality of a high-quality image to obtain a low-quality image with higher image authenticity. Among them, the high-quality image and the low-quality image are images relative to a specified image quality threshold. If the image quality of the image reaches the specified image quality threshold, it is a high-quality image; if the image quality of the image does not reach the specified image quality threshold, it is a low-quality image. The so-called image quality is an evaluation of the visual effect of an image, which is used to characterize the clarity, distinguishability, etc. of the image.

[0066] In practical applications, the execution steps of the image processing method of the present application are as follows: Figure 1 As shown:

[0067] Step S101: Obtain an image to be processed. The so-called image to be processed is a high-quality image that needs to be processed and has an image quality higher than a specified image quality threshold.

[0068] Step S102-1: Obtain a first image corresponding to the image to be processed. The so-called first image is an image obtained after performing image quality reduction processing on the image to be processed. In the process of obtaining the first image, it is necessary to first determine the image quality of the first image, and then, based on the image quality of the first image, perform image quality reduction processing on the image to be processed to obtain the first image. The image quality reduction processing may include: adding interference information to the image to be processed, such as: adding noise to the image to be processed; the image quality reduction processing may also include: removing valid information from the image to be processed, such as: changing the pixel values ​​of some pixels in the image to be processed to reduce the clarity of some image areas.

[0069] Step S102-2: Obtain a second image corresponding to the image to be processed. The so-called second image is an image obtained after the image to be processed has been subjected to image quality reduction processing. In the process of obtaining the second image, it is necessary to first determine the image quality of the second image, and then, based on the image quality of the second image, perform image quality reduction processing on the image to be processed to obtain the second image.

[0070] Step S103: Obtain a target structure weight image. The so-called target structure weight image is a weight image used to characterize the image structure information corresponding to multiple target image regions, and the weights in the target structure weight image are target weights used to characterize the image structure information corresponding to multiple target image regions. The so-called image structure information corresponding to the image to be processed is an information set of the image edge feature information and the image clarity feature information of the image to be processed. The so-called image structure information corresponding to multiple target image regions is the image structure information of the regional image corresponding to each target image region after the image to be processed is divided into multiple target image regions. The so-called target structure weight image is a weight image corresponding to the image to be processed, which is generated by the weights used to characterize the image structure information corresponding to multiple target image regions.

[0071] Since the image structure information is a collection of information about the image edge feature information and the image clarity feature information of the image to be processed, and the target structure weighted image is a weighted image corresponding to the image to be processed, generated by weights used to represent the image structure information corresponding to multiple target image regions, the target image structure weighted image can simultaneously represent the image edge feature information and image clarity feature information corresponding to multiple target image regions. The weights in the target structure weighted image are used to represent the image structure information of each target image region in the multiple target image regions, that is, the weights in the target structure weighted image are used to represent the image edge feature information and image clarity feature information corresponding to each target image region in the multiple target image regions. The larger the weight in the target structure weighted image, the more obvious the image structure of each target image region, that is, the closer the regional image corresponding to each target image region is to the edge, and the clearer the regional image is.

[0072] In general, the process of obtaining the target structure weight image is as follows: Figure 2 As shown:

[0073] Step S103 - 1 : Obtain an image to be processed.

[0074] Step S103-2: Edge detection and blur detection: That is, edge detection and blur detection are performed on the image to be processed to obtain image edge feature information corresponding to multiple target image regions and image clarity feature information corresponding to multiple target image regions.

[0075] Step S103-3: Obtaining image structure information corresponding to the image to be processed, that is, obtaining image structure information corresponding to the multiple target image regions based on image edge feature information corresponding to the multiple target image regions and image clarity feature information corresponding to the multiple target image regions.

[0076] Step S103-4: Obtain target structure weight image.

[0077] First, a regional edge feature value is obtained to characterize the image edge feature information corresponding to each target image region, and a regional clarity feature value is obtained to characterize the image clarity feature information corresponding to each target image region. The regional edge feature value is obtained by uniformly assigning values ​​to the image edge feature information corresponding to multiple target region images, and is used to characterize the edge degree of the multiple target region images in the image to be processed. The larger the regional edge feature value, the closer the target region image corresponding to the regional edge feature value is to the edge of the image to be processed; the smaller the regional edge feature value, the closer the target region image corresponding to the regional edge feature value is to the center of the image to be processed. The regional clarity feature value is obtained by uniformly assigning values ​​to the image clarity feature information corresponding to multiple target region images, and is used to characterize the clarity of the multiple target region images. The larger the regional clarity feature value, the higher the clarity of the target region image corresponding to the regional clarity feature value; the smaller the regional clarity feature value, the lower the clarity of the target region image corresponding to the regional clarity feature value.

[0078] Next, based on the regional edge feature values ​​and the regional clarity feature values, image edge feature value images corresponding to the plurality of target image regions are determined, and image clarity feature value images corresponding to the plurality of target image regions are obtained. Since the regional edge feature values ​​and the regional clarity feature values ​​correspond to each target image region, when the image edge feature value images corresponding to the plurality of target image regions are determined based on the regional edge feature values ​​and the regional clarity feature values, and the image clarity feature value images corresponding to the plurality of target image regions are obtained, a feature value image having the same size as the image to be processed and divided into the same plurality of target regions can be generated.

[0079] Next, an initial structure weight image representing the image structural information corresponding to the multiple target image regions is determined based on the image edge eigenvalue image and the image clarity eigenvalue image. A Gaussian blur operation and image morphological operations are then performed on the initial structure weight image to obtain a second structure weight image representing the image structural information corresponding to the multiple target image regions. The Gaussian blur operation involves convolving the initial structure weight image with a normal distribution. The image morphological operation involves performing image processing such as dilation and erosion on the initial structure weight image.

[0080] Finally, the weights in the second structure weight image are normalized to obtain a target structure weight image for representing the image structure information corresponding to the multiple target image regions. Normalization is to normalize the weights in the second structure weight image to a value between 0 and 1.

[0081] Please refer to Figure 1Step S104: Image fusion. That is, image fusion processing is performed on the first image and the second image based on the target structure weight image. Specifically, based on the multiple target image regions, the first image is first divided into multiple first image regions corresponding to the multiple target image regions, and the second image is divided into multiple second image regions corresponding to the multiple target image regions. Then, based on the target weights, a weighted average processing is performed on the image structure information corresponding to the multiple first image regions and the image structure information corresponding to the multiple second image regions.

[0082] Step S105: Obtaining a target image corresponding to the image to be processed, that is, performing image fusion processing on the first image and the second image to obtain an image as the target image.

[0083] The embodiments of this application do not specifically limit the application scenarios of the image processing methods provided in the embodiments of this application. For example, the image processing methods provided in this application can be used in scenarios where the image processing methods are used to work independently with a server that provides services to a client, can be used in scenarios where the client works independently, and can be used in scenarios where the client interacts with the server. These scenarios will not be described in detail here. The so-called client is a computing device that has installed a program or software for executing the image processing methods provided in the embodiments of this application.

[0084] In addition, the image processing method provided in the embodiments of the present application can be specifically used in the fields of video applications, medical treatment, aerospace, and remote sensing technology. When the image processing method provided in the embodiments of the present application is applied to the field of video applications, it can be specifically used to process video frames, obtain video frames with lower image quality, and use them together with video frames with higher image quality as input for training the target video frame quality enhancement model to train the target video frame quality enhancement model. The so-called target video frame quality enhancement model is a model for performing video frame quality enhancement processing. After the target video frame quality enhancement model is trained, the target video frame quality enhancement model can be applied to video applications to enhance the image quality of video frames, thereby improving the clarity of video playback. When the image processing method provided in the embodiments of the present application is applied to the medical field, it can be specifically used to process medical images, obtain medical images with lower image quality, and use them together with medical images with higher image quality as input for training the target medical image quality enhancement model to train the target medical image quality enhancement model. The so-called target medical image quality enhancement model is a model for performing medical image quality enhancement processing. After the target medical image quality enhancement model is trained, the target medical image quality enhancement model can be applied to enhance the image quality of medical images, thereby improving the clarity of medical images. When the image processing method provided in the embodiments of this application is applied to the aerospace field, it can be specifically used to process satellite remote sensing images, obtain satellite remote sensing images of lower image quality, and use these images, along with satellite remote sensing images of higher image quality, as input for training a target satellite remote sensing image quality enhancement model. The so-called target satellite remote sensing image quality enhancement model is a model used for performing satellite remote sensing image quality enhancement processing. After the target satellite remote sensing image quality enhancement model is trained, it can be applied to enhance the image quality of satellite remote sensing images, thereby improving the clarity of satellite remote sensing images.

[0085] It should be noted that the image processing method in the embodiment of the present application can specifically adopt a SaaS (Software-as-a-Service) service mode to provide image processing services to relevant users, that is, to use the software installed with the image processing method provided in the embodiment of the present application through the network to provide image processing services to relevant users.

[0086] The application scenarios of the above-mentioned image processing method are provided to facilitate understanding of the image processing method provided by this application, and are not used to limit the image processing method provided by this application.

[0087] First embodiment

[0088] The first embodiment provides an image processing method, which is as follows: Figure 3and Figure 4 Provide explanation.

[0089] Figure 3 This is a flowchart of an image processing method provided in the first embodiment of the present application. Figure 3 The image processing method shown includes: steps S301 to S303.

[0090] In step S301, a first image corresponding to the image to be processed is obtained, and a second image corresponding to the image to be processed is obtained. The first image and the second image are images obtained after image quality reduction processing is performed on the image to be processed, and the image quality of the first image is different from the image quality of the second image.

[0091] In the first embodiment of the present application, the image to be processed is a pre-selected image for image processing, which may be a video frame. The so-called image quality reduction process is an image processing process in which a low-quality image corresponding to the high-quality image is obtained after a series of processes are performed on the high-quality image. Specifically, the steps of obtaining a first image corresponding to the image to be processed and obtaining a second image corresponding to the image to be processed are as follows: first, obtaining the image to be processed. Then, determining the image quality of the first image and determining the image quality of the second image. Finally, based on the image quality of the first image, image quality reduction processing is performed on the image to be processed to obtain the first image, and based on the image quality of the second image, image quality reduction processing is performed on the image to be processed to obtain the second image. The so-called high-quality image and low-quality image are images relative to a specified image quality threshold. An image is a high-quality image if its image quality reaches the specified image quality threshold, and an image is a low-quality image if its image quality does not reach the specified image quality threshold. The so-called image quality is an evaluation of the visual effect of an image, which is used to characterize the clarity, distinguishability, etc. of an image.

[0092] Reducing the image quality of the image to be processed based on the image quality of the first image to obtain the first image includes: adding interference information to the image to be processed based on the image quality of the first image to obtain the first image. Reducing the image quality of the image to be processed based on the image quality of the first image to obtain the first image also includes: removing valid information from the image to be processed based on the degree of quality degradation of the first image to obtain the first image. Adding interference information to the image to be processed can be achieved by adding noise to the image to be processed, for example; removing valid information from the image to be processed can be achieved by changing the pixel values ​​of some pixels in the image to be processed to reduce the clarity of some image areas, for example.

[0093] In step S302, image structure information corresponding to the image to be processed is obtained.

[0094] In the first embodiment of the present application, a specific implementation method for obtaining image structure information corresponding to the image to be processed is: dividing the image to be processed into multiple target image areas, and obtaining image structure information corresponding to the multiple target image areas.

[0095] The image structure information corresponding to the image to be processed is a collection of information including the image edge feature information and the image clarity feature information of the image to be processed. The image structure information corresponding to the multiple target image regions is the image structure information of the regional images corresponding to each target image region after the image to be processed is divided into the multiple target image regions.

[0096] In the first embodiment of the present application, the process of obtaining image structure information corresponding to multiple target image regions is as follows: first, edge detection and blur detection are performed on the image to be processed to obtain image edge feature information corresponding to the multiple target image regions, and image clarity feature information corresponding to the multiple target image regions. Then, based on the image edge feature information corresponding to the multiple target image regions and the image clarity feature information corresponding to the multiple target image regions, image structure information corresponding to the multiple target image regions is obtained.

[0097] In step S303, image fusion processing is performed on the first image and the second image according to the image structure information corresponding to the image to be processed, so as to obtain a target image corresponding to the image to be processed.

[0098] In the first embodiment of the present application, a target image corresponding to a to-be-processed image is obtained by first obtaining target weights for representing image structural information corresponding to a plurality of target image regions. Then, based on the target weights, image fusion processing is performed on the first image and the second image to obtain the target image corresponding to the to-be-processed image.

[0099] In the process of obtaining the target weight, it is necessary to first obtain the regional weight used to characterize the image structure information corresponding to each target image area based on the image edge feature information corresponding to each target image area in multiple target image areas and the image clarity feature information corresponding to each target image area, and then obtain the target weight based on the regional weight.

[0100] In the first embodiment of the present application, the method for obtaining a regional weight for representing image structure information corresponding to each target image region is as follows: first, a regional edge feature value for representing image edge feature information corresponding to each target image region is obtained, and a regional clarity feature value for representing image clarity feature information corresponding to each target image region is obtained. Then, a regional weight is determined based on the regional edge feature value and the regional clarity feature value.

[0101] The so-called regional edge feature value is obtained by uniformly assigning values ​​to the image edge feature information corresponding to multiple target region images, and is used to characterize the edge degree of the multiple target region images in the image to be processed. The larger the regional edge feature value, the closer the target region image corresponding to the regional edge feature value is to the edge position in the image to be processed; the smaller the regional edge feature value, the closer the target region image corresponding to the regional edge feature value is to the center position in the image to be processed. The so-called regional clarity feature value for characterizing the image clarity feature information corresponding to each target image region is obtained by uniformly assigning values ​​to the image clarity feature information corresponding to multiple target region images, and is used to characterize the clarity of the multiple target region images. The larger the regional clarity feature value, the higher the clarity of the target region image corresponding to the regional clarity feature value; the smaller the regional clarity feature value, the lower the clarity of the target region image corresponding to the regional clarity feature value.

[0102] It should be noted that in the first embodiment of the present application, when obtaining the target weight based on the regional weight, it is necessary to first determine the image edge feature value images corresponding to the multiple target image regions based on the regional edge feature value and the regional clarity feature value, and obtain the image clarity feature value images corresponding to the multiple target image regions; then, determine the target weight based on the image edge feature value image and the image clarity feature value image. Since the regional edge feature value and the regional clarity feature value correspond to each target image region, when determining the image edge feature value images corresponding to the multiple target image regions based on the regional edge feature value and the regional clarity feature value, and obtaining the image clarity feature value images corresponding to the multiple target image regions, it is possible to generate a feature value image that is the same size as the image to be processed and is divided into the same multiple target regions.

[0103] In the first embodiment of the present application, the target weights are determined based on the image edge eigenvalue image and the image clarity eigenvalue image as follows: first, an initial image structure weight image for characterizing the image structure information corresponding to multiple target image regions is determined based on the image edge eigenvalue image and the image clarity eigenvalue image. Then, a Gaussian blur operation and an image morphological operation are performed on the initial image structure weight image to obtain a second image structure weight image for characterizing the image structure information corresponding to multiple target image regions. Finally, the weights in the second image structure weight image are normalized to obtain a target structure weight image for characterizing the image structure information corresponding to multiple target image regions, and the weights in the target structure weight image are used as target weights.

[0104] The so-called regional edge feature value is obtained by uniformly assigning values ​​to the image edge feature information corresponding to multiple target region images, and is used to characterize the edge degree of the multiple target region images in the image to be processed. The larger the regional edge feature value, the closer the target region image corresponding to the regional edge feature value is to the edge position in the image to be processed; the smaller the regional edge feature value, the closer the target region image corresponding to the regional edge feature value is to the center position in the image to be processed. The so-called regional clarity feature value for characterizing the image clarity feature information corresponding to each target image region is obtained by uniformly assigning values ​​to the image clarity feature information corresponding to multiple target region images, and is used to characterize the clarity of the multiple target region images. The larger the regional clarity feature value, the higher the clarity of the target region image corresponding to the regional clarity feature value; the smaller the regional clarity feature value, the lower the clarity of the target region image corresponding to the regional clarity feature value.

[0105] The so-called Gaussian blur operation is to perform a convolution operation on the initial structure weight image and the normal distribution. The so-called image morphology operation is to perform image processing such as dilation and erosion on the initial structure weight image. The so-called normalization is to normalize the weights in the second structure weight image to between 0 and 1.

[0106] In the first embodiment of the present application, after obtaining the target weight, the process of performing image fusion processing on the first image and the second image according to the target weight is as follows: Figure 4 , which is a flowchart of an image fusion method provided in the first embodiment of the present application.

[0107] Step S303 - 1 : According to the multiple target image areas, the first image is divided into multiple first image areas corresponding to the multiple target image areas, and the second image is divided into multiple second image areas corresponding to the multiple target image areas.

[0108] Step S303 - 2 : performing weighted averaging processing on the image structure information corresponding to the plurality of first image regions and the image structure information corresponding to the plurality of second image regions according to the target weight, to obtain a target image.

[0109] The image processing method provided in the first embodiment of the present application first obtains a first image and a second image of different image quality corresponding to the image to be processed, then obtains image structure information corresponding to the image to be processed, and further performs image fusion processing on the first image and the second image according to the image structure information corresponding to the image to be processed to obtain a target image corresponding to the image to be processed. The image processing method provided in the first embodiment of the present application adaptively fuses the first image and the second image of different quality intensities according to its own image structure information during the target image generation process, and can ensure the diversity of image structure information and image quality between different target images, thereby avoiding the problem of low authenticity of the target image caused by the relatively single image structure information and image quality between different target images when a globally unified generation method is adopted.

[0110] It should be noted that in the first embodiment of the present application, after obtaining the target image, a sample image to be processed for training the image quality enhancement model can be further obtained, and the sample image to be processed and the target image are used as inputs to the image quality enhancement model to train the image quality enhancement model. The so-called image quality enhancement model is a model used for performing image quality enhancement processing.

[0111] Since the target image in the first embodiment of the present application has a high degree of accuracy and the image structure information is relatively diverse, the sample image to be processed and the target image are used as inputs of the image quality enhancement model to train the image quality enhancement model, which can improve the generalization and effect of the trained image enhancement model.

[0112] Second embodiment

[0113] Corresponding to the application scenario of the image processing method provided by this application and the image processing method provided by the first embodiment, the second embodiment of this application also provides an image processing device. Since the device embodiment is basically similar to the application scenario and the first embodiment, the description is relatively simple. For relevant details, please refer to the application scenario and the partial description of the first embodiment. The device embodiment described below is merely illustrative.

[0114] Please refer to Figure 5 , which is a schematic diagram of an image processing device provided in the second embodiment of the present application.

[0115] The image processing device provided in the second embodiment of the present application includes:

[0116] A first image processing unit 501 is configured to obtain a first image corresponding to an image to be processed, and obtain a second image corresponding to the image to be processed, wherein the first image and the second image are images obtained by performing image quality reduction processing on the image to be processed, and the image quality of the first image is different from the image quality of the second image;

[0117] The second image processing unit 502 is configured to obtain image structure information corresponding to the image to be processed;

[0118] The third image processing unit 503 is configured to perform image fusion processing on the first image and the second image according to image structure information corresponding to the image to be processed, so as to obtain a target image corresponding to the image to be processed.

[0119] Optionally, the second image processing unit 502 is specifically configured to divide the image to be processed into a plurality of target image regions; and obtain image structure information corresponding to the plurality of target image regions.

[0120] Optionally, the third image processing unit 503 is specifically used to obtain target weights for characterizing image structure information corresponding to the multiple target image areas; and perform image fusion processing on the first image and the second image according to the target weights to obtain a target image corresponding to the image to be processed.

[0121] Optionally, obtaining target weights for characterizing image structure information corresponding to the multiple target image regions includes:

[0122] obtaining, based on the image edge feature information corresponding to each target image region and the image clarity feature information corresponding to each target image region, a region weight for characterizing the image structure information corresponding to each target image region;

[0123] The target weight is obtained according to the region weight.

[0124] Optionally, obtaining, based on the image edge feature information corresponding to each target image region and the image clarity feature information corresponding to each target image region, a region weight for characterizing the image structure information corresponding to each target image region among the multiple target image regions includes:

[0125] Obtaining a regional edge feature value for characterizing image edge feature information corresponding to each target image region, and obtaining a regional clarity feature value for characterizing image clarity feature information corresponding to each target image region;

[0126] The region weight is determined according to the region edge feature value and the region clarity feature value.

[0127] Optionally, obtaining the target weight according to the region weight includes:

[0128] Determining image edge feature value images corresponding to the plurality of target image regions according to the region edge feature values ​​and the region clarity feature values, and obtaining image clarity feature value images corresponding to the plurality of target image regions;

[0129] The target weight is determined according to the image edge eigenvalue image and the image clarity eigenvalue image.

[0130] Optionally, determining the target weight according to the image edge eigenvalue image and the image clarity eigenvalue image includes:

[0131] determining, based on the image edge eigenvalue image and the image clarity eigenvalue image, an initial image structure weight image for characterizing image structure information corresponding to the plurality of target image regions;

[0132] Performing a Gaussian blur operation and an image morphological operation on the initial image structure weighted image to obtain a second image structure weighted image for representing image structure information corresponding to the plurality of target image regions;

[0133] The weights in the second image structure weight image are normalized to obtain a target structure weight image for characterizing the image structure information corresponding to the multiple target image areas, and the weights in the target structure weight image are used as the target weights.

[0134] Optionally, obtaining the image structure information corresponding to the multiple target image regions includes:

[0135] Performing edge detection and blur detection on the image to be processed to obtain image edge feature information corresponding to the multiple target image regions, and obtaining image clarity feature information corresponding to the multiple target image regions;

[0136] Image structure information corresponding to the multiple target image regions is obtained based on the image edge feature information corresponding to the multiple target image regions and the image clarity feature information corresponding to the multiple target image regions.

[0137] Optionally, performing image fusion processing on the first image and the second image according to the target weight to obtain a target image corresponding to the image to be processed includes:

[0138] According to the multiple target image areas, dividing the first image into multiple first image areas corresponding to the multiple target image areas, and dividing the second image into multiple second image areas corresponding to the multiple target image areas;

[0139] According to the target weight, weighted averaging processing is performed on the image structure information corresponding to the multiple first image regions and the image structure information corresponding to the multiple second image regions to obtain the target image.

[0140] Optionally, the first image processing unit 501 is specifically used to obtain the image to be processed; determine the image quality of the first image, and determine the image quality of the second image; perform image quality reduction processing on the image to be processed according to the image quality of the first image to obtain the first image, and perform image quality reduction processing on the image to be processed according to the image quality of the second image to obtain the second image.

[0141] Optionally, performing image quality reduction processing on the image to be processed according to the image quality of the first image to obtain the first image includes: adding interference information to the image to be processed according to the image quality of the first image to obtain the first image.

[0142] Optionally, performing image quality reduction processing on the image to be processed based on the image quality of the first image to obtain the first image includes: removing valid information of the image to be processed based on the degree of quality reduction of the first image to obtain the first image.

[0143] Optionally, the image processing device provided in the second embodiment of the present application further includes:

[0144] a fourth image obtaining unit, configured to obtain a sample image to be processed for training an image quality enhancement model, wherein the image quality enhancement model is a model for performing image quality enhancement processing;

[0145] The model training acquisition unit is used to use the sample image to be processed and the target image as inputs of the image quality enhancement model to train the image quality enhancement model.

[0146] Third embodiment

[0147] Corresponding to the application scenario of the image processing method provided by this application and the image processing method provided by the first embodiment, the third embodiment of this application also provides an electronic device. Since the third embodiment is basically similar to the application scenario and the first embodiment, the description is relatively simple. For relevant details, please refer to the application scenario and the partial description of the first embodiment. The device embodiment described below is merely illustrative.

[0148] Please refer to Figure 6 , which is a schematic diagram of an electronic device provided in an embodiment of the present application.

[0149] The electronic device includes: a processor 601;

[0150] and a memory 602 for storing a program of the image processing method. After the device is powered on and the program of the image processing method is run by the processor, the following steps are performed:

[0151] Obtaining a first image corresponding to an image to be processed, and obtaining a second image corresponding to the image to be processed, wherein the first image and the second image are images obtained after image quality reduction processing is performed on the image to be processed, and image quality of the first image is different from image quality of the second image;

[0152] Obtaining image structure information corresponding to the image to be processed;

[0153] Image fusion processing is performed on the first image and the second image according to image structure information corresponding to the image to be processed to obtain a target image corresponding to the image to be processed.

[0154] It should be noted that the detailed description of the electronic device provided in the third embodiment of the present application can refer to the application scenarios of the image processing method provided in the present application and the relevant description of the image processing method provided in the first embodiment, which will not be repeated here.

[0155] Fourth embodiment

[0156] Corresponding to the application scenario of the image processing method provided by this application and the image processing method provided by the first embodiment, the fourth embodiment of this application also provides a storage medium. Since the fourth embodiment is basically similar to the application scenario and the first embodiment, the description is relatively simple. For relevant details, please refer to the application scenario and the partial description of the first embodiment. The device embodiment described below is merely illustrative.

[0157] The storage medium stores a computer program, which is executed by a processor to perform the following steps:

[0158] Obtaining a first image corresponding to an image to be processed, and obtaining a second image corresponding to the image to be processed, wherein the first image and the second image are images obtained after image quality reduction processing is performed on the image to be processed, and image quality of the first image is different from image quality of the second image;

[0159] Obtaining image structure information corresponding to the image to be processed;

[0160] Image fusion processing is performed on the first image and the second image according to image structure information corresponding to the image to be processed to obtain a target image corresponding to the image to be processed.

[0161] It should be noted that the detailed description of the storage medium provided in the fourth embodiment of the present application can refer to the application scenarios of the image processing method provided in the present application and the relevant description of the image processing method provided in the first embodiment, and will not be repeated here.

[0162] Although the present application is disclosed as above with reference to preferred embodiments, it is not intended to limit the present application. Any person skilled in the art may make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0163] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0164] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0165] 1. Computer-readable media, including permanent and non-permanent, removable and non-removable media, can be implemented using any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmitting media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0166] 2. Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. An image processing method, characterized in that: include: Obtaining a first image corresponding to an image to be processed, and obtaining a second image corresponding to the image to be processed, wherein the first image and the second image are images obtained after image quality reduction processing is performed on the image to be processed, and image quality of the first image is different from image quality of the second image; Obtaining image structure information corresponding to the image to be processed, wherein the image structure information corresponding to the image to be processed is used to represent image structure information corresponding to multiple target image regions, and the image structure information includes image edge feature information and image clarity feature information; performing image fusion processing on the first image and the second image according to image structure information corresponding to the image to be processed, to obtain a target image corresponding to the image to be processed; Among them, according to the image structure information corresponding to the image to be processed, image fusion processing is performed on the first image and the second image to obtain the target image corresponding to the image to be processed, including: according to the target weight of the image structure information corresponding to the multiple target image areas, image fusion processing is performed on the first image and the second image to obtain the target image corresponding to the image to be processed, wherein the target weight is determined based on the area weight, and the area weight is obtained by the area edge feature value and the area clarity feature value, the area edge feature value is used to characterize the image edge feature information corresponding to each target image area in the multiple target image areas, and the area clarity feature value is used to characterize the image clarity feature information corresponding to each target image area in the multiple target image areas.

2. The image processing method according to claim 1, wherein: The obtaining of image structure information corresponding to the image to be processed includes: Dividing the image to be processed into the plurality of target image areas; Image structure information corresponding to the multiple target image regions is obtained.

3. The image processing method according to claim 2, wherein: Performing image fusion processing on the first image and the second image according to target weights of the image structure information corresponding to the multiple target image regions to obtain a target image corresponding to the image to be processed, including: Obtaining target weights for characterizing image structure information corresponding to the plurality of target image regions; According to the target weight, image fusion processing is performed on the first image and the second image to obtain a target image corresponding to the image to be processed.

4. The image processing method according to claim 3, wherein: The obtaining of target weights for characterizing image structure information corresponding to the plurality of target image regions includes: obtaining, based on the image edge feature information corresponding to each target image region and the image clarity feature information corresponding to each target image region, a region weight for characterizing the image structure information corresponding to each target image region; The target weight is obtained according to the region weight.

5. The image processing method according to claim 4, characterized in that The obtaining, based on the image edge feature information corresponding to each target image region and the image clarity feature information corresponding to each target image region, a region weight for characterizing the image structure information corresponding to each target image region includes: Obtaining a regional edge feature value for characterizing the image edge feature information corresponding to each target image region, and obtaining a regional clarity feature value for characterizing the image clarity feature information corresponding to each target image region; determining the regional weight according to the regional edge feature value and the regional clarity feature value.

6. The image processing method according to claim 5, characterized in that The obtaining of the target weight according to the region weight includes: Determining image edge feature value images corresponding to the plurality of target image regions according to the region edge feature values ​​and the region clarity feature values, and obtaining image clarity feature value images corresponding to the plurality of target image regions; The target weight is determined according to the image edge eigenvalue image and the image clarity eigenvalue image.

7. The image processing method according to claim 6, characterized in that: The determining the target weight according to the image edge eigenvalue image and the image clarity eigenvalue image includes: determining, based on the image edge eigenvalue image and the image clarity eigenvalue image, an initial image structure weight image for characterizing image structure information corresponding to the plurality of target image regions; Performing a Gaussian blur operation and an image morphological operation on the initial image structure weighted image to obtain a second image structure weighted image for representing image structure information corresponding to the plurality of target image regions; The weights in the second image structure weight image are normalized to obtain a target structure weight image for characterizing the image structure information corresponding to the multiple target image areas, and the weights in the target structure weight image are used as the target weights.

8. The image processing method according to claim 2, wherein: The obtaining of image structure information corresponding to the plurality of target image regions includes: Performing edge detection and blur detection on the image to be processed to obtain image edge feature information corresponding to the multiple target image regions, and obtaining image clarity feature information corresponding to the multiple target image regions; Image structure information corresponding to the multiple target image regions is obtained based on the image edge feature information corresponding to the multiple target image regions and the image clarity feature information corresponding to the multiple target image regions.

9. The image processing method according to claim 3, wherein: The performing image fusion processing on the first image and the second image according to the target weight to obtain a target image corresponding to the image to be processed includes: According to the multiple target image areas, dividing the first image into multiple first image areas corresponding to the multiple target image areas, and dividing the second image into multiple second image areas corresponding to the multiple target image areas; According to the target weight, weighted averaging processing is performed on the image structure information corresponding to the multiple first image regions and the image structure information corresponding to the multiple second image regions to obtain the target image.

10. The image processing method according to claim 1, wherein: The obtaining of a first image corresponding to the image to be processed and obtaining a second image corresponding to the image to be processed includes: Obtaining the image to be processed; determining an image quality of the first image, and determining an image quality of the second image; According to the image quality of the first image, the image to be processed is subjected to image quality reduction processing to obtain the first image, and according to the image quality of the second image, the image to be processed is subjected to image quality reduction processing to obtain the second image.

11. The image processing method according to claim 10, wherein: The step of performing image quality reduction processing on the image to be processed according to the image quality of the first image to obtain the first image includes: According to the image quality of the first image, interference information is added to the image to be processed to obtain the first image.

12. The image processing method according to claim 10, wherein: The step of performing image quality reduction processing on the image to be processed according to the image quality of the first image to obtain the first image includes: According to the degree of quality degradation of the first image, valid information of the image to be processed is removed to obtain the first image.

13. The image processing method according to claim 1, wherein: Also includes: Obtaining a sample image to be processed for training an image quality enhancement model, wherein the image quality enhancement model is a model for performing image quality enhancement processing; The sample image to be processed and the target image are used as inputs of the image quality enhancement model to train the image quality enhancement model.

14. An image processing device, characterized in that: include: a first image processing unit, configured to obtain a first image corresponding to an image to be processed, and obtain a second image corresponding to the image to be processed, wherein the first image and the second image are images obtained by performing image quality reduction processing on the image to be processed, and the image quality of the first image is different from the image quality of the second image; a second image processing unit, configured to obtain image structure information corresponding to the image to be processed, wherein the image structure information corresponding to the image to be processed is used to represent image structure information corresponding to multiple target image regions, and the image structure information includes image edge feature information and image clarity feature information; a third image processing unit, configured to perform image fusion processing on the first image and the second image according to image structure information corresponding to the image to be processed, so as to obtain a target image corresponding to the image to be processed; Among them, according to the image structure information corresponding to the image to be processed, image fusion processing is performed on the first image and the second image to obtain the target image corresponding to the image to be processed, including: according to the target weight of the image structure information corresponding to the multiple target image areas, image fusion processing is performed on the first image and the second image to obtain the target image corresponding to the image to be processed, wherein the target weight is determined based on the area weight, and the area weight is obtained by the area edge feature value and the area clarity feature value, the area edge feature value is used to characterize the image edge feature information corresponding to each target image area in the multiple target image areas, and the area clarity feature value is used to characterize the image clarity feature information corresponding to each target image area in the multiple target image areas.

15. An electronic device, characterized in that: include: processor; and a memory for storing a program for an image processing method. After the device is powered on and the program for the image processing method is run by the processor, the following steps are performed: Obtaining a first image corresponding to an image to be processed, and obtaining a second image corresponding to the image to be processed, wherein the first image and the second image are images obtained after image quality reduction processing is performed on the image to be processed, and image quality of the first image is different from image quality of the second image; Obtaining image structure information corresponding to the image to be processed, wherein the image structure information corresponding to the image to be processed is used to represent image structure information corresponding to multiple target image regions, and the image structure information includes image edge feature information and image clarity feature information; performing image fusion processing on the first image and the second image according to image structure information corresponding to the image to be processed, to obtain a target image corresponding to the image to be processed; Wherein, performing image fusion processing on the first image and the second image according to the image structure information corresponding to the image to be processed to obtain a target image corresponding to the image to be processed includes: Based on the target weights of the image structure information corresponding to the multiple target image areas, image fusion processing is performed on the first image and the second image to obtain the target image corresponding to the image to be processed, wherein the target weight is determined based on the area weight, and the area weight is obtained through the area edge feature value and the area clarity feature value, the area edge feature value is used to characterize the image edge feature information corresponding to each target image area in the multiple target image areas, and the area clarity feature value is used to characterize the image clarity feature information corresponding to each target image area in the multiple target image areas.

16. A storage medium, characterized in that A program for an image processing method is stored, and the program is executed by a processor to perform the following steps: Obtaining a first image corresponding to an image to be processed, and obtaining a second image corresponding to the image to be processed, wherein the first image and the second image are images obtained after image quality reduction processing is performed on the image to be processed, and image quality of the first image is different from image quality of the second image; Obtaining image structure information corresponding to the image to be processed, wherein the image structure information corresponding to the image to be processed is used to represent image structure information corresponding to multiple target image regions, and the image structure information includes image edge feature information and image clarity feature information; performing image fusion processing on the first image and the second image according to image structure information corresponding to the image to be processed, to obtain a target image corresponding to the image to be processed; Among them, according to the image structure information corresponding to the image to be processed, image fusion processing is performed on the first image and the second image to obtain the target image corresponding to the image to be processed, including: according to the target weight of the image structure information corresponding to the multiple target image areas, image fusion processing is performed on the first image and the second image to obtain the target image corresponding to the image to be processed, wherein the target weight is determined based on the area weight, and the area weight is obtained by the area edge feature value and the area clarity feature value, the area edge feature value is used to characterize the image edge feature information corresponding to each target image area in the multiple target image areas, and the area clarity feature value is used to characterize the image clarity feature information corresponding to each target image area in the multiple target image areas.

Images