Image processing method, device, electronic device and storage medium

Abstract

The present disclosure provides an image processing method, apparatus, electronic device, and storage medium, relating to the field of image processing technology, particularly artificial intelligence and deep learning. A specific implementation scheme comprises: acquiring a first image and downsampling the first image to obtain a second image; acquiring a first adjustment coefficient and a second adjustment coefficient based on the second image; acquiring a third adjustment coefficient and a fourth adjustment coefficient based on the first adjustment coefficient and the second adjustment coefficient; and acquiring a target image based on the first image, the third adjustment coefficient, and the fourth adjustment coefficient.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, specifically to the fields of artificial intelligence and deep learning, and more particularly to an image processing method, device, electronic device, and storage medium. Background Art

[0002] Image beautification technology refers to the editing operation technology related to beautification of a given image. Currently, there are many image beautification technologies, including but not limited to image enhancement, image filters, image sharpening and image filtering. In addition, image differentiation can be performed based on neural networks and deep learning, as well as image beautification operations based on low-resolution image processing and then extended to high-resolution images.

[0003] Beautification methods based on extending low-resolution image processing to high-resolution images can reduce running time and avoid image artifacts, and have better image processing effects; how to extend low-resolution image processing operations to high-resolution images and achieve consistency between high-resolution image effects and low-resolution image effects is a challenge that needs to be solved urgently. Summary of the Invention

[0004] The present disclosure provides an image processing method, apparatus, electronic device, and storage medium.

[0005] According to a first aspect of the present disclosure, there is provided an image processing method, comprising:

[0006] Acquire a first image, and downsample the first image to obtain a second image;

[0007] Based on the second image, obtaining a first adjustment coefficient and a second adjustment coefficient;

[0008] Obtaining a third adjustment coefficient and a fourth adjustment coefficient according to the first adjustment coefficient and the second adjustment coefficient;

[0009] A target image is acquired according to the first image, the third adjustment coefficient, and the fourth adjustment coefficient.

[0010] According to a second aspect of the present disclosure, another image processing method is provided, comprising:

[0011] Acquire a first image, and downsample the first image to obtain a second image;

[0012] Inputting the second image into a pre-trained target image processing model, and having the target image processing model output a third adjustment coefficient and a fourth adjustment coefficient;

[0013] The first image is input into the target image processing model, and the first image is processed in the target image processing model based on the third adjustment coefficient and the fourth adjustment coefficient to obtain the target image.

[0014] According to a third aspect of the present disclosure, a method for training an image processing model is provided, comprising:

[0015] Acquire a first sample image, and downsample the first sample image at least once to obtain sample images at different resolutions;

[0016] Acquire high-frequency features based on a sample image with minimum resolution, and input the sample image with minimum resolution and the high-frequency features into an image processing model to obtain a first coefficient combination, wherein the first coefficient combination includes a first sample adjustment coefficient and a second sample adjustment coefficient;

[0017] processing the sample images at different resolutions according to the first coefficient combination to obtain adjusted images corresponding to the sample images at the respective resolutions;

[0018] According to the high-frequency features and the adjusted images corresponding to the sample images at various resolutions, the image processing model is adjusted and training is continued until a target image processing model is obtained.

[0019] According to a fourth aspect of the present disclosure, there is provided an image processing apparatus, comprising:

[0020] An image acquisition module, configured to acquire a first image and downsample the first image to obtain a second image;

[0021] A coefficient acquisition module, configured to acquire a first adjustment coefficient and a second adjustment coefficient based on the second image; and acquire a third adjustment coefficient and a fourth adjustment coefficient according to the first adjustment coefficient and the second adjustment coefficient;

[0022] An image processing module is configured to obtain a target image according to the first image, the third adjustment coefficient, and the fourth adjustment coefficient.

[0023] According to a fifth aspect of the present disclosure, another image processing apparatus is provided, comprising:

[0024] A first acquisition module is configured to acquire a first image and downsample the first image to obtain a second image;

[0025] a second acquisition module, configured to input the second image into a pre-trained target image processing model, and have the target image processing model output a third adjustment coefficient and a fourth adjustment coefficient;

[0026] A processing module is used to input the first image into the target image processing model, and process the first image based on the third adjustment coefficient and the fourth adjustment coefficient in the target image processing model to obtain the target image.

[0027] According to a sixth aspect of the present disclosure, there is provided a training device for an image processing model, comprising:

[0028] a sample acquisition module, configured to acquire a first sample image and downsample the first sample image at least once to obtain sample images at different resolutions;

[0029] a coefficient combination acquisition module, configured to acquire high-frequency features based on a sample image with minimum resolution, and input the sample image with minimum resolution and the high-frequency features into an image processing model to obtain a first coefficient combination, wherein the first coefficient combination includes a first sample adjustment coefficient and a second sample adjustment coefficient;

[0030] a beautification module, configured to process sample images at different resolutions according to the first coefficient combination to obtain adjusted images corresponding to the sample images at each resolution;

[0031] The training module is used to adjust the image processing model according to the high-frequency features and the adjustment images corresponding to the sample images at various resolutions and continue training until a target image processing model is obtained.

[0032] According to a seventh aspect of the present disclosure, there is provided an electronic device, including:

[0033] at least one processor; and

[0034] a memory communicatively connected to the at least one processor; wherein,

[0035] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of the first aspect, the second aspect, or the third aspect.

[0036] According to an eighth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in any one of the first aspect, the second aspect or the third aspect.

[0037] According to a ninth aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements the steps of the method of any one of the first aspect, the second aspect or the third aspect when executed by a processor.

[0038] The disclosed embodiments have at least the following beneficial effects: a second image is obtained by downsampling the first image to be beautified, a first adjustment coefficient and a second adjustment coefficient are obtained based on the second image, the beautification effect in the second image is reflected by the first adjustment image and the second adjustment image, and then upsampling is performed based on the first adjustment coefficient and the second adjustment coefficient to obtain a third adjustment coefficient and a fourth adjustment coefficient, the low-resolution adjustment coefficient is expanded to obtain a high-resolution adjustment coefficient, and then combined with the first image for processing, to ensure that the beautification effect of the low-resolution image can be effectively extended to the high-resolution image, thereby achieving an ideal high-resolution beautification effect.

[0039] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention.

[0041] Figure 1 A schematic diagram of an image processing method provided by an embodiment of the present disclosure;

[0042] Figure 2 A schematic diagram of another image processing method provided by an embodiment of the present disclosure;

[0043] Figure 3 A schematic diagram of another image processing method provided by an embodiment of the present disclosure;

[0044] Figure 4 A logic diagram of an image processing method provided by an embodiment of the present disclosure;

[0045] Figure 5 A schematic diagram of another image processing method provided by an embodiment of the present disclosure;

[0046] Figure 6 A schematic diagram of a training method for an image processing model provided in an embodiment of the present disclosure;

[0047] Figure 7 A schematic diagram of another image processing model training method provided by an embodiment of the present disclosure;

[0048] Figure 8 A logic diagram of a training method for an image processing model provided in an embodiment of the present disclosure;

[0049] Figure 9 A schematic structural diagram of an image processing device provided by an embodiment of the present disclosure;

[0050] Figure 10A schematic structural diagram of another image processing device provided by an embodiment of the present disclosure;

[0051] Figure 11 A schematic diagram of the structure of a training device for an image processing model provided in an embodiment of the present disclosure;

[0052] Figure 12 is a block diagram of an electronic device for implementing the image processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0053] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0054] Image processing is a technology that uses computers to analyze images to achieve the desired results. It generally refers to digital image processing. A digital image refers to a large two-dimensional array obtained by shooting with an industrial camera, video camera, scanner or other equipment. The elements of this array are called pixels. Image processing technology generally includes image compression, enhancement, restoration, matching, description and recognition.

[0055] Artificial Intelligence (AI) is an interdisciplinary subject based on computer science and integrating multiple disciplines such as computer science, psychology, and philosophy. It studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems.

[0056] Deep Learning (DL) is a new research direction in the field of machine learning. Deep learning is the inherent laws and representation levels of learning sample data. The ultimate goal is to enable machines to have analytical learning capabilities like humans and be able to recognize data such as text, images, and sounds.

[0057] Figure 1 This is a schematic diagram of an image processing method provided by an embodiment of the present disclosure. Figure 1 As shown, the method includes the following steps:

[0058] S101: Acquire a first image and downsample the first image to obtain a second image.

[0059] In some implementations, the first image is randomly extracted from a high-resolution image to be beautified, where a high-resolution image refers to an image with a higher pixel density, a larger number of pixels, and a clearer and more delicate image; the first image is downsampled, that is, the high-resolution image to be beautified is downsampled, and pixels in the high-resolution image are extracted to reduce the number of pixels to obtain a second image, which is a low-resolution image, that is, an image with low pixel density, lower clarity and detail.

[0060] S102: Acquire a first adjustment coefficient and a second adjustment coefficient based on the second image.

[0061] In some implementations, the adjustment coefficient may be a coefficient for performing operations such as enhancement, sharpening, noise reduction, or filtering on the second image, and is mainly used to perform beautification processing operations on the second image.

[0062] In some implementations, the first adjustment coefficient and the second adjustment coefficient may be feature coefficients extracted from the second image, used to represent features in the second image; for example, the second image is input into a pre-trained model, and the model outputs the first adjustment coefficient and the second adjustment coefficient of the second image.

[0063] In other implementations, an adjusted image and a guiding map of the second image can be obtained based on a pre-trained processing model, a first adjustment coefficient can be obtained based on the similarity between the adjusted image and the guiding map, and then the guiding map can be corrected based on the first adjustment coefficient, and the difference between the adjusted image and the corrected guiding map can be calculated as the second adjustment coefficient.

[0064] S103: Obtain a third adjustment coefficient and a fourth adjustment coefficient according to the first adjustment coefficient and the second adjustment coefficient.

[0065] Optionally, the first adjustment coefficient and the second adjustment coefficient may be upsampled to obtain a third adjustment coefficient and a fourth adjustment coefficient.

[0066] It can be understood that the first adjustment coefficient and the second adjustment coefficient are obtained based on the low-resolution second image, and the low-resolution second image is obtained by downsampling the high-resolution first image. Therefore, the third adjustment coefficient and the fourth adjustment coefficient obtained based on upsampling are the adjustment coefficients corresponding to the high-frequency first image.

[0067] S104: Acquire a target image according to the first image, the third adjustment coefficient, and the fourth adjustment coefficient.

[0068] In some implementations, the third adjustment coefficient and the fourth adjustment coefficient may be fused to obtain a fusion coefficient, which is used as a beautification weight for the first image. The first image is beautified based on the fusion coefficient to obtain a beautified target image.

[0069] In other implementations, a guide image of the first image can be obtained. The guide image can be an image obtained by performing a small parameter convolution on the first image. The guide image is beautified based on the third adjustment coefficient and the fourth adjustment coefficient to obtain an initial beautified image. The initial beautified image is added to the first image to obtain a target image after the first image is beautified.

[0070] In some implementations, the initial beautified image can also be denoised by determining an appropriate noise threshold, and setting the pixel values ​​in the initial beautified image whose absolute values ​​are less than the noise threshold to zero to obtain a denoised initial beautified image. The denoised initial beautified image is added to the first image to obtain a target image with a better beautification effect.

[0071] In this embodiment, the first image to be beautified is downsampled to obtain a second image, and a first adjustment coefficient and a second adjustment coefficient are obtained based on the second image. The beautification features in the second image are reflected by the first adjustment image and the second adjustment image. Then, upsampling is performed based on the first adjustment coefficient and the second adjustment coefficient to obtain a third adjustment coefficient and a fourth adjustment coefficient. The adjustment coefficients are expanded based on the beautification features of the low resolution, and then beautification processing is performed in combination with the first image to ensure that the adjustment coefficients of the low resolution image can be effectively extended to the high resolution image, thereby achieving an ideal high resolution beautification effect.

[0072] Figure 2 This is a schematic diagram of another image processing method provided by an embodiment of the present disclosure. Figure 2 As shown, the method includes the following steps:

[0073] S201: Acquire a first image and downsample the first image to obtain a second image.

[0074] In the embodiment of the present disclosure, the implementation method of step S201 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0075] S202: Acquire a second adjustment image and a second guide image of the second image.

[0076] In some implementations, the second adjusted image may be an image obtained based on a pre-trained beautification neural network, that is, the second image is input into the pre-trained beautification neural network, and the beautification neural network performs preliminary adjustments on the second image to obtain the second adjusted image.

[0077] Optionally, the beautification neural network can be any neural network model that outputs pixel-level prediction tasks, such as convolutional neural network (CNN), recursive neural network (RNN), and image generative adversarial network (GANS).

[0078] In some implementations, the second guide map can be an image obtained based on a small-parameter convolution, that is, the second image is subjected to a small-parameter convolution process to obtain the second guide map. Optionally, the small-parameter convolution process can include a small-parameter convolution, batch normalization, and an activation function, resulting in a small amount of computation.

[0079] S203 , performing similarity calculation on the second adjustment image and the second guide image to obtain a first adjustment coefficient.

[0080] In some implementations, a cross-attention process may be performed on the second adjusted image and the second guiding map to obtain a first adjustment coefficient. That is, a cross-attention calculation is performed on the second adjusted image and the second guiding map to obtain a similarity, and the similarity is used as the first adjustment coefficient to more accurately reflect the beautification effect of the low-resolution image. Optionally, the calculation of the similarity may be expressed as:

[0081]

[0082]

[0083]

[0084]

[0085] d k =im(K)

[0086] Among them, a k is the first adjustment coefficient; w k is each sliding window; Adjust the image for the second; is the second guide map; Q is the important feature in the second guide map, K, V are the global features of the second adjusted image, Q, K, V are feature vectors of the same dimension; d k is the dimension of K.

[0087] S204 : Obtain a second adjustment coefficient according to the first adjustment coefficient, the second adjustment image, and the second guide image.

[0088] Optionally, the second adjusted image may be filtered to obtain a filtered adjusted image; and the second guiding map may be filtered to obtain a filtered guiding map. In other words, the second adjusted image and the second guiding map may be filtered to obtain corresponding filtered adjusted images and filtered guiding maps, respectively, thereby reducing the impact of noise in the image and improving the image beautification effect. Optionally, the filtering process may be mean filtering.

[0089] Furthermore, the filtered guidance map is corrected based on the first adjustment coefficient to obtain a corrected guidance map; and a subtraction operation is performed on the filtered adjustment image and the corrected guidance map to obtain a second adjustment coefficient.

[0090] Optionally, the second adjustment coefficient can be expressed as:

[0091]

[0092] Among them, b k is the second adjustment coefficient; Adjust the image for filtering; is the filtering guidance map; represents a modified guidance map, that is, a modified guidance map obtained by modifying the filtered guidance map based on the first adjustment coefficient.

[0093] S205 : Upsample the first adjustment coefficient and the second adjustment coefficient based on the resolution of the first image to obtain a fifth adjustment coefficient and a sixth adjustment coefficient.

[0094] In some implementations, the resolution of the first image can be used as a basis for upsampling, that is, the first adjustment coefficient and the second adjustment coefficient are upsampled to obtain the fifth adjustment coefficient and the sixth adjustment coefficient that are consistent with the resolution of the first image.

[0095] S206: Optimize the fifth adjustment coefficient and the sixth adjustment coefficient to obtain a third adjustment coefficient and a fourth adjustment coefficient.

[0096] Optionally, the fifth adjustment coefficient and the sixth adjustment coefficient may be optimized based on a pre-trained optimization network to achieve better beautification effects, thereby obtaining a third adjustment coefficient and a fourth adjustment coefficient with better optimized effects.

[0097] S207 : Acquire a target image according to the first image, the third adjustment coefficient, and the fourth adjustment coefficient.

[0098] In the embodiment of the present disclosure, the implementation method of step S207 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0099] In this embodiment, a second adjustment image and a second guiding map are obtained based on the second image, a similarity calculation is performed based on the second beautification map and the second guiding map to obtain a first adjustment coefficient, and then the second adjustment coefficient is obtained based on the first adjustment coefficient. The first adjustment coefficient and the second adjustment coefficient can more comprehensively reflect the beautification effect of low resolution. Upsampling and optimization processing are performed based on the first adjustment coefficient and the second adjustment coefficient to obtain a third adjustment coefficient and a fourth adjustment coefficient with better optimization effect, ensuring that the beautification effect on the low-resolution image can be better extended to the high-resolution image. The first image is beautified based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a target image with better beautification effect.

[0100] Figure 3 This is a schematic diagram of another image processing method provided by an embodiment of the present disclosure. Figure 3 As shown, the method includes the following steps:

[0101] S301: Acquire a first image and downsample the first image to obtain a second image.

[0102] In the embodiment of the present disclosure, the implementation method of step S301 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0103] S302: Acquire a second adjustment image and a second guide image of the second image.

[0104] In the embodiment of the present disclosure, the implementation method of step S302 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0105] S303: Calculate the similarity between the second adjustment image and the second guide image to obtain a first adjustment coefficient.

[0106] In the embodiment of the present disclosure, the implementation method of step S303 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0107] S304 : Obtain a second adjustment coefficient according to the first adjustment coefficient, the second adjustment image, and the second guide image.

[0108] In the embodiment of the present disclosure, the implementation method of step S304 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0109] S305 : Upsample the first adjustment coefficient and the second adjustment coefficient based on the resolution of the first image to obtain a fifth adjustment coefficient and a sixth adjustment coefficient.

[0110] In the embodiment of the present disclosure, the implementation method of step S305 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0111] S306: Optimize the fifth adjustment coefficient and the sixth adjustment coefficient to obtain a third adjustment coefficient and a fourth adjustment coefficient.

[0112] In the embodiment of the present disclosure, the implementation method of step S306 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0113] S307: Acquire a first guiding image of the first image.

[0114] Optionally, a small-parameter convolution process may be performed on the first image to obtain a first guide image after the small-parameter convolution process.

[0115] S308 : Process the first guide image based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a first adjusted image of the first image.

[0116] In some implementations, soft light processing may be performed on the first guide image based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a third adjusted image of the first image.

[0117] Alternatively, the third adjusted image can be expressed as:

[0118]

[0119] Among them, R hr Indicates the third adjustment image; A k represents the third adjustment coefficient; represents the fourth adjustment coefficient; Represents the first guide graph.

[0120] Furthermore, the third adjusted image is subjected to denoising to obtain a first adjusted image of the first image. Optionally, an appropriate noise threshold may be determined, and pixel values ​​in the third adjusted image whose absolute values ​​are less than the noise threshold may be set to zero to remove noise, thereby obtaining a first adjusted image with a better beautification effect after denoising.

[0121] S309 , adding the first image and the first adjusted image to obtain a target image.

[0122] It can be understood that the first adjusted image is an adjusted image obtained based on the adjustment coefficient, and the first adjusted image is added to the first image to obtain a target image after beautification of the first image, and the target image is a high-resolution image with a better beautification effect.

[0123] like Figure 4 As shown, it shows a logic diagram of an image processing method provided by an embodiment of the present disclosure, for a high-resolution image to be processed (first image), it is down-sampled as an input image to obtain a low-resolution input image (second image), the low-resolution input image (second image) is beautified by a neural network to obtain a low-resolution soft light image (second adjustment image), the low-resolution input image (second image) is processed by a small-parameter convolution to obtain a low-resolution guide image (second guide image), and the low-resolution soft light image (second adjustment image) and the low-resolution guide image (second guide image) are further cross-attention calculated to obtain a beautification coefficient a (first adjustment coefficient) and a beautification coefficient b (second adjustment coefficient); the beautification coefficient a (first adjustment coefficient) and beautification coefficient b (second adjustment coefficient) are respectively upsampled and input into the optimization network for optimization to obtain high-resolution beautification coefficient a (third adjustment coefficient) and high-resolution beautification coefficient b (fourth adjustment coefficient); at the same time, the high-resolution input image (first image) is processed by small parameter convolution to obtain a high-resolution guide map (first guide map), and the high-resolution beautification coefficient a (third adjustment coefficient) and high-resolution beautification coefficient b (fourth adjustment coefficient) perform soft light processing and denoising on the high-resolution guide map (first guide map) to obtain a first adjusted image, and the first adjusted image is added to the original high-resolution input image (first image) to obtain the final high-resolution effect image (target image).

[0124] In this embodiment, after determining the fully expanded third adjustment coefficient and fourth adjustment coefficient, the first guide image of the first image is obtained, and the first guide image is processed based on the third adjustment coefficient and the fourth adjustment coefficient to obtain the first adjustment image, and then the first adjustment image is fused and added with the first image to obtain the beautified target image, so that the low-resolution beautification operation can be fully extended to the high-resolution image, realizing real-time high-resolution image beautification and better beautification effect.

[0125] Figure 5 This is a schematic diagram of another image processing method provided by an embodiment of the present disclosure. Figure 5 As shown, the method includes the following steps:

[0126] S501: Acquire a first image and downsample the first image to obtain a second image.

[0127] In the embodiment of the present disclosure, the implementation method of step S501 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0128] S502: Input the second image into a pre-trained target image processing model, and the target image processing model outputs a third adjustment coefficient and a fourth adjustment coefficient.

[0129] Optionally, the target image processing model may include a first network, a second network, a cross attention layer, an upsampling layer, a third network and a processing layer.

[0130] Optionally, the first network may be a beautification neural network, and the first network in the target image processing model outputs a second adjusted image of the second image.

[0131] Optionally, the second network may be a small-parameter convolutional network model, which performs small-parameter convolution processing on the second image and outputs a second guide map of the second image.

[0132] The cross attention layer can perform similarity calculation on the second adjusted image and the second guide map to obtain a first adjustment coefficient.

[0133] After the first adjustment coefficient, the second adjustment image, and the second guide map are determined, the second adjustment coefficient may be obtained based on the first adjustment coefficient, the second adjustment image, and the second guide map.

[0134] The upsampling layer can upsample the first adjustment coefficient and the second adjustment coefficient based on the resolution of the first image to obtain the fifth adjustment coefficient and the sixth adjustment coefficient, and the resolution of the fifth adjustment coefficient and the sixth adjustment coefficient is consistent with the resolution of the first image.

[0135] Optionally, the third network can be an optimization network. The third network in the target image processing model optimizes the fifth adjustment coefficient and the sixth adjustment coefficient to obtain the third adjustment coefficient and the fourth adjustment coefficient. The image beautification effect based on the third adjustment coefficient and the fourth adjustment coefficient is better.

[0136] The implementation method of each network layer in the target image processing model can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0137] S503: Input the first image into a target image processing model, and process the first image in the target image processing model based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a target image.

[0138] In some implementations, the second network in the target image processing model can output the first guidance map of the first image; that is, the first image is input into the target image processing model, and the small-parameter convolutional network model in the target image processing model processes the first image to obtain the first guidance map.

[0139] Furthermore, the processing layer in the target image processing model processes the first guide map based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a first adjusted image of the first image. That is, the processing layer processes the first guide map based on the third adjustment coefficient and the fourth adjustment coefficient to obtain the first adjusted image. The acquisition of the first adjusted image can be implemented in any of the embodiments of the present disclosure. This is not limited to this and will not be repeated here.

[0140] The processing layer adds the first image and the first adjusted image to obtain a target image, that is, an image after beautifying the first image.

[0141] In this embodiment, a target image processing model including multiple sub-networks can be constructed, and the second image obtained by downsampling is directly input into the target image processing model. The second image is processed by each sub-network layer in the target image processing model to obtain a third adjustment coefficient and a fourth adjustment coefficient. The first image is input into the target image processing model, and the first image is beautified in the model based on the third adjustment coefficient and the fourth adjustment coefficient, thereby ensuring the beautification effect while improving the efficiency and timeliness of image beautification.

[0142] Figure 6 A schematic diagram of a training method for an image processing model provided by an embodiment of the present disclosure. Figure 6 As shown, the method includes the following steps:

[0143] S601: Acquire a first sample image, and downsample the first sample image at least once to obtain sample images at different resolutions.

[0144] In some implementations, the first sample image is a high-resolution image.

[0145] Optionally, the first sample image may be downsampled multiple times, and each downsampling may obtain a sample image with a different resolution; for example, the first sample image may be downsampled twice to obtain a sample image with a medium resolution and a sample image with a low resolution, respectively.

[0146] S602: Obtain high-frequency features based on the sample image with the minimum resolution, and input the sample image with the minimum resolution and the high-frequency features into an image processing model to obtain a first coefficient combination.

[0147] In some implementations, the sample image with the smallest resolution can be recorded as a low-resolution sample image, and high-frequency features of the low-resolution sample image, such as Laplace features, are obtained; the low-resolution sample image and the high-frequency features are input into the image processing model, and the image processing model outputs a first coefficient combination.

[0148] The first coefficient combination includes a first sample adjustment coefficient and a second sample adjustment coefficient.

[0149] In some implementations, the image processing model may include multiple sub-network layers, such as a first network, a second network, a third network, a cross-attention layer, an upsampling layer, and a processing layer. High-frequency features are input into the image processing model along with the low-resolution sample image as auxiliary features. Each network layer in the image processing model processes the low-resolution sample image to obtain a first sample adjustment coefficient and a second sample adjustment coefficient. The method for obtaining the first sample adjustment coefficient and the second sample adjustment coefficient can be implemented using any of the methods in the various embodiments of the present disclosure, and this is not limited to this and will not be further described.

[0150] S603 : Processing the sample images at different resolutions respectively according to the first coefficient combination to obtain adjusted images corresponding to the sample images at each resolution.

[0151] In some implementations, the first coefficient combination can be upsampled to obtain a second coefficient combination corresponding to a different resolution; the number of times the first coefficient combination is upsampled should be consistent with the number of times the first sample image is downsampled, that is, if the first sample image is downsampled twice, the first coefficient combination needs to be upsampled twice to obtain a second coefficient combination corresponding to a different resolution, and the second coefficient combination may include a third adjustment coefficient and a fourth adjustment coefficient.

[0152] Furthermore, the sample images at each resolution are beautified based on the second coefficient combination at each resolution. For example, a guide map corresponding to the sample images at each resolution is obtained, and the guide map is processed based on the second coefficient combination corresponding to the resolution to obtain an adjusted image at the resolution.

[0153] S604: Adjust the image processing model according to the high-frequency features and the adjustment images corresponding to the sample images at each resolution, and continue training until a target image processing model is obtained.

[0154] In some implementations, a loss function can be obtained based on high-frequency features and adjusted images corresponding to sample images at various resolutions, and the image processing model can be adjusted using the loss function and continued to be trained until the loss function converges to obtain the target image processing model.

[0155] In some implementations, high-frequency features are input into an image processing model to obtain corresponding reconstructed high-frequency features. The cross-entropy loss between the high-frequency features and the reconstructed high-frequency features is calculated as a first loss. Label-adjusted images at various resolutions are obtained, and using the label-adjusted images as a reference, the loss between the adjusted images corresponding to the sample images at various resolutions output by the image processing model and the corresponding label-adjusted images is obtained as a second loss. Optionally, the first and second losses can be accumulated to obtain a loss function for the image processing model, which is then used to adjust the image processing model and continue training.

[0156] In this embodiment, the first sample image is downsampled to obtain sample images at different resolutions, the high-frequency features of the sample image with the smallest resolution are obtained, and the high-frequency features and the sample image with the smallest resolution are input into the image processing model to obtain a first coefficient combination, and then the sample images at different resolutions are beautified according to the first coefficient combination to obtain adjusted images corresponding to the sample images at each resolution. The current image processing model is adjusted and trained with the adjusted images and high-frequency features, and the model is trained in two aspects: high-frequency features and image beautification, thereby improving the model training effect and obtaining a target image processing model with better beautification effect.

[0157] Figure 7 A schematic diagram of another image processing model training method provided by an embodiment of the present disclosure. Figure 7 As shown, the method includes the following steps:

[0158] S701: Acquire a first sample image, and downsample the first sample image at least once to obtain sample images at different resolutions.

[0159] In the embodiment of the present disclosure, the implementation method of step S701 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0160] S702 , obtaining high-frequency features based on the sample image with the minimum resolution, and inputting the sample image with the minimum resolution and the high-frequency features into an image processing model to obtain a first coefficient combination.

[0161] In the embodiment of the present disclosure, the implementation method of step S702 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0162] S703 : Obtain coefficient combinations corresponding to different resolutions based on the first sample adjustment coefficient and the second sample adjustment coefficient.

[0163] Optionally, the first sample adjustment coefficient and the second sample adjustment coefficient can be upsampled and optimized for the first time to obtain a second coefficient combination; that is, the first sample adjustment coefficient and the second sample adjustment coefficient are upsampled for the first time, and optimized after upsampling to obtain a second coefficient combination, which may include the adjustment coefficient corresponding to the resolution after the first upsampling.

[0164] Furthermore, the second coefficient combination is upsampled N-1 times, and the coefficient combination obtained by each upsampling corresponds to a different resolution; N is the number of times the first sample image is downsampled, that is, the coefficients of the first coefficient combination upsampled are consistent with the number of times the first sample image is downsampled, ensuring that the coefficient combination obtained by upsampling corresponds to the sample images at each resolution, so as to ensure the beautification effect of the sample image.

[0165] For example, assuming that the first sample image is downsampled twice to obtain a medium-resolution sample image and a low-resolution sample image, the first coefficient combination is upsampled for the first time to obtain a second coefficient combination at medium resolution, and the second coefficient combination is upsampled again to obtain a coefficient combination at low resolution.

[0166] S704 , for resolution i, processing the sample image at resolution i according to the coefficient combination corresponding to resolution i, to obtain an adjusted image corresponding to the sample image at resolution i.

[0167] Wherein, i is a positive integer, 1≤i≤N+1, and N is the number of times the first sample image is downsampled.

[0168] In some implementations, the coefficient combination corresponding to resolution i may include a third adjustment coefficient and a fourth adjustment coefficient. The sample image at resolution i is processed based on the third adjustment coefficient and the fourth adjustment coefficient to obtain the corresponding adjusted images at each resolution, so that the adjustment coefficient of the low-resolution image can be effectively extended to the high-resolution image. The method for processing the sample image can be implemented in any of the embodiments of the present disclosure. This is not limited to this and will not be repeated here.

[0169] S705 , for resolution i, determine the image loss corresponding to the resolution i according to the adjusted image corresponding to the sample image at resolution i and the reference adjusted image.

[0170] Wherein, i is a positive integer, 1≤i≤N+1, and N is the number of downsampling times.

[0171] In some implementations, for resolution i, a sample image at resolution i can be processed to obtain an initial reference adjusted image corresponding to the sample image at resolution i. The initial reference adjusted image is an image with a standard beautification effect. To improve the effectiveness of training based on the initial reference adjusted image, the initial reference adjusted image is further denoised to obtain a reference adjusted image for the sample image at resolution i. That is, for each standard initial reference adjusted image at each resolution, the initial reference adjusted image is denoised to obtain a reference adjusted image, thereby improving the effectiveness of model training based on the reference adjusted image and reducing the impact of other noise in the image.

[0172] In other implementations, the pixel value difference between the sample image at resolution i and the corresponding candidate reference adjusted image can be obtained, and an image mask can be determined based on the pixel value difference of each pixel; the candidate adjusted image can be masked based on the image mask to obtain a first masked adjusted image; morphological operations can be performed on the masked adjusted image to obtain a second masked adjusted image; and the sample image at resolution i and the second masked adjusted image can be fused to obtain a reference adjusted image of the sample image at resolution i. Optionally, a candidate reference adjusted image corresponding to the first sample image can be obtained, and the candidate reference adjusted image can be downsampled to obtain a candidate reference adjusted image corresponding to the sample image at each resolution.

[0173] As an example, the pixel value difference of each pixel between the sample image at low resolution and the corresponding candidate reference adjustment image is obtained, a noise threshold is preset, and all pixels whose pixel value difference is greater than the threshold are obtained to form an image mask. The candidate adjustment image is masked based on the image mask to obtain a first mask adjustment image, and the area in the first mask adjustment image is approximately the required non-noise area; morphological operations are performed on the first mask adjustment image, such as corrosion, dilation and other operations to optimize the mask coverage area to obtain a second mask adjustment image, and the second mask adjustment image can better cover the non-noise area while retaining the complete beautification area; finally, the sample image at low resolution and the second mask adjustment image are fused, the mask area uses the pixel value of the second mask adjustment image, and the non-mask area uses the original pixel value of the sample image at low resolution to obtain a reference adjustment image of the sample image at low resolution, and the model training based on the reference adjustment image is more effective and generates less noise.

[0174] Furthermore, the reference adjusted image is used as the label image, and the loss between the adjusted image corresponding to the sample image at resolution i and the reference adjusted image is calculated as the image loss corresponding to resolution i; optionally, the peak signal-to-noise ratio loss between the adjusted image corresponding to the sample image at resolution i and the reference adjusted image can be calculated as the image loss.

[0175] S706 , obtaining a reconstructed high-frequency feature according to the high-frequency feature, and determining a feature loss according to the high-frequency feature and the reconstructed high-frequency feature.

[0176] Optionally, the high-frequency features and the corresponding sample images can be input into an image processing model, the reconstructed high-frequency features can be output, and the loss between the reconstructed high-frequency features and the high-frequency features can be calculated as the feature loss, for example, the cross entropy loss between the reconstructed high-frequency features and the high-frequency features can be calculated as the feature loss.

[0177] S707: Determine the total loss of the image processing model according to the image loss and feature loss corresponding to each resolution, and adjust the image processing model based on the total loss.

[0178] Optionally, the image loss and feature loss corresponding to each resolution can be weighted and summed to obtain the total loss of the image processing model, where the weight of each loss can be preset in advance.

[0179] The image processing model is adjusted based on the total loss of the image processing model and training is continued until the total loss of the image processing model converges, thereby obtaining a target image processing model after training.

[0180] Figure 8 A logic diagram of a training method for an image processing model provided in an embodiment of the present disclosure is provided. A medium-resolution input image and a low-resolution input image are obtained by downsampling a high-resolution input image twice. High-frequency features are obtained based on the low-resolution input image and input into a beautification model set to obtain a first coefficient combination. Upsampling and optimization are performed according to the first coefficient combination to obtain a medium-resolution coefficient combination and a high-resolution coefficient combination. Then, the sample image is beautified according to the coefficient combinations at different resolutions to obtain a high-resolution adjusted image, a medium-resolution adjusted image, and a low-resolution adjusted image. Then, based on the feature loss between the reconstructed high-frequency features and the high-frequency features, the image loss between the adjusted images at each resolution and the reference adjusted image is obtained to obtain the total loss, thereby performing image processing model training.

[0181] In this embodiment, coefficient combinations are obtained based on sample images at different resolutions, and adjusted images at different resolutions are obtained based on the coefficient combinations. Image loss is obtained based on the losses of the adjusted image and the reference adjusted image, wherein the reference adjusted image can be an image with better beautification effect after denoising. Feature loss is obtained based on high-frequency features and reconstructed high-frequency features. The total loss of the fusion of feature loss and image loss is used as the loss function of the image processing model. When the total loss converges, the trained target image processing model is obtained, thereby improving the training effect of the target image processing model and making the target image processing model have a better beautification effect on the image.

[0182] Figure 9This is a schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure. Figure 9 As shown, the image processing device 900 includes:

[0183] The image acquisition module 901 is used to acquire a first image and downsample the first image to obtain a second image;

[0184] A coefficient acquisition module 902 is configured to acquire a first adjustment coefficient and a second adjustment coefficient based on the second image; and acquire a third adjustment coefficient and a fourth adjustment coefficient based on the first adjustment coefficient and the second adjustment coefficient;

[0185] The image processing module 903 is configured to obtain a target image according to the first image, the third adjustment coefficient, and the fourth adjustment coefficient.

[0186] In some implementations, the coefficient acquisition module 902 includes:

[0187] acquiring a second adjusted image and a second guide image of the second image;

[0188] Performing similarity calculation on the second adjustment image and the second guide image to obtain a first adjustment coefficient;

[0189] A second adjustment coefficient is obtained according to the first adjustment coefficient, the second adjustment image, and the second guide map.

[0190] In some implementations, the coefficient acquisition module 902 includes:

[0191] performing filtering processing on the second adjusted image to obtain a filtered adjusted image;

[0192] Filtering the second guidance map to obtain a filtered guidance map, and correcting the filtered guidance map based on the first adjustment coefficient to obtain a corrected guidance map;

[0193] A subtraction operation is performed on the filtered adjustment image and the corrected guide image to obtain a second adjustment coefficient.

[0194] In some implementations, the coefficient acquisition module 902 includes:

[0195] Cross-attention processing is performed on the second adjustment image and the second guiding image to obtain a first adjustment coefficient.

[0196] In some implementations, the coefficient acquisition module 902 includes:

[0197] Upsampling the first adjustment coefficient and the second adjustment coefficient based on a resolution of the first image to obtain a fifth adjustment coefficient and a sixth adjustment coefficient;

[0198] The fifth adjustment coefficient and the sixth adjustment coefficient are optimized to obtain a third adjustment coefficient and a fourth adjustment coefficient.

[0199] In some implementations, the image processing module 903 includes:

[0200] acquiring a first guide image of the first image;

[0201] processing the first guide image based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a first adjusted image of the first image;

[0202] The target image is obtained by adding the first image and the first adjustment image.

[0203] In some implementations, the image processing module 903 includes:

[0204] performing soft light processing on the first guide image based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a third adjusted image of the first image;

[0205] De-noising is performed on the third adjusted image to obtain a first adjusted image of the first image.

[0206] In this embodiment, the first image to be beautified is downsampled to obtain a second image, and a first adjustment coefficient and a second adjustment coefficient are obtained based on the second image. The beautification features in the second image are reflected by the first adjustment image and the second adjustment image. Then, upsampling is performed based on the first adjustment coefficient and the second adjustment coefficient to obtain a third adjustment coefficient and a fourth adjustment coefficient. The adjustment coefficients are expanded based on the beautification features of the low resolution, and then beautification processing is performed in combination with the first image to ensure that the adjustment coefficients of the low resolution image can be effectively extended to the high resolution image, thereby achieving an ideal high resolution beautification effect.

[0207] Figure 10 A schematic structural diagram of another image processing device provided by an embodiment of the present disclosure.

[0208] like Figure 10 As shown, the image processing device 1000 includes:

[0209] A first acquisition module 1001 is configured to acquire a first image and downsample the first image to obtain a second image;

[0210] A second acquisition module 1002 is configured to input the second image into a pre-trained target image processing model, and the target image processing model outputs a third adjustment coefficient and a fourth adjustment coefficient;

[0211] The processing module 1003 is configured to input the first image into a target image processing model, and process the first image based on the third adjustment coefficient and the fourth adjustment coefficient in the target image processing model to obtain a target image.

[0212] In some implementations, the second obtaining module 1002 includes:

[0213] Outputting, by the first network in the target image processing model, a second adjusted image of the second image;

[0214] Outputting a second guidance map for the second image by a second network in the target image processing model;

[0215] The cross attention layer performs similarity calculation on the second adjustment image and the second guidance map to obtain a first adjustment coefficient;

[0216] The upsampling layer upsamples the first adjustment coefficient and the second adjustment coefficient based on the resolution of the first image to obtain a fifth adjustment coefficient and a sixth adjustment coefficient;

[0217] The fifth adjustment coefficient and the sixth adjustment coefficient are optimized by the third network in the target image processing model to obtain the third adjustment coefficient and the fourth adjustment coefficient.

[0218] In some implementations, the processing module 1003 includes:

[0219] Outputting a first guidance map of the first image by a second network in the target image processing model;

[0220] The processing layer in the target image processing model processes the first guide map based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a first adjusted image of the first image, and adds the first image and the first adjusted image to obtain the target image.

[0221] In this embodiment, a target image processing model including multiple sub-networks can be constructed, and the second image obtained by downsampling is directly input into the target image processing model. The second image is processed by each sub-network layer in the target image processing model to obtain a third adjustment coefficient and a fourth adjustment coefficient. The first image is input into the target image processing model, and the first image is processed in the model based on the third adjustment coefficient and the fourth adjustment coefficient, thereby ensuring the beautification effect while improving the efficiency and timeliness of image beautification.

[0222] Figure 11 This is a structural diagram of a training device for an image processing model provided by an embodiment of the present disclosure. Figure 11 As shown, the training device 1100 of the image processing model includes:

[0223] The sample acquisition module 1101 is configured to acquire a first sample image and downsample the first sample image at least once to obtain sample images at different resolutions.

[0224] A coefficient combination acquisition module 1102 is configured to acquire high-frequency features based on the sample image with the minimum resolution, and input the sample image with the minimum resolution and the high-frequency features into an image processing model to obtain a first coefficient combination, where the first coefficient combination includes a first sample adjustment coefficient and a second sample adjustment coefficient.

[0225] A beautification module 1103 is configured to process the sample images at different resolutions according to the first coefficient combination to obtain adjusted images corresponding to the sample images at each resolution;

[0226] The training module 1104 is used to adjust the image processing model according to the high-frequency features and the adjustment images corresponding to the sample images at various resolutions, and continue training until a target image processing model is obtained.

[0227] In some implementations, the training module 1104 includes:

[0228] For resolution i, determine the image loss corresponding to resolution i based on the adjusted image corresponding to the sample image at resolution i and the reference adjusted image, where i is a positive integer, 1≤i≤N+1, and N is the number of downsampling times;

[0229] According to the high-frequency features, reconstructed high-frequency features are obtained, and feature loss is determined according to the high-frequency features and the reconstructed high-frequency features;

[0230] According to the image loss and feature loss corresponding to each resolution, the total loss of the image processing model is determined, and the image processing model is adjusted based on the total loss.

[0231] In some implementations, the beautification module 1103 includes:

[0232] Based on the first sample adjustment coefficient and the second sample adjustment coefficient, coefficient combinations corresponding to different resolutions are obtained;

[0233] For resolution i, the sample image at resolution i is processed according to the coefficient combination corresponding to resolution i to obtain an adjusted image corresponding to the sample image at resolution i, where i is a positive integer, 1≤i≤N+1, and N is the number of times the first sample image is downsampled.

[0234] In some implementations, the beautification module 1103 includes:

[0235] Performing a first upsampling and optimization process on the first sample adjustment coefficient and the second sample adjustment coefficient to obtain a second coefficient combination;

[0236] The second coefficient combination is upsampled N-1 times, and the coefficient combination obtained by each upsampling corresponds to a different resolution.

[0237] In some implementations, the apparatus 1100 further includes:

[0238] For resolution i, the sample image at resolution i is processed to obtain an initial reference adjusted image corresponding to the sample image at resolution i, and the initial reference adjusted image is denoised to obtain a reference adjusted image of the sample image at resolution i.

[0239] In some implementations, the apparatus 1100 further includes:

[0240] Obtain the pixel value difference between the sample image at resolution i and the corresponding candidate reference adjusted image, and determine the image mask based on the pixel value difference of each pixel;

[0241] Masking the candidate adjusted image according to the image mask to obtain a first masked adjusted image;

[0242] Performing morphological operations on the masked adjusted image to obtain a second masked adjusted image;

[0243] The sample image at resolution i and the second mask-adjusted image are fused to obtain a reference-adjusted image of the sample image at resolution i.

[0244] In this embodiment, coefficient combinations are obtained based on sample images at different resolutions, and adjusted images at different resolutions are obtained based on the coefficient combinations. Image loss is obtained based on the losses of the adjusted image and the reference adjusted image, wherein the reference adjusted image can be an image with better beautification effect after denoising. Feature loss is obtained based on high-frequency features and reconstructed high-frequency features. The total loss of the fusion of feature loss and image loss is used as the loss function of the image processing model. When the total loss converges, the trained target image processing model is obtained, thereby improving the training effect of the target image processing model and making the target image processing model have a better beautification effect on the image.

[0245] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0246] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0247] Figure 12A schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0248] like Figure 12 As shown, the device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. Various programs and data required for the operation of the device 1200 can also be stored in the RAM 1203. The computing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0249] Various components in device 1200 are connected to I / O interface 1205, including an input unit 1206, such as a keyboard and mouse; an output unit 1207, such as various types of displays and speakers; a storage unit 1208, such as a magnetic disk and optical disk; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. Communication unit 1209 allows device 1200 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0250] The computing unit 1201 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above, such as the image processing method. For example, in some embodiments, the image processing method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 1200 via the ROM 1202 and / or the communication unit 1209. When the computer program is loaded into the RAM 1203 and executed by the computing unit 1201, one or more steps of the image processing method described above can be performed. Alternatively, in other embodiments, the computing unit 1201 can be configured to perform the image processing method by any other appropriate means (e.g., by means of firmware).

[0251] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0252] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0253] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0254] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0255] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0256] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0257] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0258] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. An image processing method, wherein: The method comprises: Acquire a first image, and downsample the first image to obtain a second image; Based on the second image, obtaining a first adjustment coefficient and a second adjustment coefficient; wherein the first adjustment coefficient and the second adjustment coefficient are coefficients representing features of the second image; Upsampling the first adjustment coefficient and the second adjustment coefficient to obtain a third adjustment coefficient and a fourth adjustment coefficient; Obtain a first guide image of the first image, process the first guide image based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a first adjustment image of the first image, and add the first image and the first adjustment image to obtain a target image.

2. The method according to claim 1, wherein The acquiring, based on the second image, a first adjustment coefficient and a second adjustment coefficient, includes: acquiring a second adjusted image and a second guide image of the second image; performing similarity calculation on the second adjustment image and the second guide map to obtain the first adjustment coefficient; The second adjustment coefficient is obtained according to the first adjustment coefficient, the second adjustment image and the second guide map.

3. The method according to claim 2, wherein: The obtaining the second adjustment coefficient according to the first adjustment coefficient, the second adjustment image and the second guide map includes: performing filtering processing on the second adjusted image to obtain a filtered adjusted image; performing filtering processing on the second guidance map to obtain a filtered guidance map, and correcting the filtered guidance map based on the first adjustment coefficient to obtain a corrected guidance map; A subtraction operation is performed on the filtered adjustment image and the corrected guide map to obtain the second adjustment coefficient.

4. The method according to claim 2, wherein: The performing similarity calculation on the second adjustment image and the second guide map to obtain the first adjustment coefficient includes: Cross-attention processing is performed on the second adjustment image and the second guiding image to obtain the first adjustment coefficient.

5. The method according to claim 1, wherein The upsampling of the first adjustment coefficient and the second adjustment coefficient to obtain a third adjustment coefficient and a fourth adjustment coefficient includes: Upsampling the first adjustment coefficient and the second adjustment coefficient based on a resolution of the first image to obtain a fifth adjustment coefficient and a sixth adjustment coefficient; The fifth adjustment coefficient and the sixth adjustment coefficient are optimized to obtain the third adjustment coefficient and the fourth adjustment coefficient.

6. The method according to claim 1, wherein The processing of the first guide image based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a first adjusted image of the first image includes: performing soft light processing on the first guide image based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a third adjusted image of the first image; De-noising is performed on the third adjusted image to obtain a first adjusted image of the first image.

7. An image processing method, wherein: The method comprises: Acquire a first image, and downsample the first image to obtain a second image; Inputting the second image into a pre-trained target image processing model, and having the target image processing model output a third adjustment coefficient and a fourth adjustment coefficient; The first image is input into the target image processing model, a first guide image of the first image is obtained in the target image processing model, the first guide image is processed based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a first adjustment image of the first image, and the first image and the first adjustment image are added to obtain a target image.

8. The method according to claim 7, wherein: Outputting the third adjustment coefficient and the fourth adjustment coefficient by the target image processing model includes: outputting, by a first network in the target image processing model, a second adjusted image of the second image; Outputting, by a second network in the target image processing model, a second guidance map for the second image; performing similarity calculation on the second adjusted image and the second guide map by a cross attention layer to obtain a first adjustment coefficient; An upsampling layer upsamples the first adjustment coefficient and the second adjustment coefficient based on a resolution of the first image to obtain a fifth adjustment coefficient and a sixth adjustment coefficient; The fifth adjustment coefficient and the sixth adjustment coefficient are optimized by the third network in the target image processing model to obtain the third adjustment coefficient and the fourth adjustment coefficient.

9. The method according to claim 7, wherein: The first guide image of the first image is output by a second network in the target image processing model; The target image is determined by processing layers in the target image processing model.

10. A method for training an image processing model, wherein: The method comprises: Acquire a first sample image, and downsample the first sample image at least once to obtain sample images at different resolutions; Acquire high-frequency features based on a sample image with minimum resolution, and input the sample image with minimum resolution and the high-frequency features into an image processing model to obtain a first coefficient combination, wherein the first coefficient combination includes a first sample adjustment coefficient and a second sample adjustment coefficient; processing the sample images at different resolutions according to the first coefficient combination to obtain adjusted images corresponding to the sample images at the respective resolutions; According to the high-frequency features and the adjusted images corresponding to the sample images at each resolution, the image processing model is adjusted and trained continuously until a target image processing model is obtained, and the target image processing model is used to implement the method as described in any one of claims 7 to 9.

11. The method according to claim 10, wherein: The image processing model is adjusted and trained continuously according to the high-frequency features and the adjusted images corresponding to the sample images at each resolution until a target image processing model is obtained, including: For resolution i , according to the resolution i The sample image below corresponds to the adjusted image and the reference adjusted image, and the resolution is determined i The corresponding image loss is, where i is a positive integer, 1≤ i ≤N+1, N is the number of downsampling; Obtaining a reconstructed high-frequency feature according to the high-frequency feature, and determining a feature loss according to the high-frequency feature and the reconstructed high-frequency feature; The total loss of the image processing model is determined according to the image loss corresponding to each resolution and the feature loss, and the image processing model is adjusted based on the total loss.

12. The method according to claim 10, wherein: The processing of the sample images at different resolutions according to the first coefficient combination to obtain the adjusted images corresponding to the sample images at the respective resolutions includes: Obtaining coefficient combinations corresponding to different resolutions based on the first sample adjustment coefficient and the second sample adjustment coefficient; For resolution i , according to the resolution i The corresponding coefficient combination, for the resolution i The sample image is processed to obtain the resolution i The sample image below corresponds to the adjusted image, where i is a positive integer, 1≤ i ≤N+1, where N is the number of times the first sample image is downsampled.

13. The method according to claim 12, wherein: The obtaining of coefficient combinations corresponding to different resolutions based on the first sample adjustment coefficient and the second sample adjustment coefficient includes: Performing a first upsampling and optimization process on the first sample adjustment coefficient and the second sample adjustment coefficient to obtain a second coefficient combination; The second coefficient combination is upsampled N-1 times, and the coefficient combination obtained by each upsampling corresponds to a different resolution.

14. The method according to claim 11, wherein The method further comprises: For the resolution i , for the resolution i The sample image is processed to obtain the resolution i The initial reference adjustment image corresponding to the sample image under the condition of denoising is performed on the initial reference adjustment image to obtain the resolution i The sample image below is the reference adjustment image.

15. The method according to claim 11, wherein The initial reference adjusted image is subjected to denoising to obtain the resolution i The following sample images are reference adjustment images, including: Get the resolution i The pixel value difference between the sample image and the corresponding candidate reference adjusted image is calculated, and an image mask is determined based on the pixel value difference of each pixel; masking the candidate adjusted image according to the image mask to obtain a first masked adjusted image; Performing morphological operations on the masked adjusted image to obtain a second masked adjusted image; For the resolution i The sample image under the resolution is fused with the second mask adjustment image to obtain the resolution i The sample image below is the reference adjustment image.

16. An image processing apparatus, comprising: An image acquisition module, configured to acquire a first image and downsample the first image to obtain a second image; a coefficient acquisition module, configured to acquire a first adjustment coefficient and a second adjustment coefficient based on the second image; and upsample the first adjustment coefficient and the second adjustment coefficient to obtain a third adjustment coefficient and a fourth adjustment coefficient; wherein the first adjustment coefficient and the second adjustment coefficient are coefficients representing features of the second image; An image processing module is used to obtain a first guide image of the first image, process the first guide image based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a first adjustment image of the first image, and add the first image and the first adjustment image to obtain a target image.

17. The device according to claim 16, wherein The coefficient acquisition module includes: acquiring a second adjusted image and a second guide image of the second image; performing similarity calculation on the second adjustment image and the second guide map to obtain the first adjustment coefficient; The second adjustment coefficient is obtained according to the first adjustment coefficient, the second adjustment image and the second guide map.

18. The device according to claim 17, wherein The coefficient acquisition module includes: performing filtering processing on the second adjusted image to obtain a filtered adjusted image; performing filtering processing on the second guidance map to obtain a filtered guidance map, and correcting the filtered guidance map based on the first adjustment coefficient to obtain a corrected guidance map; A subtraction operation is performed on the filtered adjustment image and the corrected guide map to obtain the second adjustment coefficient.

19. The device according to claim 17, wherein The coefficient acquisition module includes: Cross-attention processing is performed on the second adjustment image and the second guiding image to obtain the first adjustment coefficient.

20. The apparatus according to claim 16, wherein The coefficient acquisition module includes: Upsampling the first adjustment coefficient and the second adjustment coefficient based on a resolution of the first image to obtain a fifth adjustment coefficient and a sixth adjustment coefficient; The fifth adjustment coefficient and the sixth adjustment coefficient are optimized to obtain the third adjustment coefficient and the fourth adjustment coefficient.

21. The apparatus according to claim 16, wherein The image processing module includes: performing soft light processing on the first guide image based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a third adjusted image of the first image; De-noising is performed on the third adjusted image to obtain a first adjusted image of the first image.

22. An image processing device, comprising: A first acquisition module is configured to acquire a first image and downsample the first image to obtain a second image; a second acquisition module, configured to input the second image into a pre-trained target image processing model, and have the target image processing model output a third adjustment coefficient and a fourth adjustment coefficient; A processing module is used to input the first image into the target image processing model, obtain a first guide image of the first image in the target image processing model, process the first guide image based on the third adjustment coefficient and the fourth adjustment coefficient to obtain a first adjustment image of the first image, and add the first image and the first adjustment image to obtain a target image.

23. The device according to claim 22, wherein The second acquisition module includes: outputting, by a first network in the target image processing model, a second adjusted image of the second image; Outputting, by a second network in the target image processing model, a second guidance map for the second image; performing similarity calculation on the second adjusted image and the second guide map by a cross attention layer to obtain a first adjustment coefficient; An upsampling layer upsamples the first adjustment coefficient and the second adjustment coefficient based on a resolution of the first image to obtain a fifth adjustment coefficient and a sixth adjustment coefficient; The fifth adjustment coefficient and the sixth adjustment coefficient are optimized by the third network in the target image processing model to obtain the third adjustment coefficient and the fourth adjustment coefficient.

24. The apparatus according to claim 22, wherein The first guidance map of the first image is output by a second network in the target image processing model; The target image is determined by the processing layers in the target image processing model.

25. A training device for an image processing model, comprising: a sample acquisition module, configured to acquire a first sample image and downsample the first sample image at least once to obtain sample images at different resolutions; a coefficient combination acquisition module, configured to acquire high-frequency features based on a sample image with minimum resolution, and input the sample image with minimum resolution and the high-frequency features into an image processing model to obtain a first coefficient combination, wherein the first coefficient combination includes a first sample adjustment coefficient and a second sample adjustment coefficient; a beautification module, configured to process sample images at different resolutions according to the first coefficient combination to obtain adjusted images corresponding to the sample images at each resolution; A training module is used to adjust the image processing model according to the high-frequency features and the adjustment images corresponding to the sample images at each resolution and continue training until a target image processing model is obtained, and the target image processing model is used to implement the method described in any one of claims 7 to 9.

26. The device according to claim 25, wherein The training module includes: For resolution i , according to the resolution i The sample image below corresponds to the adjusted image and the reference adjusted image, and the resolution is determined i The corresponding image loss is, where i is a positive integer, 1≤ i ≤N+1, N is the number of downsampling; Obtaining a reconstructed high-frequency feature according to the high-frequency feature, and determining a feature loss according to the high-frequency feature and the reconstructed high-frequency feature; The total loss of the image processing model is determined according to the image loss corresponding to each resolution and the feature loss, and the image processing model is adjusted based on the total loss.

27. The apparatus according to claim 25, wherein The beautification module includes: Obtaining coefficient combinations corresponding to different resolutions based on the first sample adjustment coefficient and the second sample adjustment coefficient; For resolution i , according to the resolution i The corresponding coefficient combination, for the resolution i The sample image is processed to obtain the resolution i The sample image below corresponds to the adjusted image, where i is a positive integer, 1≤ i ≤N+1, where N is the number of times the first sample image is downsampled.

28. The apparatus according to claim 27, wherein The beautification module includes: Performing a first upsampling and optimization process on the first sample adjustment coefficient and the second sample adjustment coefficient to obtain a second coefficient combination; The second coefficient combination is upsampled N-1 times, and the coefficient combination obtained by each upsampling corresponds to a different resolution.

29. The apparatus according to claim 26, wherein The device further comprises: For the resolution i , for the resolution i The sample image is processed to obtain the resolution i The initial reference adjustment image corresponding to the sample image under the condition of denoising is performed on the initial reference adjustment image to obtain the resolution i The sample image below is the reference adjustment image.

30. The apparatus according to claim 29, wherein The device further comprises: Get the resolution i The pixel value difference between the sample image and the corresponding candidate reference adjusted image is calculated, and an image mask is determined based on the pixel value difference of each pixel; masking the candidate adjusted image according to the image mask to obtain a first masked adjusted image; Performing morphological operations on the masked adjusted image to obtain a second masked adjusted image; For the resolution i The sample image under the resolution is fused with the second mask adjustment image to obtain the resolution i The sample image below is the reference adjustment image.

31. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 15.

32. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-15.

33. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 15.

Images