Training method of image noise reduction model

Abstract

The invention provides a training method for an image noise reduction model, and relates to the technical field of image processing, and the method comprises the steps: obtaining a plurality of first-class sample pairs in a target training set; inputting the noise image in each first sample pair and the corresponding image quality score into a to-be-trained target image noise reduction model to obtain a denoising result corresponding to the noise image in each sample pair; determining a target loss value of the target image noise reduction model based on the denoising result corresponding to the noise image in each first type of sample pair and the target image in each first type of sample pair; and in response to determining that the target image noise reduction model is not converged according to the target loss value, performing parameter adjustment on the target image noise reduction model, and returning to the step of obtaining the plurality of first-class sample pairs in the target training set. According to the scheme, the noise reduction effect of the image noise reduction model obtained through training in an actual scene can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a training method for an image denoising model. Background Art

[0002] Image denoising is a common processing task in the field of image processing, in which image denoising can remove noise from an image, improve the quality and clarity of the image, and thus restore the true information of the image.

[0003] In the related art, the image denoising model is usually trained using sample pairs in the training set, thereby obtaining a trained image denoising model, so that the image denoising model can be used for image denoising later. The sample pair includes a target image and a noise image corresponding to the target image. The training purpose of the image denoising model is to obtain the target image in the sample pair after denoising the noise image in the sample pair. The noise image corresponding to the target image can be considered as an image in which noise is added to the target image, and the target image can usually be considered as a noise-free image.

[0004] However, in the related art, due to the fact that the noise images of the sample pairs in the training set are not images with real noise added to the target images of the sample pairs, or the image denoising model only focuses on the image features of the input image itself during training, the trained image denoising model has a poor denoising effect on the image and is not suitable for image denoising tasks in actual scenes. Therefore, there is an urgent need for a training method for an image denoising model to improve the denoising effect of the trained image denoising model in actual scenes. Summary of the invention

[0005] The purpose of the embodiment of the present application is to provide a method for training an image denoising model to improve the denoising effect of the trained image denoising model in actual scenes. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for training an image denoising model, the method comprising:

[0007] Acquire multiple first-category sample pairs in a target training set; wherein each first-category sample pair includes a target image and a noise image corresponding to the target image, and the noise image in any first-category sample pair is generated for the target image in the first-category sample pair according to a predetermined noise image generation method; the predetermined noise image generation method includes: selecting a spare noise image from multiple spare noise images as a target noise image; determining noise information to be used on each color channel of each pixel in the target noise image as noise information to be used for the target noise image; performing a specified noise addition process on the target image based on the noise information to be used for the target noise image; determining the noise image corresponding to the target image based on the image after the specified noise addition process; each spare noise image is generated based on a target area in an initial image among the initial images; each initial image is an image obtained by photographing a target color card; the target color card includes multiple color blocks, and the target area is an image area of ​​a color block among the multiple color blocks;

[0008] Inputting the noisy image and the corresponding image quality score in each first-category sample pair into the target image denoising model to be trained to obtain the denoising result corresponding to the noisy image in each sample pair; wherein the target image denoising model comprises an initial denoising module and an image content restoration module connected in series, the initial denoising module is used to process the input image to obtain a feature map of the same size as the input image; the image content restoration module is used to perform feature splicing processing on the feature map obtained by the initial denoising module and the image quality score input to the image content restoration module to obtain spliced ​​image data; the spliced ​​image data is processed to generate the denoising result of the image input to the initial denoising module;

[0009] Determine a target loss value of the target image denoising model based on a denoising result corresponding to the noise image in each first-category sample pair and a target image in each first-category sample pair;

[0010] In response to determining that the target image denoising model has not converged according to the target loss value, adjusting parameters of the target image denoising model, and returning to the step of obtaining multiple first category sample pairs in the target training set.

[0011] Beneficial effects of the embodiments of the present application:

[0012] The embodiment of the present application provides a method for training an image denoising model, in which the noise image in each first-category sample pair is an image obtained by performing a specified noise adding process on the target image based on the noise information to be used of the target noise image, and the target noise image is generated based on the color block area in the initial image obtained by photographing the target color card, so that the noise information added to the noise image is more in line with the real noise information. Since the target training set is suitable for image denoising tasks (i.e., the data authenticity of the sample pairs is good); and when training the target image denoising model, the present application takes into account the noise image and the corresponding image quality score, the target image denoising model can use more features to adaptively denoise the noise image, and can achieve reasonable denoising of the noise image based on the image quality score of the noise image. It can be seen that the noise reduction effect of the trained image denoising model in actual scenes can be improved through this solution.

[0013] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0015] Figure 1 A flowchart of a method for training an image denoising model provided in an embodiment of the present application;

[0016] Figure 2 A schematic diagram of a flow chart of a method for generating any one of the backup noise images provided in the embodiments of the present application;

[0017] Figure 3 A schematic diagram of the overall process of a training method for an image denoising model provided in an embodiment of the present application;

[0018] Figure 4 Another schematic diagram of a flow chart of a method for training an image denoising model provided in an embodiment of the present application;

[0019] Figure 5 A schematic diagram of a processing flow of a target image denoising model provided in an embodiment of the present application;

[0020] Figure 6 A schematic diagram of the processing flow of an image restoration attention module provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0022] For image acquisition scenarios, the noise and clarity of the images captured by the image acquisition device will be greatly affected by the module settings in the device ISP (Image Signal Processor) process. The captured images are usually noisy and low-definition images, which need to be image denoised. For image denoising algorithms, it is necessary to distinguish between details and noise in the image, remove noise in the image, restore details in the image, and obtain clear and natural images.

[0023] In actual applications, it is difficult to obtain sample pairs for model training. This difficulty makes it impossible to effectively train the deep learning model (image denoising model), resulting in the deep learning model being unable to achieve ideal image denoising processing effects on noisy images.

[0024] Among them, the sample pairs in the training set usually come from public data sets, artificially photographed data sets or image degradation data sets. The target images and the corresponding noise images in the sample pairs in the training set constructed by the public data sets are not suitable for real image denoising tasks. Although the sample pairs in the training set constructed by the artificially photographed data sets are suitable for image denoising tasks, due to the richness requirements for sample pairs (that is, the more the number and types of sample pairs, the better), the construction efficiency of the training set is very low, that is, the images in the artificially photographed data sets are difficult to obtain and difficult to operate, and the image acquisition is complex, inefficient and time-consuming (for example: the image of the target scene can be captured by the camera. In some scenes (such as daytime scenes), the image is a noise-free target image. At this time, the camera parameters need to be adjusted to capture the image with noise about the target scene as the noise image corresponding to the target image; in other scenes (such as night scenes), the image is a noise image. At this time, it is difficult to adjust the camera parameters to obtain a noise-free target image for the target scene). The sample pairs in the training set constructed by other image degradation methods are not suitable for all image denoising tasks. In addition, the difficulty of image degradation lies in how to generate a real noise image corresponding to the target image (that is, the training set constructed by the image degradation dataset is not suitable for a variety of image denoising tasks. Although the sample pairs in the training set constructed by the image degradation dataset include the target image and the noisy image of the target image generated by other image degradation methods, the generated noise image is not just an image with noise added to the target image, and the added noise is not real).

[0025] In the image acquisition scenario, the image noise levels of the images captured by the image acquisition device at different gains vary greatly, and are also quite different from the images in the public data set. For the noisy images captured by the image acquisition device, the image noise levels and noise performances obtained at different gains may vary greatly. It is difficult for the deep learning model to achieve a real and natural processing effect on noisy images in the entire gain range (the sample pairs in the public data set can be used to train the deep learning model, but there are large differences between the images in the public data set and the images in the image acquisition scenario, which means that the deep learning model cannot be effectively trained; and the existing deep learning models cannot perform reasonable image denoising on images with various gains and noise levels).

[0026] Moreover, the image denoising model only focuses on the image features of the input image during training. As a result, the trained image denoising model has a poor denoising effect on the image and is not suitable for image denoising tasks in actual scenes. It cannot perform reasonable image denoising on images with various gains and noise levels.

[0027] Based on this, the present application provides a method for training an image denoising model to improve the denoising effect of the trained image denoising model in actual scenarios.

[0028] The following first introduces a training method for an image denoising model provided by this application.

[0029] Among them, a training method for an image denoising model provided by the present application can be applied to an electronic device, and the electronic device is used to train the image denoising model. In a specific application, the electronic device can be a mobile phone, a computer, etc., and the present application does not limit the specific form of the electronic device. A training method for an image denoising model provided by the present application can be applied to a scenario of image denoising of a noisy image, and the image denoising model is trained based on a first type of sample pair of a target image and a noise image corresponding to the target image; and the target image can be an image in an image acquisition scenario, so as to train the image denoising model for the image acquisition scenario.

[0030] The following is an introduction to a training method for an image denoising model provided in an embodiment of the present application in conjunction with the accompanying drawings.

[0031] like Figure 1 As shown, a training method for an image denoising model provided in an embodiment of the present application may include the following steps:

[0032] S101: Obtain multiple first-category sample pairs in a target training set;

[0033] Wherein, each first-category sample pair includes a target image and a noise image corresponding to the target image, and the noise image in any first-category sample pair is generated for the target image in the first-category sample pair according to a predetermined noise image generation method; the predetermined noise image generation method includes: selecting a spare noise image from a plurality of spare noise images as the target noise image; determining the noise information to be used on each color channel of each pixel in the target noise image as the noise information to be used of the target noise image; performing a specified noise addition process on the target image based on the noise information to be used of the target noise image; determining the noise image corresponding to the target image based on the image after the specified noise addition process; each spare noise image is generated based on a target area in an initial image among the initial images; each initial image is an image obtained by photographing a target color card; the target color card includes a plurality of color blocks, and the target area is an image area of ​​a color block among the plurality of color blocks;

[0034] When training a target image denoising model, multiple first-category sample pairs in the target training set can be obtained first, each first-category sample pair including a target image and a noise image corresponding to the target image; thus, abundant first-category sample pairs can be obtained for model training.

[0035] For each target image, the present application can generate one or more noise images of the target image according to a predetermined noise image generation method, so as to construct a target training set and use the target training set to train the target image denoising model; in order to generate a noise image corresponding to the target image, the present application also provides multiple spare noise images, and a spare noise image can be selected from the multiple spare noise images as a target noise image to add noise to the target image and generate a noise image corresponding to the target image. The target image can be an image selected from noise-free images (which can be images captured in an image acquisition scenario), an image can be randomly selected from noise-free images as the target image, and a spare noise image can be randomly selected from multiple spare noise images as the target noise image. Different target noise images can be used to add noise to the same target image to generate multiple noise images of the target image, so as to enrich the number of first-category sample pairs. It should be emphasized that the method of selecting the target image from the clean images and the method of selecting the target noise image from multiple spare noise images can be the same or different; and the selection method can include: random selection, sequential selection, etc., which is not limited in this application.

[0036] It should be noted that each spare noise image is generated based on a target area in an initial image among the initial images, and each initial image is an image obtained by photographing a target color card. Exemplarily, in an optional implementation method, different parameter values ​​of target shooting parameters are used when shooting different initial images, and the target shooting parameters include environmental parameters and / or camera parameters; that is, each spare noise image is generated based on an image area of ​​a color block in an initial image under a target shooting parameter; exemplarily, each spare noise image can be understood as an image obtained by extracting image data from an image area of ​​a color block in an initial image under a target shooting parameter, and the diversity of the spare noise images is enriched by using different parameter values ​​of the target shooting parameters and color blocks of different colors. The present application does not limit the number of color blocks included in the target color card. All color cards existing in the related art can be used as the target color card of the present application. For example, the target color card can be a 24-color card, including 24 color blocks of different colors. The target color card can be placed in an area (such as an area in a light box), and then the light source, imaging size, focal length and other parameters are adjusted, that is, the environmental parameters and / or camera parameters are adjusted to capture different initial images.

[0037] In addition, the target shooting parameters include environmental parameters and / or camera parameters. Environmental parameters may include lighting conditions, such as the type and intensity of light sources. The types of light sources may include: natural light sources, artificial light sources, etc.; camera parameters may include internal and external parameters of the camera, etc., such as: aperture parameters, focal length parameters, parameters of each module in the ISP process (that is, the image processing process may include multiple processes, each process may be implemented by a module, each module may involve processing parameters corresponding to the module, and the processing parameters corresponding to each module may be adjusted), etc., and this application does not limit this. Among them, the initial image captured under the parameters of each module in the same ISP process can be understood as the initial image of the same ISP style, and when the parameters of each module in the ISP process are the same, other parameters (such as aperture parameters, focal length parameters or the type and intensity of the light source, etc.) may be the same or different.

[0038] It should be noted that, since a spare noise image is generated based on a target area in an initial image among the initial images, it does not matter whether the initial image contains a background area other than the target color card; that is, the initial image can be shot in any scene (or background) under a stable light source, and different initial images can be obtained by simply adjusting different target shooting parameters during shooting.

[0039] Among them, the method for generating the backup noise image will be introduced in detail in the subsequent embodiments and will not be elaborated here; and any method that can generate the backup noise image is applicable to the present application, and the present application does not limit this.

[0040] After determining the target noise image, the noise information to be used on each color channel of each pixel in the target noise image can be further determined as the noise information to be used for the target noise image, so as to adapt the target image for subsequent noise addition processing. The target image is usually an image in RGB format, and the noise information to be used on the R (red) channel, G (green) channel, and B (blue) channel of each pixel in the target noise image can be determined as the noise information to be used for the target noise image, which can be added to the target image.

[0041] After determining the noise information to be utilized of the target noise image, a designated noise addition process can be performed on the target image based on the noise information to be utilized of the target noise image. Exemplarily, in one implementation, the designated noise addition process includes a first type of noise addition process and a second type of noise addition process. The first type of noise addition process is image processing for performing noise addition on all pixels of the image to be processed; the second type of noise addition process includes at least one of a first image processing, a second image processing and a third image processing; the first image processing is image processing for performing noise addition on edge pixels of the image to be processed, the second image processing is image processing for performing noise addition on the image to be processed according to the brightness of the pixels, and the third image processing is image processing for performing noise addition on an image area of ​​a designated object of the image to be processed.

[0042] In the present application, the second type of noise addition processing includes the first image processing, the second image processing and the third image processing (only as an example, the second type of noise addition processing can also include only the first image processing, or include the first image processing and the second image processing, which is not limited in the present application), that is, noise is added through various dimensions to add real noise to the image to be processed. In addition, the specified noise addition processing can also include the processing of directly adding the target noise image to the target image (such as adding the pixel value of the pixel point on the target noise image to the pixel value of the corresponding pixel point on the target image), which is not limited in the present application.

[0043] The target image is a noise-free image, which can be understood as a high-quality image. The noise image corresponding to the target image contains the same object, object position, background, etc. as the target image. The target image is a noise-free, high-quality image, and the noise image of the target image is a low-quality noisy image in the same scene (there is a difference in clarity between the target image and the noise image corresponding to the target image (for example, the clarity of the target image is greater than the clarity of the noise image corresponding to the target image), but they are usually not compared by clarity; the difference between the two is that the target image is a noise-free, high-quality image, and the noise image corresponding to the target image is a low-quality image with noise added to the target image). Among them, any noisy image can be used as a target image to generate a noise image corresponding to the image, and this application does not limit this. In addition, after generating the noise image corresponding to the target image, a first type of sample pair can be constructed and stored in the target training set.

[0044] For each target image, one or more noise images of the target image can be generated through the above steps. Any noise image of the target image can be a noise image obtained through cropping, blurring, noise addition (through a target noise image, performing a first type of noise addition processing, a first image processing, a second image processing and a third image processing) and quality compression (or only noise addition, etc., not limited). The target noise images used for noise addition of the multiple noise images of the target image are different.

[0045] Afterwards, a target training set containing multiple sample pairs can be obtained based on multiple target images and corresponding noise images, and the target training set can be used for training the image denoising model; wherein each sample pair includes a target image and a noise image corresponding to the target image, and in different sample pairs, the target image included can be the same target image, and the corresponding noise images are different, for example: sample pair 1 includes target image 1 and noise image 1 corresponding to target image 1, and sample pair 2 includes target image 1 and noise image 2 corresponding to target image 1, and the present application does not limit this.

[0046] S102: Inputting the noisy image and the corresponding image quality score in each first-category sample pair into the target image denoising model to be trained, so as to obtain a denoising result corresponding to the noisy image in each sample pair;

[0047] The target image denoising model includes an initial denoising module and an image content restoration module connected in series, wherein the initial denoising module is used to process the input image to obtain a feature map of the same size as the input image; the image content restoration module is used to perform feature splicing processing on the feature map obtained by the initial denoising module and the image quality score input to the image content restoration module to obtain spliced ​​image data; the spliced ​​image data is processed to generate a denoising result of the image input to the initial denoising module;

[0048] The target image in each first-category sample pair can be used as the true value for subsequent loss calculation; the noisy image in each first-category sample pair can be input into the target image denoising model to be trained, and the target image denoising model includes an initial denoising module and an image content restoration module connected in series, and the initial denoising module is used to remove the extremely large noise (noise block) in the image and generate a feature map of the same size as the input image for subsequent denoising processing; the image content restoration module performs feature splicing processing on the feature map obtained by the initial denoising module and the input image quality score, and processes the spliced ​​image data to generate a denoising result corresponding to the image input to the initial denoising module, that is, to obtain the denoising result corresponding to the noisy image in each first-category sample pair.

[0049] In the present application, a quality score of an image may also be generated, and the predetermined noise image generation method may further include:

[0050] In response to the completion of any image processing, a quality score of an image obtained after the image processing is calculated based on an image quality score of the image processed by the image processing, a predetermined processing parameter used by the image processing, a current image quality degradation parameter corresponding to the image processing, and a current weight parameter set for the predetermined processing parameter; wherein the image quality degradation parameter is used to characterize a degree of image degradation;

[0051] And, after obtaining the quality score of the image obtained by the image processing, based on the quality score of the image obtained by the image processing and a preset prior score, the current image quality degradation parameter and the current weight parameter set for the predetermined processing parameter are adjusted.

[0052] After any image processing process (any image processing process including the first type of noise addition processing, the first image processing, the second image processing, the third image processing and the blurring processing, etc.) is completed, the present application can calculate the quality score of the image obtained after this type of image processing based on the image quality score of the image processed by this type of image processing, the current image quality degradation parameter corresponding to this type of image processing (each image processing process has a corresponding current image quality degradation parameter, that is, the image quality score that the image processing process can currently reduce, for example: the blurring processing can currently reduce the quality score of the image by 0.1, such as: before the blurring processing, the quality score of the image is 1, and after the blurring processing, the quality score of the image is 0.9), the predetermined processing parameters used in this type of image processing (which can be understood as the degradation parameters used in the image processing process) and the current weight parameters set for the predetermined processing parameters, so as to characterize the image quality after the completion of this type of image processing. Among them, the image quality degradation parameter can characterize the degree of image degradation. For any processing process among the first type of noise addition processing, the first image processing, the second image processing and the third image processing, the image quality degradation parameter of the processing process is used to characterize the degree of image noise addition; and the image quality score is used to characterize the quality of the image. The numerical range of the quality score is (0, 1), and the image quality score that can be currently reduced in each processing process is related to the predetermined processing parameters of the processing process.

[0053] Exemplarily, the quality score of the image obtained by the previous image processing process of the image processing process (i.e., the image quality score of the image processed by the image processing, which can be understood as the quality score of the image before the image processing), the current image quality degradation parameter corresponding to the image processing, the predetermined processing parameter used by the image processing, and the product of the current weight parameter set for the predetermined processing parameter can be subtracted to obtain the quality score of the image after the image processing process is completed. In addition, in the actual application process, for each degraded processing process, the image quality score that can be reduced by the processing process (i.e., the product of the current image quality degradation parameter corresponding to the image processing, the predetermined processing parameter used by the image processing, and the current weight parameter set for the predetermined processing parameter) can be calculated, and the quality score of the target image can be subtracted from the image quality score that can be reduced by each degraded processing process to obtain the quality score of the noise image corresponding to the target image. For example, if the quality score of the target image is set to 1, the quality score of the noise image of the target image obtained can be: 1-0.1-0.05-0.2-0.05=0.6. It should be noted that any method that can calculate the quality score of the noise image is applicable to the present application, and the above calculation method should not constitute a limitation on the present application.

[0054] Exemplarily, the image quality score after the image processing process is completed can be calculated according to the following formula:

[0055] ; (1)

[0056] in, and are the image quality scores corresponding to the low-quality degraded data obtained in degradation step S-1 and degradation step S respectively ( That is, the image quality score after the image processing process is completed). is the image quality degradation parameter of the Sth step, is the degradation parameter value corresponding to the degradation step (i.e., the predetermined processing parameter), is the weight coefficient (i.e., the weight parameter of the predetermined processing parameter).

[0057] Furthermore, in order to ensure that the quality scores of images of the same quality are matched (i.e., to ensure that the quality scores of the images are accurate), the present application may also adjust the current image quality degradation parameter and the current weight parameter set for the predetermined processing parameter based on the quality score of the image obtained by the image processing and the preset prior score after obtaining the image quality score obtained by the image processing (i.e., to adjust the image quality degradation parameter based on the image quality score obtained by the image processing and the preset prior score). as well as The image quality score obtained after the adjustment is adjusted to match the preset prior score. The preset prior score can be understood as the real quality score corresponding to the image (the prior information used to indicate the quality level corresponding to the image data, such as the noise level, image visibility information, etc.).

[0058] It can be seen that in the present application, at the end of any image processing process, a quality score of the image obtained after the image processing can be generated to characterize the image quality; and, in order to ensure the accuracy of the image quality score, the image quality degradation parameters and the current weight coefficients set for the predetermined processing parameters can be adjusted; thereby, the noise image corresponding to the obtained target image has an accurate, degraded quality score, so that more features can be used to train the target image denoising model.

[0059] S103: determining a target loss value of a target image denoising model based on a denoising result corresponding to the noise image in each first-category sample pair and a target image in each first-category sample pair;

[0060] After obtaining the denoising result of the noisy image, the target loss value of the target image denoising model can be determined based on the denoising result corresponding to the noisy image of each first-category sample pair and the target image (ie, the true value) in each first-category sample pair.

[0061] Exemplarily, based on the denoising results corresponding to the noise image in each first-category sample pair and the target image in each first-category sample pair, determining the target loss value of the target image denoising model includes: calculating the first-category loss value, the second-category loss value and the third-category loss value based on the denoising results corresponding to the noise image in each first-category sample pair and the target image in each first-category sample pair; determining the target loss value of the target image denoising model based on the first-category loss value, the second-category loss value and the third-category loss value; wherein the first-category loss value is a loss value obtained based on the difference in color dimension between the target image in each first-category sample pair and the denoising results corresponding to the corresponding noise image; the second-category loss value is a loss value obtained based on the absolute value of the difference in pixel value between the target image in each first-category sample pair and the denoising results corresponding to the corresponding noise image; the third-category loss value is a loss value obtained based on the feature difference in a specified feature space between the target image in each first-category sample pair and the denoising results corresponding to the corresponding noise image.

[0062] In the present application, taking into account the differences in color dimension, pixel value dimension and specified feature space dimension, the first type of loss value representing the difference in color dimension, the second type of loss value regarding the absolute value of the difference in pixel value, and the third type of loss value of the feature difference in the specified feature control can be calculated based on the denoising result corresponding to the noise image in each first type of sample and the target image in each first type of sample pair; and based on the first type of loss value, the second type of loss value and the second type of loss value, the target loss value of the target image denoising model is determined, for example: directly summing the first type of loss value, the second type of loss value and the second type of loss value, or assigning corresponding weight coefficients to the first type of loss value, the second type of loss value and the second type of loss value, and then summing them, etc., which is not limited in the present application.

[0063] Exemplarily, the target loss value may be calculated according to the following formula:

[0064] ; (2)

[0065] in, is the total loss function of the model (i.e., the calculation function of the target loss value), represents the loss function of the whole image, Represents the loss function of the image mask area, is the weight of the mask area loss function.

[0066] ; (3)

[0067] ; (4)

[0068] Among them, mask is the mask image of the facial features area at the edge of the image obtained by the segmentation model Seg_net. is the weight coefficient of perceptual loss, is the calculation function of the first type of loss value, is the calculation function of the second type of loss value, is the calculation function of the third type of loss value, represents the prediction result of the model (denoising result), and gt is the target image.

[0069] ; (5)

[0070] Where N is the number of samples, represents the prediction result of the model (denoising result), and gt represents the HQ image, that is, the target image.

[0071] ; (6)

[0072] ; (7)

[0073] in, and They respectively represent the denoising result of the noise image and the feature representation of the target image in the i-th layer in the pre-trained neural network, and M is the number of feature layers. It should be noted that the above method of calculating the target loss value is only an example, and any method that can calculate the target loss value of the target image denoising model is applicable to this application (for example: calculating the cosine similarity or Euclidean distance between the denoising result corresponding to the noise image and the target image, etc.), and this application does not limit this.

[0074] S104: In response to determining that the target image denoising model has not converged according to the target loss value, adjusting parameters of the target image denoising model, and returning to the step of obtaining multiple first category sample pairs in the target training set.

[0075] After determining the target loss value of the target image denoising model, it can be judged whether the target image denoising model converges based on the target loss value. If it converges, such as: the target loss value is less than the preset threshold, the trained target image denoising model is directly obtained; if it does not converge, such as: the target loss value is not less than the preset threshold, it is necessary to continue training the target image denoising model, and the model parameters of the target image denoising model can be adjusted first, such as: adjusting the model parameters of the initial denoising module and the image content restoration module respectively, and re-obtaining multiple first-category sample pairs in the target training set for training.

[0076] Among them, for each module, the parameters of the module can be adjusted so that the output result based on the adjusted parameters is closer to the target image in the first type of sample pair. The specific adjustment method of the parameters can be similar to the existing technology and will not be repeated here.

[0077] In the technical solution of the present application, the operations of acquiring, storing, using, processing, transmitting, providing and disclosing the target image, backup noise image, noise information and the noise image corresponding to the target image, etc., are all carried out with the user's authorization.

[0078] The embodiment of the present application provides a method for training an image denoising model, in which the noise image in each first-category sample pair is an image obtained by performing a specified noise adding process on the target image based on the noise information to be used of the target noise image, and the target noise image is generated based on the color block area in the initial image obtained by photographing the target color card, so that the noise information added to the noise image is more in line with the real noise information. Since the target training set is suitable for image denoising tasks (i.e., the data authenticity of the sample pairs is good); and when training the target image denoising model, the present application takes into account the noise image and the corresponding image quality score, the target image denoising model can use more features to adaptively denoise the noise image, and can achieve reasonable denoising of the noise image based on the image quality score of the noise image. It can be seen that the noise reduction effect of the trained image denoising model in actual scenes can be improved through this solution. The noise information added to the noise image is closer to the real noise information, the sample pairs of the training set have good authenticity and data richness, and the data volume is large; and, the present application can denoise the noise image in combination with the image quality score corresponding to the noise image, and can achieve reasonable denoising of images with various noise levels based on the image quality score of the noise image.

[0079] Compared with the training set constructed by the public data set, the target training set provided by the present application can generate one or more noise images of each target image by using a variety of target images and target noise images, and can generate a training set suitable for adapting to the real image denoising scene; compared with the training set constructed by the artificial shooting data set, the present application can automatically generate one or more noise images of the target image according to the target image and the target noise image, and the number and types of sample pairs contained in the constructed training set are rich, and the construction efficiency of the training set is greatly improved; compared with the training set constructed by the image degradation data set of other image degradation methods, the present application obtains the real noise image corresponding to the target image by adding noise to all pixels of the image to be processed, and adding noise to edge pixels, adding noise according to the brightness of pixels and / or adding noise to the image area of ​​the specified object; the noise image generated by the present application is a noise image in which real noise is added to the target image, and the constructed target training set is suitable for image denoising tasks. It can be seen that through this application, a real training set suitable for image denoising tasks can be generated, thereby improving the denoising effect of the trained image denoising model in actual scenarios.

[0080] The process of generating a noise image corresponding to the target image, that is, the process of performing image degradation on the target image, may include other steps in addition to adding noise; optionally, in another embodiment of the present application, based on the noise information to be used of the target noise image, before performing the specified noise adding process on the target image, the predetermined noise image generation method further includes:

[0081] Performing a specified preprocessing on the target image to obtain a preprocessed target image; wherein the specified preprocessing includes: image processing for cropping an image of a target size and / or image processing for blurring the image;

[0082] Based on the noise information to be utilized of the target noise image, a specified noise adding process is performed on the target image, including:

[0083] Based on the noise information to be utilized of the target noise image, performing a specified noise adding process on the preprocessed target image;

[0084] Before performing the specified noise addition processing on the target image, the target image can also be subjected to specified preprocessing, that is, the target image is cropped and / or image blurred to obtain a preprocessed target image; illustratively, the target image can be cropped based on a predetermined cropping function, such as a crop function (a function for lossless cropping of images) to obtain an image of a target size, wherein the target size can be 256, that is, the image of the target size is an image with a length and a width of 256; and the target image or the cropped target image can be blurred based on a blur kernel to obtain a preprocessed target image. Adaptively, the preprocessed target image is subjected to specified noise addition processing based on the noise information to be utilized of the target noise image.

[0085] In one implementation, the target image may be cropped, blurred, denoised, and subsequently quality compressed according to the following formula to obtain a noise image corresponding to the target image:

[0086] ; (8)

[0087] in, For cutting processing, is the convolution operation, K represents the blur kernel of the blur processing, represents noise addition processing (which may include first type noise addition processing, first image processing, second image processing, and third image processing noise addition), represents the quality compression of the JPEG format with a quality factor of q (for example only and not limiting), For high-quality target images, is the noise image corresponding to the degraded target image.

[0088] In this application, different target noise images may have different noise intensities. Based on the noise information to be utilized of the target noise image, the first type of noise addition processing includes:

[0089] Acquire a first noise addition coefficient corresponding to the target noise image; based on the noise information to be utilized of the target noise image and according to the first noise addition coefficient corresponding to the target noise image, add noise to all the pixel points of the image to be processed;

[0090] Among them, the method for determining the first noise coefficient corresponding to any backup noise image includes: based on the pixel value information of each pixel point of the backup noise image, determining the evaluation value of the backup noise image used to characterize the noise intensity as the target evaluation value; based on the pre-set correspondence between each evaluation value range and the value range of the noise coefficient, determining the value range corresponding to the evaluation value range to which the target evaluation value belongs, and selecting a value in the determined value range as the first noise coefficient corresponding to the backup noise image; wherein the evaluation value range in the corresponding relationship is negatively correlated with the value range.

[0091] When performing the first type of noise addition processing, a first noise addition coefficient corresponding to the target noise image can be first obtained, and then based on the noise information to be used of the target noise image and according to the first noise addition coefficient corresponding to the target noise image, all pixel points of the image to be processed (the target image, or the preprocessed target image, that is, the image obtained by the previous processing process of the noise addition processing process) are denoised.

[0092] In the present application, a corresponding first noise addition coefficient is set based on the noise intensity of any spare noise image, and the first noise addition coefficient can characterize the overall addition ratio of the spare noise image when it is added to a target image. The value range of the first noise addition coefficient of any spare noise image can be (0, 1); based on the pixel value information of each pixel point of the spare noise image, an evaluation value for characterizing the noise intensity of the spare noise image can be determined, such as: the value of the variance of each pixel point of the spare noise image is used as the target evaluation value; based on the pre-set correspondence between each evaluation value range and the value range of the noise addition coefficient, a value range corresponding to the evaluation value range to which the target evaluation value belongs is determined, and a value in the determined value range is selected as the first noise addition coefficient corresponding to the spare noise image; considering the authenticity of the noise addition, the higher the target evaluation value of any spare noise image, that is, the higher the noise intensity, the lower the noise addition coefficient should be, that is, the evaluation value range is negatively correlated with the value range. For example, the ranges of evaluation values ​​include: (0, T1], (T1, T2], (T2, T3], etc., and the corresponding ranges of noise coefficients are [k1, k2], [k3, k4], [k5, k6], etc., where k1>k3>k5, k2>k4>k6.

[0093] In addition, when performing the first type of noise addition processing, the noise information of each pixel in the noise information to be used of the target noise image (the size of the target noise image can be the same as the size of the target image or the image of the target size, and the pixels correspond one to one) can be multiplied by the first noise addition coefficient corresponding to the target noise image to obtain the first type of noise to be added to each pixel, and then the first type of noise of each pixel is added to the pixel corresponding to the image to be processed, thereby realizing the noise addition of all pixels.

[0094] Exemplarily, the first type of noise addition processing may be performed according to the following formula:

[0095] ; (9)

[0096] in, and represent the images obtained after degradation step S-1 and degradation step S, respectively. represents the noise information to be used in the target noise image, k represents the ratio of noise addition, that is, the first noise addition coefficient, and i represents the value of each pixel point in the color channel of the target image.

[0097] In the present application, in order to add noise more realistically, in addition to adding noise to all pixels, noise can also be added according to at least one of the first image processing, the second image processing and the third image processing. The specified object can be any object in the image, for example, a facial object, a vehicle object, a tree object, etc. The image area of ​​the facial object can include: an eye area, a nose area and a mouth area, etc. The image area of ​​the vehicle object can include: a headlight area and a window area, etc. It should be noted that for any processing process, the so-called image to be processed is the image obtained by the previous processing process of the processing process, for example: for the first image processing, the image to be processed is the image obtained by the first type of noise addition processing; for the second image processing, the image to be processed can be the image obtained by the first type of noise addition processing or the image obtained by the first image processing (the previous processing process of the second image processing process can be the processing process of the first noise addition processing or the processing process of the first image processing); and in the present application, the order of the processing processes of the first image processing, the second image processing and the third image processing is not limited.

[0098] In the present application, the image is denoised by taking into account the edge pixels, brightness and darkness of the image, and the noise requirements of the image area of ​​the specified object to obtain a more realistic noise image corresponding to the target image.

[0099] Exemplarily, the present application can perform differential noise addition on the image according to the edge pixels, brightness and darkness of the image, and the image area of ​​the specified object, and perform a first image processing method based on the noise information to be utilized of the target noise image, including:

[0100] Perform edge detection on the target image to obtain the target position of each pixel point on the edge; wherein the current image to be processed is an image to be subjected to noise addition processing based on edge information;

[0101] According to the first method or the second method, based on the noise information to be used of the target noise image, the image to be processed is subjected to noise processing; wherein the first method is: for each pixel point at the target position of the image to be processed, based on the noise information to be used of each color channel of the pixel point located at the target position in the noise information to be used of the target noise image, and according to the second noise addition coefficient, the pixel point at the target position of the image to be processed is subjected to noise processing; the second method is: for each pixel point at the designated position of the image to be processed, based on the noise information to be used of each color channel of the pixel point located at the designated position in the noise information to be used of the target noise image, and according to the second noise addition coefficient, the pixel point at the designated position of the image to be processed is subjected to noise processing, wherein each designated position is a target position among the target positions, the pixel value representing the edge strength of which is greater than a predetermined threshold value;

[0102] And / or, a processing method for performing a second image processing based on the noise information to be utilized of the target noise image includes:

[0103] For each pixel in the image to be processed, based on the noise information to be used of each color channel of the designated pixel in the noise information to be used of the target noise image, and according to the third noise addition coefficient and the noise addition principle, the pixel is noised, wherein the designated pixel is a pixel in the target noise image at the same position as the pixel, and the noise addition principle is that the noise addition degrees of pixels with different brightness in the target image are different;

[0104] And / or, a third image processing method is performed based on the noise information to be utilized of the target noise image, including:

[0105] Performing region detection on the target image with respect to the specified object to obtain a mask image of the specified object in the target image;

[0106] Based on the noise information to be utilized of the target noise image, and according to the fourth noise addition coefficient and the obtained mask image, the pixel points related to the designated object in the image to be processed are noised.

[0107] When performing the first image processing, edge detection can be performed on the target image first (any edge detection algorithm can be used to perform edge detection) to obtain the target position of each pixel at the edge. At this time, the current image to be processed is an image to be subjected to noise processing based on edge information; thereafter, the image to be processed can be subjected to noise processing based on the noise information to be utilized of the target noise image in the first manner or the second manner; the first manner is that for each pixel at the target position of the image to be processed, the pixel at the target position is subjected to noise according to the second noise coefficient based on the noise information to be utilized of each color channel of the pixel at the target position in the noise information to be processed of the target noise image; the second manner is that the pixel at the specified position is subjected to noise in the above manner, that is, the target position at which the pixel representing the edge strength is greater than a predetermined threshold is subjected to noise, and differential noise addition for edge pixels can be achieved through the second manner.

[0108] Among them, the second noise addition coefficient can characterize the addition ratio of the noise information to be used of the target noise image when it is added to the image to be processed with respect to the edge. When performing the first image processing, with respect to the first method, for each pixel point at the target position, the noise information to be used in the noise information to be used of the target noise image and the noise information of each color channel of the pixel point located at the target position can be multiplied by the second noise addition coefficient to obtain the first image noise to be added to the pixel point at each target position, and then the first image noise at each target position is added to the pixel point corresponding to the image to be processed. With respect to the second method, for each pixel point at a specified position, the noise information to be used in the noise information to be used of the target noise image and the noise information of each color channel of the pixel point located at the specified position can be multiplied by the second noise addition coefficient to obtain the second image noise to be added to the pixel point at each specified position, and then the second image noise at each specified position is added to the pixel point corresponding to the image to be processed.

[0109] Exemplarily, in one implementation, the following formula may be used to perform edge detection on the target image:

[0110] ; (10)

[0111] in, is an edge detection function, is the edge detection result, is the target image.

[0112] In this application, the edge information of the target image is obtained in the following manner:

[0113] ; (11)

[0114] in, , ; is the edge detection operator in the horizontal direction, is the edge detection operator in the vertical direction, It is the grayscale image information corresponding to the target image.

[0115] And, the following formula may be used to perform noise addition in the first image processing according to the first method:

[0116] ; (12)

[0117] Alternatively, the following formula is used to perform differential noise addition for the first image processing according to the second method:

[0118] ; (13)

[0119] ; (14)

[0120] in, is the low-quality noisy image obtained in the degradation step, is the image degraded in the previous step, The weight coefficient for adding noise to the edge area of ​​the image (the second noise coefficient), is the image edge information obtained by calculation, is the maximum value of the image edge information, which is used to normalize the edge signal. is the noise information to be used in the target noise image, is the threshold parameter set, For The judgment result is 0 or 1, c is the color channel, and (x, y) is the coordinate of the pixel.

[0121] When performing the second image processing, the image to be processed is an image to be noised based on the brightness of the pixel points. For each pixel in the image to be processed, based on the noise information to be used in the target noise image and the noise information of each color channel of the specified pixel point, the pixel point is noised according to the third noise addition coefficient and the noise addition principle; wherein, in the present application, the third noise addition coefficient is determined based on the brightness of the target image (or understood as the brightness), and is negatively correlated with the brightness of the target image, that is, the brighter the target image, the lower the third noise addition coefficient, and the darker the target image, the higher the third noise addition coefficient; and the noise addition degrees of pixels with different brightness in the target image are different, the higher the brightness, the lower the noise addition degree, and the lower the brightness, the higher the noise addition degree, so as to reasonably perform the noise addition of the second image processing.

[0122] Among them, the third noise addition coefficient can represent the addition ratio of the noise information to be used of the target noise image when it is added to the image to be processed regarding the brightness and darkness. When performing the second image processing, for each pixel in the image to be processed, the noise information to be used of the target noise image and the noise information to be used of each color channel of the specified pixel can be multiplied by the third noise addition coefficient respectively to obtain the third image noise of the specified pixel, and according to the third principle, the third image noise of the specified pixel is added to the corresponding pixel in the image to be processed.

[0123] Exemplarily, in one implementation, the noise addition of the second image processing may be performed according to the following formula:

[0124] ; (15)

[0125] in, is the low-quality noisy image obtained in the degradation step, is the image degraded in the previous step, is the target image, The weight coefficient for adding noise to the bright and dark areas of the image, that is, the third noise coefficient.

[0126] When performing the third image processing, the image to be processed is an image to be noised based on the image area of ​​the specified object. The target image can first be subjected to area detection regarding the specified object (for example, area detection of the specified object is performed through a fully convolutional image segmentation neural network) to obtain a mask image of the specified object in the target image (i.e., a mask image, which can be a binary image); then, based on the noise information to be utilized of the target noise image, and in accordance with the fourth noise addition coefficient and the obtained mask image, the pixel points regarding the specified object in the image to be processed can be noised.

[0127] Among them, the fourth noise addition coefficient can represent the ratio of noise addition for the image area of ​​the designated object when the noise information to be used of the target noise image is added to the image to be processed. When performing the third image processing, the noise information of the pixel points belonging to the designated object in the noise information to be used of the target noise image (determine whether the pixel points belong to the designated object according to the mask image) can be multiplied by the fourth noise addition coefficient to obtain the fourth image noise to be added to each pixel point belonging to the designated object, and then the fourth image noise of each pixel point belonging to the designated object is added to the pixel point corresponding to the image to be processed.

[0128] Exemplarily, in one implementation, the noise addition of the third image processing may be performed according to the following formula:

[0129] ; (16)

[0130] ; (17)

[0131] in, is the low-quality noisy image obtained by this step degradation, is the image degraded in the previous step, is the target image (i.e., high-quality HQ image), is the weight coefficient for adding noise to the facial mask area in the target image, i.e., the fourth noise adding coefficient. To display the binary images of the eyes, nose, and mouth areas in the target image (the designated objects may be the facial features of the human body, which is only an example and not limited to); It is a deep fully convolutional neural network for image segmentation.

[0132] It should be noted that the above-mentioned second noise coefficient, third noise coefficient and fourth noise coefficient are noise coefficients for adding noise to edge pixels, brightness and darkness, and image areas of designated objects, respectively (other pixels that do not belong to edge pixels are not noised, which can be understood as the second noise coefficient being 0; pixels that do not belong to the image area of ​​the designated object are not noised, which can be understood as the fourth noise coefficient being 0), the third noise coefficient is determined based on the brightness and darkness of the target image, the second noise coefficient, the third noise coefficient and the fourth noise coefficient may be the same or different, and their value range may be (0.1, 0.5). Through each noise coefficient, reasonable edge noise, brightness and darkness noise and / or noise in the image area of ​​the designated object can be performed; and any method that can implement the above-mentioned first type of noise processing and second type of noise processing is applicable to the present application, and the present application does not limit this.

[0133] After the specified noise adding process, the noise image corresponding to the target image can be determined based on the image after the specified noise adding process.

[0134] Exemplarily, determining a noise image corresponding to a target image based on a specified image after noise processing includes:

[0135] Perform quality compression on the specified noise-added image to obtain the noise image corresponding to the target image.

[0136] The image after the specified noise processing can also be quality compressed to obtain a noise image corresponding to the target image. When quality compression is performed, format conversion can also be performed, and the quality compression can be quality compression for a specified format, and the specified format can include: JPEG (Joint Photographic Experts Group, a common image format), PNG (Portable Network Graphics, a lossless compressed bitmap graphic format); quality compression can be performed according to the quality factor of the specified format, and this application does not limit this.

[0137] Optionally, in another embodiment of the present application, Figure 2 As shown, the generation method of any spare noise image includes:

[0138] S201: Determine an image region of a color block from an initial image to obtain a target region;

[0139] S202: for each pixel point in the target area, extracting target noise information on each color channel corresponding to the pixel point according to a predetermined extraction method;

[0140] The predetermined extraction method includes determining, for each color channel, the target noise information of the color channel corresponding to the pixel point based on the value of the color channel of the pixel point and the color mean value corresponding to the color channel; the color mean value corresponding to any color channel is: the value obtained by averaging the values ​​of the color channel of each pixel point in the target area;

[0141] S203: generating a backup noise image corresponding to the target area of ​​the initial image based on the target noise information of each color channel corresponding to each pixel point of the target area;

[0142] The values ​​of each color channel of each pixel in the backup noise image are determined based on the target noise information of each color channel corresponding to a pixel in the target area;

[0143] When generating a backup noise image, firstly, an image area of ​​a color block is determined from an initial image to obtain a target area; for each pixel point in the target area and for each color channel, based on the value of the color channel of the pixel point and the color mean value corresponding to the color channel, the target noise information of the color channel corresponding to the pixel point is determined, and the target noise information on each color channel corresponding to the pixel point is extracted; based on the target noise information of each color channel corresponding to each pixel point in the target area, a backup noise image corresponding to the target area of ​​the initial image is generated.

[0144] It should be noted that the values ​​of each color channel of each pixel in the backup noise image are determined based on the target noise information of each color channel corresponding to a corresponding pixel in the target area. For example, the target noise information of each color channel corresponding to a pixel in the target area is directly determined as the values ​​of each color channel of the corresponding pixel in the backup noise image, that is, the target noise information of the R channel, G channel and B channel corresponding to the pixel in the target area is respectively determined as the values ​​of the R channel, G channel and B channel of the corresponding pixel in the backup noise image.

[0145] Exemplarily, for each color channel, based on the value of the color channel of the pixel and the color mean corresponding to the color channel, determining the target noise information of the color channel corresponding to the pixel includes:

[0146] For each color channel, the value of the pixel on the color channel is subtracted from the color mean value corresponding to the color channel to obtain an intermediate value, and the intermediate value is summed with a predetermined offset value to obtain the target noise information of the color channel corresponding to the pixel;

[0147] Determining the noise information to be utilized on each color channel of each pixel in the target noise image as the noise information to be utilized of the target noise image includes: subtracting a predetermined offset value from the value of each color channel of each pixel in the target noise image to obtain the noise information to be utilized on each color channel of each pixel; and determining the noise information to be utilized on each color channel of each pixel as the noise information to be utilized of the target noise image.

[0148] When determining the target noise information of each pixel for each color channel, for each color channel, first subtract the color mean value corresponding to the color channel from the value of the pixel on the color channel to obtain an intermediate value. Since the intermediate value may be a negative value, for the convenience of storage, the intermediate value and the predetermined bias value may be summed to obtain the target noise information of the color channel corresponding to the pixel, and the target noise information of the color channel corresponding to the pixel is stored. The predetermined bias value is used to correct the intermediate value so that the corrected value of the target noise information of the color channel corresponding to the pixel meets the predetermined range, so as to save the target noise information.

[0149] Adaptively, since the target noise information of each color channel corresponding to the pixel point is the noise information with the predetermined offset value added, when adding noise, if the noise information to be used on each color channel of the pixel point needs to be used, the value of each color channel of each pixel point in the target noise image can be subtracted from the predetermined offset value to obtain the noise information to be used on each color channel of each pixel point, and determined as the noise information to be used of the target noise image (i.e., the real noise information to be added), so as to add the real noise information to be used of the target noise image to the target image. Exemplarily, the predetermined offset value can be 128, which can be adjusted according to actual needs or experience, and this application does not limit this.

[0150] Exemplarily, in one implementation, the following formula may be used to extract the target noise information:

[0151] ; (18)

[0152] in, Represents the pixel value of color channel i, Represents the target noise information of the pixel in the color channel, i is the data of the color channel R / G / B, Represents the corresponding color mean of the color channel, and 128 is the predetermined bias value.

[0153] And, when adding noise, the following formula can be used to calculate the real noise information to be used:

[0154] ; (19)

[0155] Represents the noise information to be used extracted from the color channel of the target noise image. Represents the value of the color channel of the target noise image, and 128 is the offset value.

[0156] It can be seen that in the present application, an image area of ​​a color block, i.e., a target area, can be determined from an initial image, and the target noise information of each color channel corresponding to each pixel can be extracted to generate a backup noise image of the initial image; and, by means of a predetermined bias value, the target noise information of each color channel corresponding to each pixel can be stored, and the bias value can be subsequently eliminated to add the real noise information to be used of the target noise image to the target image.

[0157] Optionally, in another embodiment of the present application, the initial denoising module in the target image denoising model is a pre-trained module; the method of pre-training the initial denoising module includes:

[0158] Acquire multiple second sample pairs; input the noise image in each second sample pair into the initial denoising module to obtain a feature map corresponding to the noise image in each second sample pair output by the initial denoising module; determine the loss value of the initial denoising module based on the feature map corresponding to the noise image in each second sample pair and the auxiliary image in each second sample pair; in response to determining that the initial denoising module has not converged according to the loss value of the initial denoising module, adjust the parameters of the initial denoising module, and return to the step of acquiring multiple second sample pairs; wherein each second sample pair includes an auxiliary image and a noise image corresponding to the auxiliary image, the noise image corresponding to the auxiliary image is an image after the auxiliary image is denoised according to a target denoising method, and the target denoising method is a method for adding noise of specified particles.

[0159] In the present application, the initial denoising module can be a pre-trained module, and a plurality of second sample pairs can be obtained, namely, a sample pair of an auxiliary image and a noise image corresponding to the auxiliary image. The noise image corresponding to the auxiliary image is an image after the auxiliary image is denoised according to a target denoising method. The target denoising method is used to add noise of specified particles, and the specified particles can be extra-large particles, that is, the target denoising method can be used to add noise of extra-large particles; and the method of adding noise of specified particles to the auxiliary image in the target denoising method can be similar to the method of the first type of denoising processing, except that the noise coefficient of the target denoising method can be larger, which is not limited in the present application. In addition, there is a difference in clarity between the auxiliary image and the noise image corresponding to the auxiliary image (for example, the clarity of the auxiliary image is greater than the clarity of the noise image corresponding to the auxiliary image), but the comparison is usually not made by clarity. The auxiliary image is a high-quality image without noise, and the noise image corresponding to the auxiliary image is a low-quality image with noise added to the auxiliary image. The auxiliary image and the noise image corresponding to the auxiliary image may be similar to the target image and the noise image corresponding to the target image. The difference lies in that the noise image corresponding to the target image is a noise image obtained by subjecting the target image to the first type of noise addition processing and the second type of noise addition processing, and the noise image corresponding to the auxiliary image is a noise image obtained by subjecting the auxiliary image to the target noise addition method (which may be similar to the first type of noise addition processing).

[0160] Then, the noise image in each second sample pair is input into the initial denoising module, and the initial denoising module performs large particle denoising on the input noise image and generates a corresponding feature map, that is, the feature map corresponding to the noise image in each second type sample pair is obtained; the loss value of the initial denoising module can be determined based on the feature map of the noise image in each second type sample pair and the auxiliary image in each second type sample pair (specifically, the feature map of the auxiliary image); according to the loss value, it can be judged whether the initial denoising module converges. If not, the parameters of the initial denoising module can be adjusted, and multiple second sample pairs can be re-acquired for continued training. Among them, the noise image corresponding to the auxiliary image in the second type sample pair can be understood as a noise image in which only the auxiliary image is subjected to the first type of noise addition processing.

[0161] In addition, the image content restoration module can perform splicing processing on the feature map obtained by the initial denoising module and the image quality score input into the image content restoration module (the image quality score is the image quality score of the noisy image input into the initial denoising module; that is, the noisy image is input into the initial denoising module, and the image quality score corresponding to the noisy image is input into the image content restoration module) to obtain spliced ​​image data; the spliced ​​image data can be understood as a feature map containing the characteristics of the image quality score. The image content restoration module performs denoising processing on the noisy image under the premise of taking the image quality score into consideration to generate a denoising result of the noisy image.

[0162] It can be seen that the target image denoising module and the image content restoration module trained in the present application can also utilize the image quality score of the noise image to perform content restoration, i.e., denoising, on the noise image, and can reasonably generate denoising results for noise images with different image quality scores.

[0163] Optionally, the image content restoration module includes a predetermined attention mechanism module. The image content restoration module in the present application includes a predetermined attention mechanism module, which may be an image restoration attention module (Restoration Attention Block, RAB module). The image restoration attention module may include: an upper convolution module, a lower convolution module, a convolution module, and a CBAM module (Convolutional Block Attention Module, an attention mechanism module of the convolution module, which is a module that combines channel attention and spatial attention). Its specific structure will be described in detail in subsequent embodiments and will not be repeated here.

[0164] The present application designs a target image denoising model based on image score prior information, and the target image denoising model includes: a specific noise removal module (initial denoising module) and an image restoration module (image content restoration module). The specific noise removal module can eliminate the obvious large granular high-level noise (which can be extremely large noise) in the input noise image, and is used to assist the model in removing extremely large noise in the noise image, and generate a feature map corresponding to the noise image after removing the extremely large noise; the image restoration module is based on the image quality score corresponding to the input noise image, and on the basis of the feature map of the noise image output by the specific noise removal module, performs adaptive denoising and restoration of the noise image. Among them, the image quality score (which can be understood as an accurate quality score aligned with the prior information) helps the target image denoising model determine the degree of denoising and restoration of the input noise image, and helps to achieve a natural and realistic denoising effect for noise images of various gains and various noise levels.

[0165] like Figure 3 As shown in the figure, during the training process, firstly, the target image is subjected to extremely large noise addition (i.e., only noise of specified particles is added) to generate data set 1 (second type sample pair) to train the specific noise removal module, i.e., train part of the module; after the training is completed, for the target image, based on the differential degradation scheme (i.e., including differential noise addition processing), data set 2 (first type sample pair) is generated. The LQ image (LowQuality, low-quality image data for training and evaluating the model, i.e., noisy image) in data set 2 contains rich degradation and is annotated with the corresponding image quality score. Based on data set 2, the entire target image denoising model including the specific noise removal module and the image restoration module is trained, i.e., the entire model is trained and supervised based on the constructed loss function. The data set degradation scheme can provide a training data set containing rich degradation, improve the performance of the target image denoising model, and the calculation of the image quality score in the degradation scheme enables the image quality score to act on the target image denoising model, so as to achieve a natural and real denoising effect for noisy images with various gains and noise levels.

[0166] Differential degradation scheme: In order to enable the target image denoising model to better process real noisy images and generate real denoised images; in the training data set (i.e., each sample pair) corresponding to the target image denoising model, the LQ image should be as close to the real noisy image data as possible, and the HQ image (High Quality, high-quality image data used for training and evaluating the model, i.e., the target image) should be as close to the desired effect as possible (i.e., as clean as possible and without noise). Only then can the trained target image denoising model perform ideal image denoising and restoration on the real noisy image.

[0167] like Figure 4As shown, in the process of making data set 2, the HQ image (target image) is subjected to operations such as cropping, blurring, denoising, and quality compression to obtain the corresponding low-quality noisy image (noise image), and the quality score of the noisy image is calculated at the same time (the corresponding quality score can be calculated for each process).

[0168] For HQ images, we first crop the image to a suitable size of 256×256 (i.e., the target size), blur the image, and then add noise to the entire image. We also perform differential noise processing based on light and dark distinction, edge flat area distinction, and specified area distinction. Finally, we perform image quality compression to obtain a low-quality noise image (LQ image) corresponding to the HQ image.

[0169] The whole process can be expressed as a degradation process using formula (8).

[0170] The specific operation process of the noise addition step in the degradation process is as follows:

[0171] 1. Obtain noisy image data: Place the 24-color card in a light box and adjust the lighting conditions to be stable. Place the 24-color card under stable light conditions to ensure that the pixel size of a single color block in the obtained color card image (initial image) is at least 100×100. Adjust the focal length of the device (such as a camera or other shooting equipment) to ensure that the obtained color card image is clearly focused.

[0172] During the capture process, in order to ensure the diversity of noise image data collection, the light source can be switched to capture noise images under different light source lighting conditions; at the same time, the parameters of each module of the ISP module are adjusted to capture noise images under different gains, different ISP parameters and modes, so as to obtain a rich variety of ISP style noise information covering the full gain level, that is, to obtain the initial image of various shooting parameters, environmental parameters, and image signal processing parameters.

[0173] For the captured color card noise image data with different gains, extract a single color block area in the image and extract the noise information on the R / G / B channels respectively (that is, determine the target noise information corresponding to the pixel point in each color channel). Specifically, the target noise information can be extracted according to formula (18). Based on the extracted noise information, a large-size spare noise image can be generated (for example, a large-size spare noise image can be generated by splicing). The obtained spare noise images are the real noise images of various ISP styles. Noise addition processing can be performed on the clean HQ image to generate the corresponding noise image.

[0174] When adding noise based on the HQ image, it is necessary to extract the noise information to be used from the target noise image. Specifically, the real noise information to be used can be extracted according to formula (19).

[0175] 2. Add noise to create low-quality noise images: Since each backup noise image is extracted and generated under different illumination, different gain, different ISP parameters, and different color blocks, the noise level and morphology of each backup noise image are different. The backup noise image can be grouped according to its noise level (noise intensity), and different degradation parameters (first noise addition coefficient) can be set for different groups of backup noise images to avoid the situation where the noise images generated by the same set of degradation parameters for the backup noise images of different noise levels are too different from each other and the distribution gap with the real noise image is too large, resulting in the inability to train the target image denoising model well.

[0176] In order to have a more accurate classification of each spare noise image, the variance of the noise information of the pixel points of each spare noise image data is calculated: ; Where N represents the noise information of the pixel points of the backup noise image, Represents the variance value of the backup noise image. The larger the value, the higher the noise level of the backup noise image.

[0177] Set the noise level threshold [T1, T2, T3, ...], compare According to the numerical relationship between the target evaluation value and each threshold value, the evaluation value range to which the target evaluation value belongs is determined, and according to the corresponding relationship between each evaluation value range and the value range of the noise addition coefficient, the value range corresponding to the evaluation value range to which the target evaluation value belongs is determined, and a value in the determined value range is selected as the first noise addition coefficient corresponding to the standby noise image; the noise level of the real noise image data is grouped for each standby noise image, and the value range of the first noise addition coefficient k corresponding to the standby noise images of different groups is as follows:

[0178] ;

[0179] After determining the first noise addition coefficient, firstly, the whole image noise information is added to the image using formula (9).

[0180] By observing and analyzing the real noise image, it can be found that the noise information in the edge area of ​​the noise image is richer, and the noise in the flat area is relatively weak. Therefore, in the process of degrading the low-quality noise image, in order to be more in line with the actual low-quality noise image, this application also distinguishes according to the edge and flat area of ​​the face, and adds noise to different degrees.

[0181] First, the edge detection algorithm shown in formula (10) is used to obtain the edge information of the target image, and then the noise level is added based on the judgment of the edge and non-edge areas of the image.

[0182] In addition, edge detection operators such as the Sobel operator and the Canny operator may be used for calculation. In actual use, the present application uses the Sobel operator as shown in formula (11) to obtain edge information of the HQ image, and performs noise addition in the first image processing according to formula (12).

[0183] The present application can also perform differential noise addition on images whose edge information pixel values ​​exceed a certain threshold according to formula (13)-formula (14).

[0184] Similarly, the noise in the bright and dark areas of the noisy image will also be different. The noise level in the bright area will be weaker, and the noise in the dark area will be more obvious (i.e., the noise addition principle). Therefore, during the degradation operation, the image will be distinguished according to its brightness and darkness, and different degrees of noise will be added according to formula (15). In order to make the image after noise addition more realistic and natural, the noise addition coefficient is related to the brightness and darkness of the image itself.

[0185] In order to enable the model to better learn the mapping relationship of the image information of the facial features (specified areas, taken as an example, not limited), find the mask images of the facial eyes, nose, mouth and other areas in the image, and perform additional differential noise addition on the facial features of the facial image based on the facial features mask images. For the clean facial HQ image data, use the facial detection algorithm shown in formula (16) to obtain the corresponding facial features mask image, and then perform differential noise addition on the facial features according to formula (17). The mask image is a binary image of the facial eyes, nose and mouth areas in the displayed image.

[0186] Through the above steps, based on the clean and noise-free high-resolution image HQ image, degradation is performed to obtain a low-resolution noisy image as the LQ image, and the image data pairs (sample pairs) of HQ and LQ are formed as the training data set of the neural model to train the target image denoising model. In the whole noisy image degradation process, at each degradation step, according to the degree of influence of the degradation step on the image quality and the size of the parameters used in the degradation step, the quality score of the low-quality image obtained by degradation is calculated according to formula (1), and the quality score is aligned with the preset score prior.

[0187] like Figure 5As shown, the neural model structure of the present application consists of two parts, one is SpecialNoiseReduceBlock, that is, a specific noise removal module (also called an extra-large noise removal module), and the other is an image restoration module ImageRestorationBlock based on image quality prior input.

[0188] Specific noise removal module: mainly realizes the removal of large noise in the input noise image. By inputting a low-quality noisy image into the model, a feature map with large noise particles removed from the input noise image can be obtained. The specific noise removal module consists of two convolution blocks (ConvBlock, conv) and two residual blocks (Residualblock, RB); the convolution kernel size in the convolution block and the residual block is 3×3. For a noise image of size H×W×C, an output feature map of the same size H×W×C can be obtained through this module. The pixels in the feature map correspond to each pixel in the input noise image one by one. The output of this module is used as the direct input of the subsequent image restoration module, which is beneficial to the entire target image denoising model for low-quality noise images with large noise, such as image data with low illumination and high gain at night, to achieve better noise reduction and restoration effects, and can remove large noise in the noise image.

[0189] Image restoration module: Its function is to adaptively output the corresponding image restoration effect according to the input noise image (feature map) and the quality score of the noise image, that is, the denoising effect corresponding to the noise image. The image restoration module splices the output feature map and image quality score of the specific noise removal module for input. The input size is H×W×(C+1), and the output size is H×W×C, that is, the size of the denoising result finally output by the target image denoising model is also H×W×C. The input feature map is first processed by a convolution module to expand the feature dimension, and then processed by three convolution downsampling modules (Conv_D) in the encoder part of the image restoration module to extract the H / 4×W / 4×256 feature map, which is sent to the RAB module (RestorationAttentionBlock, image restoration attention module). The RAB module can help the model better extract and represent features and guide the model to pay attention to some feature information in the image. Then the image information is restored through the three convolution upsampling modules (Conv_U) of the decoder part, and the last convolution module of the model (converting the 64-channel image data into 3 channels) is used to obtain the H×W×C denoising result for output.

[0190] The structure of the RAB module is as follows Figure 6As shown, the input of this module is the output feature of the last Conv_D convolution downsampling module in the image restoration module. Figure 1 FeatureMap1 (input features). Features Figure 1 After inputting into the RAB module, it is processed in two ways. The first way is processed by the convolution module, and then the result is added to the result of the feature processed by the attention mechanism module CBAM. The second way continues to be processed by the convolution module and the convolution downsampling module to obtain a feature map of H / 2×W / 2×2c, and the feature map is also added to the result of the attention mechanism module CBAM; the feature map obtained by the second addition is processed by the convolution module and the convolution upsampling module to restore the feature map to the size of H×W×C, and then it is spliced ​​with the feature map added by the first path in the channel dimension to obtain the feature. Figure 2 FeatureMap2 (intermediate feature). Figure 2 Process the convolution module to output the output features Figure 3 The size of FeatureMap3 (output feature) is H×W×2C. This module can combine attention to perform cross-scale feature extraction, increase the ability to extract and express key features, and help the model improve the image restoration effect.

[0191] Target image denoising model training method:

[0192] When training the model, split training is required. First, add only the large noise with obvious particles to the clean HQ image to construct the clean-noise image pair (the second type of sample pair) of the specific noise removal module, and supervise the model parameters of the specific noise removal module based on the L1 loss function.

[0193] After completing the training of the specific noise removal module, the parameter information of the pre-trained specific noise removal module is imported, and the clean-noise image dataset (the first type of sample pairs) obtained by the above differential degradation is used to train the structural parameters of the entire model, and a loss function is designed for supervised learning.

[0194] The total loss function of the entire model is composed of the full image loss function and the image facial feature mask area loss function, and is trained using the Adam (Adaptive Moment Estimation) optimizer. The total loss function is shown in formula (2).

[0195] It consists of three parts: L1 loss function (the calculation function of the second type of loss value), color loss function (the calculation function of the first type of loss value) and perceptual loss function (the calculation function of the third type of loss value). It consists of two parts: L1 loss function and perceptual loss function. That is, it can be calculated according to formula (3) and formula (4) respectively: and .

[0196] The color loss function is implemented by calculating the cosine similarity between the predicted image and the HQ image, which can be expressed by the following formula (5).

[0197] L1 loss is the sum of the absolute values ​​of the differences between the predicted values ​​and the true values, which can be expressed by formula (6).

[0198] Perceptual loss is to calculate the difference between two images through a pre-trained neural model. Usually, pre-trained convolutional neural models are used. These models have been trained on large-scale datasets and can extract high-level features of images. For example, the convolutional layer in the VGG-19 network (Visual Geometry Group, convolutional neural network architecture) can extract the texture and structure information of the image, and the fully connected layer can extract the semantic information of the image.

[0199] The perceptual loss is usually calculated by passing the denoising results of the target image and the noise image through the pre-trained neural network to obtain their feature representations in the network. These feature representations are then used as inputs to the loss function to calculate the Euclidean distance or Manhattan distance between them. As shown in formula (7), the perceptual loss can minimize the distance between the denoising result of the noise image and the target image in the feature space.

[0200] When using a deep learning model for image denoising, the present application ① designs a degradation strategy to produce low-quality degraded data (noise image corresponding to the target image) that is close to the actual noise image based on high-quality image data, and constructs a data set for neural network training to solve the problem of lack of training data under specific noise reduction tasks; ② builds a neural network, inputs the image quality score corresponding to the noise image while inputting the noise image, designs a specific noise removal module and an image restoration attention module to help the target image denoising model achieve full-gain adaptive denoising and achieve a real and natural restoration effect for various types of noise images; ③ constructs a loss function to guide the training of the model to achieve the denoising and restoration task of facial noise images captured in video acquisition.

[0201] Noise image degradation strategy: ① The extracted real noise information is grouped according to the noise level, and different degradation parameters are set to ensure that the degraded data contains real noise images of various types of noise levels and forms;

[0202] ② Differentiate and add noise to the edges and smooth areas in the image, and perform differential degradation;

[0203] ③ Differentiate and add noise according to the bright and dark areas of the image, and perform differential degradation to make the degraded data more consistent with the real data;

[0204] ④ Detect the facial features in the image to obtain a mask, and further process the degraded data and HQ image in the facial features mask area, so that the model can learn better facial image processing effects on this training data.

[0205] ⑤ During the degradation process, the image quality scores corresponding to the low-quality image data obtained in each degradation step are calculated, so that the model can learn the relationship between the image and its quality score, and achieve a natural and realistic denoising effect for images of different qualities.

[0206] Neural network construction: ① Design a specific noise removal module to better remove the large flat noise in the image, better remove the large particle noise in the image, and improve the image quality; ② Introduce the image restoration attention module in the image restoration module to extract and express features, which helps to improve the processing effect of the model; ③ Input the image quality score corresponding to the noisy image into the model to help the model establish a connection between the noisy image and the quality score, and achieve a natural and realistic noise reduction and restoration effect for noisy images with different levels;

[0207] Construction of loss function: The denoising result predicted by the model and the HQ image are used to calculate the full-image loss function and the face mask area loss function respectively, and the total loss function of the network is obtained by weighting. The full-image loss function is calculated by weighting the L1 loss, color loss, and perceptual loss; the face mask area loss function is calculated by weighting the L1 loss and perceptual loss function of the mask area of ​​the denoising result and the HQ image. Using this loss function to supervise the learning process of the model helps the model to achieve better image denoising and restoration effects, especially better performance in the facial area of ​​the image.

[0208] The present application provides a training method for an image denoising model, including data set preparation, network structure design, and loss function construction. The method can perform different degrees of denoising and restoration according to the quality level of the noise image itself, achieve effective image restoration in the full gain range, restore image details, and improve image quality.

[0209] The present application provides a target image denoising model based on image score prior information and noise image degradation, which can achieve a good restoration effect on the full gain segment of a real noise image (for an image acquisition scenario, the noise image is a noise image under the image acquisition scenario), and the restored image is natural and realistic.

[0210] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A training method for an image denoising model, characterized in that: The method comprises: Acquire multiple first-category sample pairs in a target training set; wherein each first-category sample pair includes a target image and a noise image corresponding to the target image, and the noise image in any first-category sample pair is generated for the target image in the first-category sample pair according to a predetermined noise image generation method; the predetermined noise image generation method includes: selecting a spare noise image from multiple spare noise images as a target noise image; determining noise information to be used on each color channel of each pixel in the target noise image as noise information to be used for the target noise image; performing a specified noise addition process on the target image based on the noise information to be used for the target noise image; determining the noise image corresponding to the target image based on the image after the specified noise addition process; each spare noise image is generated based on a target area in an initial image among the initial images; each initial image is an image obtained by photographing a target color card; the target color card includes multiple color blocks, and the target area is an image area of ​​a color block among the multiple color blocks; Inputting the noisy image and the corresponding image quality score in each first-category sample pair into the target image denoising model to be trained to obtain the denoising result corresponding to the noisy image in each sample pair; wherein the target image denoising model comprises an initial denoising module and an image content restoration module connected in series, the initial denoising module is used to process the input image to obtain a feature map of the same size as the input image; the image content restoration module is used to perform feature splicing processing on the feature map obtained by the initial denoising module and the image quality score input to the image content restoration module to obtain spliced ​​image data; the spliced ​​image data is processed to generate the denoising result of the image input to the initial denoising module; Determine a target loss value of the target image denoising model based on a denoising result corresponding to the noise image in each first-category sample pair and a target image in each first-category sample pair; In response to determining that the target image denoising model has not converged according to the target loss value, adjusting parameters of the target image denoising model, and returning to the step of obtaining multiple first category sample pairs in the target training set.

2. The method according to claim 1, characterized in that The target shooting parameters used when shooting different initial images have different parameter values, and the target shooting parameters include environment parameters and / or camera parameters; The designated noise addition processing includes a first type of noise addition processing and a second type of noise addition processing. The first type of noise addition processing is image processing for performing noise addition on all pixels of the image to be processed; the second type of noise addition processing includes at least one of a first image processing, a second image processing and a third image processing; the first image processing is image processing for performing noise addition on edge pixels of the image to be processed, the second image processing is image processing for performing noise addition on the image to be processed according to the brightness of the pixels, and the third image processing is image processing for performing noise addition on an image area of ​​a designated object of the image to be processed.

3. The method according to claim 2, characterized in that Based on the noise information to be utilized of the target noise image, a first type of noise addition processing method includes: Acquire a first noise addition coefficient corresponding to the target noise image; Based on the noise information to be utilized of the target noise image and according to the first noise adding coefficient corresponding to the target noise image, adding noise to all the pixel points of the image to be processed; The method for determining the first noise addition coefficient corresponding to any spare noise image includes: Based on the pixel value information of each pixel point of the backup noise image, determining an evaluation value for characterizing the noise intensity of the backup noise image as a target evaluation value; Based on a preset correspondence between each evaluation value range and a value range of the noise addition coefficient, determine a value range corresponding to the evaluation value range to which the target evaluation value belongs, and select a value in the determined value range as a first noise addition coefficient corresponding to the backup noise image; Among them, the evaluation value range and the value range in the corresponding relationship are negatively correlated.

4. The method according to claim 2, characterized in that: The method of performing first image processing based on the noise information to be utilized of the target noise image includes: Performing edge detection on the target image to obtain the target position of each pixel point at the edge; wherein the current image to be processed is an image to be subjected to noise addition processing based on edge information; According to the first method or the second method, based on the noise information to be used of the target noise image, the image to be processed is subjected to noise processing; wherein the first method is: for each pixel point at the target position of the image to be processed, based on the noise information to be used of each color channel of the pixel point located at the target position in the noise information to be used of the target noise image, and according to the second noise addition coefficient, the pixel point at the target position of the image to be processed is subjected to noise processing; the second method is: for each pixel point at the designated position of the image to be processed, based on the noise information to be used of each color channel of the pixel point located at the designated position in the noise information to be used of the target noise image, and according to the second noise addition coefficient, the pixel point at the designated position of the image to be processed is subjected to noise processing, wherein each designated position is a target position among the target positions, the pixel value representing the edge strength of which is greater than a predetermined threshold value; and / or, The method of performing second image processing based on the noise information to be utilized of the target noise image includes: For each pixel in the image to be processed, based on the noise information to be used of each color channel of the designated pixel in the noise information to be used of the target noise image, and according to the third noise addition coefficient and the noise addition principle, the pixel is noised, wherein the designated pixel is a pixel in the target noise image at the same position as the pixel, and the noise addition principle is that the noise addition degrees of pixels with different brightness in the target image are different; and / or, The method of performing the third image processing based on the noise information to be utilized of the target noise image includes: Performing region detection on the target image with respect to a designated object to obtain a mask image of the designated object in the target image; Based on the noise information to be utilized of the target noise image, and according to the fourth noise addition coefficient and the obtained mask image, the pixel points related to the designated object in the image to be processed are denoised.

5. The method according to claim 2, characterized in that: The predetermined noise image generation method also includes: In response to the completion of any image processing, a quality score of an image obtained after the image processing is calculated based on an image quality score of the image processed by the image processing, a predetermined processing parameter used by the image processing, a current image quality degradation parameter corresponding to the image processing, and a current weight parameter set for the predetermined processing parameter; wherein the image quality degradation parameter is used to characterize a degree of image degradation; as well as, After obtaining the quality score of the image obtained by the image processing, the current image quality degradation parameter and the current weight parameter set for the predetermined processing parameter are adjusted based on the quality score of the image obtained by the image processing and a preset priori score.

6. The method according to claim 1 or 2, characterized in that: Any alternate noise image can be generated by: From an initial image, determine an image area of ​​a color block to obtain a target area; For each pixel point in the target area, extract the target noise information on each color channel corresponding to the pixel point according to a predetermined extraction method; wherein the predetermined extraction method includes, for each color channel, determining the target noise information of the color channel corresponding to the pixel point based on the value of the color channel of the pixel point and the color mean value corresponding to the color channel; the color mean value corresponding to any color channel is: the value obtained by averaging the values ​​of the color channel of each pixel point in the target area; Based on the target noise information of each color channel corresponding to each pixel point of the target area, a backup noise image corresponding to the target area of ​​the initial image is generated; wherein the value of each color channel of each pixel point in the backup noise image is determined based on the target noise information of each color channel corresponding to a pixel point of the target area.

7. The method according to claim 6, characterized in that The step of determining, for each color channel, target noise information of the color channel corresponding to the pixel point based on the value of the color channel of the pixel point and the color mean value corresponding to the color channel, includes: For each color channel, subtract the color mean value corresponding to the color channel from the value of the pixel on the color channel to obtain an intermediate value, and sum the intermediate value with a predetermined offset value to obtain the target noise information of the color channel corresponding to the pixel; The step of determining the noise information to be utilized on each color channel of each pixel in the target noise image as the noise information to be utilized of the target noise image includes: Subtracting the predetermined offset value from the value of each color channel of each pixel in the target noise image to obtain the noise information to be used on each color channel of each pixel; The noise information to be utilized on each color channel of each pixel is determined as the noise information to be utilized of the target noise image.

8. The method according to any one of claims 1 to 5, characterized in that: The initial denoising module in the target image denoising model is a pre-trained module; The method of pre-training the initial denoising module includes: Acquire multiple second sample pairs; wherein each second type sample pair includes an auxiliary image and a noise image corresponding to the auxiliary image, wherein the noise image corresponding to the auxiliary image is an image after the auxiliary image is noised according to a target noise adding method, wherein the target noise adding method is a method for adding noise of specified particles; Inputting the noise image in each second sample pair into the initial denoising module to obtain a feature map corresponding to the noise image in each second type sample pair output by the initial denoising module; Determine a loss value of an initial denoising module based on a feature map corresponding to the noise image in each second-class sample pair and an auxiliary image in each second-class sample pair; In response to determining that the initial denoising module has not converged according to the loss value of the initial denoising module, adjusting parameters of the initial denoising module, and returning to the step of obtaining a plurality of second sample pairs.

9. The method according to any one of claims 1 to 5, characterized in that: The step of determining a target loss value of the target image denoising model based on a denoising result corresponding to the noise image in each first-category sample pair and a target image in each first-category sample pair includes: Based on the denoising result corresponding to the noise image in each first-category sample pair and the target image in each first-category sample pair, a first-category loss value, a second-category loss value, and a third-category loss value are calculated; Determining a target loss value of the target image denoising model based on the first type of loss value, the second type of loss value and the third loss value; The first type of loss value is a loss value obtained based on the difference in color dimension between the denoising results corresponding to the target image and the corresponding noise image in each first type sample pair; The second type of loss value is a loss value obtained based on the absolute value of the difference between the pixel values ​​of the denoising results corresponding to the target image and the corresponding noise image in each first type of sample pair; The third type of loss value is a loss value obtained based on the feature difference in a specified feature space between the denoising results corresponding to the target image and the corresponding noise image in each first type sample pair.

10. The method according to any one of claims 1 to 5, characterized in that: Before performing the designated noise adding process on the target image based on the noise information to be utilized of the target noise image, the predetermined noise image generating method further includes: Performing a specified preprocessing on the target image to obtain a preprocessed target image; wherein the specified preprocessing includes: image processing for cropping an image of a target size and / or image processing for blurring the image; The step of performing a specified noise adding process on the target image based on the noise information to be utilized of the target noise image comprises: Based on the noise information to be utilized of the target noise image, performing a specified noise adding process on the preprocessed target image; The step of determining the noise image corresponding to the target image based on the specified image after noise addition processing includes: The quality of the image after the specified noise processing is compressed to obtain a noise image corresponding to the target image.

Images