Image denoising model training method, image denoising method, device and electronic equipment

Abstract

The embodiment of the present application provides an image denoising model training method, an image denoising method, a device and an electronic device, which relate to the field of image processing technology. In the image denoising model training method, each noise image in the sample data is input into the image denoising model to be trained to obtain a predicted image; for each pixel position in the noise image, if the pixel value of the pixel position in the predicted image is within a first pixel value interval, the loss value at the pixel position is calculated based on the pixel value of the pixel position in the predicted image and the second pixel value interval; based on the loss value of the noise image calculated using the loss values ​​at each pixel position, the model parameters of the image denoising model to be trained are adjusted until the convergence conditions are reached and the training is completed. Based on this, the overfitting phenomenon of the image denoising model is avoided to a certain extent, and the image quality of the denoised image can be improved by processing the image based on the trained image denoising model.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image denoising model training method, an image denoising method, a device and an electronic device. Background Art

[0002] With the rapid development of computer technology, people are using image information more and more widely, and the requirements for image quality are gradually increasing. At present, the sensor in the image acquisition device can obtain an image based on the photons of incident light. However, the sensor is easily affected by various factors such as the intensity of ambient light during the image acquisition process, so that the obtained image contains more noise. Therefore, it is usually necessary to denoise the image to obtain a high-quality image.

[0003] In the related art, image denoising is performed through an image denoising model. During the training process of the image denoising model, the loss function value is calculated based on the difference between the denoised image output by the image denoising model and the noise-free label data (also called true value data, i.e., the noise-free image), and the parameters of the image denoising model are updated based on the loss function value.

[0004] Based on this method, the noisy image needs to be fitted to a completely noise-free image. The fitting is difficult and will cause the image denoising model to overfit. As a result, the denoised image will be blurred and the details of the image will be lost, resulting in low image quality of the denoised image. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an image denoising model training method, an image denoising method, a device and an electronic device, which can avoid the overfitting phenomenon of the image denoising model to a certain extent, and then, by processing the image based on the obtained image denoising model, the image quality of the denoised image can be improved. The specific technical solution is as follows:

[0006] In a first aspect of the implementation of the present application, a method for training an image denoising model is first provided, the method comprising:

[0007] Acquire sample data; wherein the sample data includes: a plurality of noise images containing noise, first label images containing no noise corresponding to each noise image, and second label images corresponding to each noise image; wherein each second label image is: a weighted sum of the corresponding noise image and the first label image;

[0008] Input each noisy image into the image denoising model to be trained, and obtain an image output by the image denoising model as a predicted image;

[0009] For each pixel position in the noise image, if the pixel value of the pixel position in the predicted image is within the first pixel value interval, then based on the pixel value of the pixel position in the predicted image and the second pixel value interval, calculate the loss value at the pixel position;

[0010] The endpoints of the first pixel value interval include: the pixel value of the pixel position in the corresponding second label image and the pixel value of the pixel position in the noise image; one endpoint of the second pixel value interval is the pixel value of the pixel position in the corresponding second label image, and the second pixel value interval belongs to a third pixel value interval; the endpoints of the third pixel value interval include: the pixel value of the pixel position in the corresponding first label image and the pixel value of the pixel position in the corresponding second label image;

[0011] Based on the loss value at each pixel position, the loss value of the noise image is calculated;

[0012] Based on the loss value of the noisy image, the model parameters of the image denoising model to be trained are adjusted until the convergence condition is reached, thereby obtaining a trained image denoising model.

[0013] Optionally, if the pixel value of the pixel position in the predicted image is within a first pixel value interval, calculating the loss value at the pixel position based on the pixel value of the pixel position in the predicted image and a second pixel value interval includes:

[0014] If the pixel value of the pixel position in the predicted image is within the first pixel value interval, the loss value at the pixel position is calculated based on the difference between the pixel value of the pixel position in the predicted image and the pixel value of the pixel position in the corresponding second label image.

[0015] Optionally, the method further includes:

[0016] If the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval, the loss value at the pixel position is calculated based on the pixel value of the pixel position in the predicted image and the fifth pixel value interval; wherein the endpoints of the fourth pixel value interval include: the pixel value of the pixel position in the corresponding first label image and the pixel value of the pixel position in the noise image; one endpoint of the fifth pixel value interval is the pixel value of the pixel position in the corresponding first label image, and the fifth pixel value interval belongs to the third pixel value interval.

[0017] Optionally, if the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval, calculating the loss value at the pixel position based on the pixel value of the pixel position in the predicted image and the fifth pixel value interval, includes:

[0018] If the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval, the loss value at the pixel position is calculated based on the difference between the pixel value of the pixel position in the predicted image and the pixel value of the pixel position in the corresponding first label image.

[0019] Optionally, the method further includes:

[0020] If the pixel value of the pixel position in the predicted image is within the third pixel value interval, the loss value at the pixel position is determined to be 0.

[0021] Optionally, each image in the sample data is an image in RAW format.

[0022] In a second aspect of the present application, a method for image denoising is provided, the method comprising:

[0023] Obtain an image to be denoised;

[0024] The image to be denoised is input into a pre-trained image denoising model to obtain a denoised image; wherein the image denoising model is trained based on any of the above-mentioned method steps.

[0025] In a third aspect of the present application, there is provided an image denoising model training device, the device comprising:

[0026] A sample data acquisition module, used to acquire sample data; wherein the sample data includes: a plurality of noise images containing noise, first label images containing no noise corresponding to each noise image, and second label images corresponding to each noise image; wherein each second label image is: a weighted sum of the corresponding noise image and the first label image;

[0027] A prediction image acquisition module, used for inputting each noise image into the image denoising model to be trained, and obtaining an image output by the image denoising model as a prediction image;

[0028] A first calculation module is used for calculating, for each pixel position in the noise image, a loss value at the pixel position based on the pixel value of the pixel position in the predicted image and a second pixel value interval if the pixel value of the pixel position in the predicted image is within a first pixel value interval;

[0029] The endpoints of the first pixel value interval include: the pixel value of the pixel position in the corresponding second label image and the pixel value of the pixel position in the noise image; one endpoint of the second pixel value interval is the pixel value of the pixel position in the corresponding second label image, and the second pixel value interval belongs to a third pixel value interval; the endpoints of the third pixel value interval include: the pixel value of the pixel position in the corresponding first label image and the pixel value of the pixel position in the corresponding second label image;

[0030] A second calculation module, used for calculating the loss value of the noise image based on the loss value at each pixel position;

[0031] The model parameter adjustment module is used to adjust the model parameters of the image denoising model to be trained based on the loss value of the noisy image until the convergence condition is reached to obtain a trained image denoising model.

[0032] Optionally, the first calculation module is specifically used to calculate the loss value at the pixel position based on the difference between the pixel value of the pixel position in the predicted image and the pixel value of the pixel position in the corresponding second label image if the pixel value of the pixel position in the predicted image is within a first pixel value interval.

[0033] Optionally, the device further comprises:

[0034] A third calculation module is used to calculate the loss value at the pixel position based on the pixel value of the pixel position in the predicted image and the fifth pixel value interval if the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval; wherein the endpoints of the fourth pixel value interval include: the pixel value of the pixel position in the corresponding first label image and the pixel value of the pixel position in the noise image; one endpoint of the fifth pixel value interval is the pixel value of the pixel position in the corresponding first label image, and the fifth pixel value interval belongs to the third pixel value interval.

[0035] Optionally, the third calculation module is specifically used to calculate the loss value at the pixel position based on the difference between the pixel value of the pixel position in the predicted image and the pixel value of the pixel position in the corresponding first label image if the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval.

[0036] Optionally, the device further comprises:

[0037] The fourth calculation module is used to determine the loss value at the pixel position as 0 if the pixel value of the pixel position in the predicted image is within the third pixel value interval.

[0038] Optionally, each image in the sample data is an image in RAW format.

[0039] In a fourth aspect of the implementation of the present application, there is also provided an image denoising device, the device comprising:

[0040] A module for acquiring an image to be denoised, used for acquiring an image to be denoised;

[0041] A denoising module is used to input the image to be denoised into a pre-trained image denoising model to obtain a denoised image; wherein the image denoising model is trained based on any of the above-mentioned image denoising model training method steps.

[0042] In another aspect of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0043] Memory, used to store computer programs;

[0044] The processor is used to implement any of the above-mentioned image denoising model training methods or image denoising method steps when executing the program stored in the memory.

[0045] In another aspect of the implementation of the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned image denoising model training methods or image denoising method steps is implemented.

[0046] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-described image denoising model training methods or image denoising methods.

[0047] An image denoising model training method provided in an embodiment of the present application can obtain sample data; wherein the sample data includes: a plurality of noise images containing noise, a first label image containing no noise corresponding to each noise image, and a second label image corresponding to each noise image; wherein each second label image is: a weighted sum of the corresponding noise image and the second label image; each noise image is input into the image denoising model to be trained, and an image output by the image denoising model is obtained as a predicted image; for each pixel position in the noise image, if the pixel value of the pixel position in the predicted image is within a first pixel value interval, then based on the pixel value of the pixel position in the predicted image and the second pixel value interval, the pixel position is calculated. loss value; wherein, the endpoints of the first pixel value interval include: the pixel value of the pixel position in the corresponding second label image and the pixel value of the pixel position in the noise image; one endpoint of the second pixel value interval is the pixel value of the pixel position in the corresponding second label image, and the second pixel value interval belongs to the third pixel value interval; the endpoints of the third pixel value interval include: the pixel value of the pixel position in the corresponding first label image and the pixel value of the pixel position in the corresponding second label image; based on the loss value at each pixel position, the loss value of the noise image is calculated; based on the loss value of the noise image, the model parameters of the image denoising model to be trained are adjusted until the convergence condition is reached to obtain the trained image denoising model.

[0048] Based on the above processing, if the pixel value of the pixel position in the predicted image is within the first pixel value interval, that is, between the pixel value of the pixel position in the corresponding second label image and the pixel value of the pixel position in the noise image, then the loss value at the pixel position can be calculated based on the pixel value of the pixel position in the predicted image and the second pixel value interval. Accordingly, adjusting the model parameters of the image denoising model based on the loss value can make the pixel value of the pixel position output by the image denoising model tend to the second pixel value interval. Since the second pixel value interval belongs to the third pixel value interval with the pixel value of the pixel position in the corresponding first label image (i.e., the image without noise) and the pixel value in the corresponding second label image as endpoints, the trained image denoising model can effectively denoise the image.

[0049] In addition, the related art uses the pixel value of the pixel position in the corresponding first label image to calculate the loss value, so as to force the noisy image to be fitted to the image without noise. In the solution of the present application, the loss value calculated based on the second pixel value interval will cause the output pixel value to be within the third pixel value interval, thus avoiding the forced fitting of the noisy image to the image without noise, and further, avoiding the overfitting phenomenon of the image denoising model to a certain extent. Further, by processing the image based on the obtained image denoising model, the image quality of the denoised image can be improved.

[0050] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0052] Figure 1 A first flow chart of the image denoising model training method provided in an embodiment of the present application;

[0053] Figure 2A A schematic diagram of a noise image provided in an embodiment of the present application;

[0054] Figure 2B The image denoising model obtained by using the method in the related art is Figure 2A A schematic diagram of an image obtained by denoising the noisy image shown;

[0055] Figure 2C The image denoising model obtained by the method in the embodiment of the present application is Figure 2A A schematic diagram of an image obtained by denoising the noisy image shown;

[0056] Figure 2D A method provided in the embodiment of the present application Figure 2A Schematic diagram of a noise-free image corresponding to the noise image shown;

[0057] Figure 3A A schematic diagram of another noise image provided in an embodiment of the present application;

[0058] Figure 3B The image denoising model obtained by using the method in the related art is Figure 3A A schematic diagram of an image obtained by denoising the noisy image shown;

[0059] Figure 3C The image denoising model obtained by the method in the embodiment of the present application is Figure 3A A schematic diagram of an image obtained by denoising the noisy image shown;

[0060] Figure 4 A structural schematic diagram of an image denoising model provided in an embodiment of the present application;

[0061] Figure 5A A first schematic diagram of an interval of pixel values ​​in a predicted image provided by an embodiment of the present application;

[0062] Figure 5B A second schematic diagram of the interval of pixel values ​​in the predicted image provided in an embodiment of the present application;

[0063] Figure 5C A third schematic diagram of the interval of pixel values ​​in the predicted image provided by the embodiment of the present application;

[0064] Figure 6 A second flow chart of the image denoising model training method provided in an embodiment of the present application;

[0065] Figure 7 A third flow chart of the image denoising model training method provided in an embodiment of the present application;

[0066] Figure 8 A flow chart of an image denoising method provided in an embodiment of the present application;

[0067] Fig. 9 A structural schematic diagram of an image denoising model training device provided in an embodiment of the present application;

[0068] Fig.10 A schematic diagram of the structure of an image denoising device provided in an embodiment of the present application;

[0069] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0071] With the rapid development of computer technology, people are using image information more and more widely, and the requirements for image quality are gradually increasing. At present, the sensor in the image acquisition device can obtain an image based on the photons of incident light. However, the sensor is easily affected by various factors such as the intensity of ambient light during the image acquisition process, so that the obtained image contains more noise. Therefore, it is usually necessary to denoise the image to obtain a high-quality image.

[0072] For example, the image acquisition device may be a mobile phone or a surveillance camera and other devices. When the intensity of the ambient light is weak (such as shooting at night), this type of device can denoise the captured image based on the image denoising model obtained by the method provided in the embodiment of the present application, and thus obtain a denoised image.

[0073] In addition, the image denoising model can be pre-trained in this type of device, or the image denoising model can be trained in other devices. Accordingly, this type of device can obtain the trained image denoising model from other devices and denoise the image.

[0074] In the related art, image denoising is performed through an image denoising model. During the training process of the image denoising model, the loss function value is calculated based on the difference between the denoised image output by the image denoising model and the noise-free label data (also called true value data, i.e., the noise-free image), and the parameters of the image denoising model are updated based on the loss function value.

[0075] Based on this method, the noisy image needs to be fitted to a completely noise-free image. The fitting is difficult and will cause the image denoising model to overfit. As a result, the denoised image will be blurred and the details of the image will be lost, resulting in low image quality of the denoised image.

[0076] In order to avoid the phenomenon of overfitting of the image denoising model to a certain extent, and then process the image based on the obtained image denoising model to improve the image quality of the denoised image, an embodiment of the present application provides an image denoising model training method. The method is applied to an electronic device, and the electronic device can train the image denoising model. In addition, the electronic device can also denoise the image based on the trained image denoising model. For example, the electronic device can be a mobile phone or a surveillance camera.

[0077] See also Figure 1 , Figure 1 The first flow chart of the image denoising model training method provided in the embodiment of the present application, the method may include the following steps:

[0078] Step S101: Obtain sample data.

[0079] The sample data includes: multiple noise images containing noise, first label images not containing noise corresponding to each noise image, and second label images corresponding to each noise image; wherein each second label image is: the weighted sum of the corresponding noise image and the second label image.

[0080] Step S102: input each noisy image into the image denoising model to be trained, and obtain an image output by the image denoising model as a predicted image.

[0081] Step S103: For each pixel position in the noise image, if the pixel value of the pixel position in the predicted image is within the first pixel value interval, then based on the pixel value of the pixel position in the predicted image and the second pixel value interval, calculate the loss value at the pixel position.

[0082] Among them, the endpoints of the first pixel value interval include: the pixel value of the pixel position in the corresponding second label image and the pixel value of the pixel position in the noise image; one endpoint of the second pixel value interval is the pixel value of the pixel position in the corresponding second label image, and the second pixel value interval belongs to the third pixel value interval; the endpoints of the third pixel value interval include: the pixel value of the pixel position in the corresponding first label image and the pixel value of the pixel position in the corresponding second label image.

[0083] Step S104: Based on the loss value at each pixel position, the loss value of the noise image is calculated.

[0084] Step S105: Based on the loss value of the noisy image, the model parameters of the image denoising model to be trained are adjusted until a convergence condition is reached, thereby obtaining a trained image denoising model.

[0085] Based on the above processing, if the pixel value of the pixel position in the predicted image is within the first pixel value interval, that is, between the pixel value of the pixel position in the corresponding second label image and the pixel value of the pixel position in the noise image, then the loss value at the pixel position can be calculated based on the pixel value of the pixel position in the predicted image and the second pixel value interval. Accordingly, adjusting the model parameters of the image denoising model based on the loss value can make the pixel value of the pixel position output by the image denoising model tend to the second pixel value interval. Since the second pixel value interval belongs to the third pixel value interval with the pixel value of the pixel position in the corresponding first label image (i.e., the image without noise) and the pixel value in the corresponding second label image as endpoints, the trained image denoising model can effectively denoise the image.

[0086] In addition, the related art uses the pixel value of the pixel position in the corresponding first label image to calculate the loss value, so as to force the noisy image to be fitted to the image without noise. In the solution of the present application, the loss value calculated based on the second pixel value interval will cause the output pixel value to be within the third pixel value interval, thus avoiding the forced fitting of the noisy image to the image without noise, and further, avoiding the overfitting phenomenon of the image denoising model to a certain extent. Further, by processing the image based on the obtained image denoising model, the image quality of the denoised image can be improved.

[0087] In one embodiment, Figure 2A As shown, Figure 2A A schematic diagram of a noise image provided in an embodiment of the present application, for example, the noise image may be a visualization result of a RAW image captured by an image acquisition device under low ambient light conditions. Figure 2A The image contains more noise.

[0088] like Figure 2B As shown, Figure 2B The image denoising model obtained by using the method in the related art is Figure 2A A schematic diagram of an image obtained by denoising the noise image shown in FIG. Figure 2A The noisy image shown, Figure 2B The image in contains less noise, however, the image is over-smoothed and more image details are lost.

[0089] like Figure 2C As shown, Figure 2C The image denoising model obtained by the method in the embodiment of the present application is Figure 2A A schematic diagram of an image obtained by denoising the noise image shown in FIG. Figure 2A The noisy image shown, Figure 2C The image in contains less noise and is Figure 2B , Figure 2C The image in retains more image details.

[0090] In addition, if Figure 2D As shown, Figure 2D A method provided in the embodiment of the present application Figure 2A The schematic diagram of the noise-free image corresponding to the noise image shown is, for example, the image may be a long exposure image captured by an image capture device under the same scene.

[0091] In one embodiment, Figure 3A As shown, Figure 3A A schematic diagram of another noise image provided in an embodiment of the present application. Figure 3A The image contains more noise.

[0092] like Figure 3B As shown, Figure 3B The image denoising model obtained by using the method in the related art is Figure 3A A schematic diagram of an image obtained by denoising the noise image shown in FIG. Figure 3A The denoised image shown, Figure 3B The image in contains less noise, however, the image is over-smoothed and more image details are lost.

[0093] like Figure 3C As shown, Figure 3C The image denoising model obtained by the method in the embodiment of the present application is Figure 3A A schematic diagram of an image obtained by denoising the noise image shown in FIG. Figure 3A The noisy image shown, Figure 3C The image in contains less noise, that is, the image denoising model obtained by the method in the embodiment of the present application has an obvious denoising effect on the noisy image. And compared with Figure 3B , Figure 3C The image in retains image details such as texture edges.

[0094] With respect to step S101 , sample data may be obtained before training the image denoising model.

[0095] For example, the noise image in the sample data may be an image captured by an image acquisition device under low ambient light conditions, and accordingly, the first label image corresponding to the noise image may be a long-exposure image captured by the image acquisition device under the same environment. Alternatively, the first label image in the sample data may be an image captured by an image acquisition device under high ambient light conditions, and accordingly, noise may be added to the first label image to obtain a corresponding noise image.

[0096] Each second label image is a weighted sum of the corresponding noise image and the first label image, wherein the sum of the weight of the noise image and the weight of the first label image is 1. That is, for each pixel position in each second label image, the pixel value of the pixel position in the second label image is between the pixel value of the pixel position in the corresponding noise image and the pixel value of the pixel position in the corresponding first label image. A noise image containing noise, a first label image not containing noise corresponding to the noise image, and a second label image corresponding to the noise image can be called a set of sample images. That is, the sample data can include multiple sets of sample images. The first label image can also be called "hard" label data, and the second label image can also be called "softened" label data.

[0097] In one embodiment, the pixel value of each pixel position in the second label image may be calculated based on a preset formula. For example, the preset formula may be formula (1):

[0098] gt ′ =input*alpha+gt*(1-alpha),alpha∈[0,1] (1)

[0099] Among them, input is the pixel value of each pixel position in the noise image, gt is the pixel value of the pixel position in the first label image corresponding to the noise image, gt ′ is the pixel value of the pixel position in the second label image corresponding to the noise image, and alpha is a preset coefficient.

[0100] In formula (1), alpha can control the strength of noise reduction. The closer alpha is to 0, the stronger the noise reduction strength is. For example, when alpha = 0, gt ′ Same as gt, that is, the first label image and the second label image are the same. At this time, if the image denoising model is trained based on the loss value obtained from the second label image, that is, the image denoising model is trained based on the loss value obtained from the first label image, it can be understood that the denoising strength of the image denoising model obtained by training according to the method in the relevant technology is the largest (that is, strong denoising); when alpha = 1, gt ′ The same as the input, that is, the second label image and the noise image are the same. At this time, if the image denoising model is trained based on the loss value obtained from the second label image, the denoising strength of the obtained image denoising model is the smallest (that is, no denoising). The preset coefficient can be set based on experience and is not specifically limited. For example, the preset coefficient can be 0.2 or 0.3. That is, in an embodiment of the present application, when calculating the second label image, the weights of the first label image and the noise image are not 0.

[0101] With respect to step S102, when training the image denoising model, a noise image may be input into the image denoising model to be trained to obtain a loss value of the noise image, and then the model parameters of the image denoising model may be adjusted based on the loss value of the noise image. When the adjustment is completed, the next noise image may be input into the image denoising model to be trained to continue adjusting the model parameters of the image denoising model.

[0102] Alternatively, the noise images may be grouped, and then a group of noise images may be input into the image denoising model to be trained to obtain the loss value of the group of noise images, and then the model parameters of the image denoising model may be adjusted based on the loss value of the group of noise images. When the adjustment is completed, the next group of noise images may be input into the image denoising model to be trained, and the model parameters of the image denoising model may be further adjusted.

[0103] For each noisy image, after the noisy image is input into the image denoising model to be trained, a predicted image corresponding to the noisy image can be obtained.

[0104] The image denoising model to be trained may be a DnCNN (Denoising Convolutional Neural Network) or a UNet (U-type network structure).

[0105] For example, the structure of the image denoising model to be trained can be found in Figure 4 , Figure 4 A schematic diagram of the structure of an image denoising model provided in an embodiment of the present application. Figure 4 The image denoising model in Figure 1 contains multiple conv1 (Convolution1, convolution layer 1), conv2 (Convolution2, convolution layer 2), relu (Rectified Linear Unit, activation function layer) and deconv (Transposed Convolution, transposed convolution layer). The size of the convolution kernel of conv1 is 3×3, and the stride (convolution step) is 1. The size of the convolution kernel of conv2 is 2×2, and the stride is 2. The size of the convolution kernel of deconv is 2×2, and the stride is 2. The plus sign indicates superposition processing. Inputting the noisy image into the image denoising model can obtain the predicted image.

[0106] For step S103, for each pixel position in each noise image, the pixel value of the pixel position in the corresponding second label image (which can be expressed as gt ′ denoted by ), is between the pixel value of the pixel position in the noise image (which can be represented by input) and the pixel value of the pixel position in the corresponding first label image (which can be represented by gt). Figure 5A As shown, Figure 5A This is a first schematic diagram of the interval of pixel values ​​in the predicted image provided by the embodiment of the present application. If the pixel value of the pixel position in the predicted image (which can be represented by pred) is ′and input, the loss value at the pixel position can be calculated based on pred and the second pixel value interval. The second pixel value interval belongs to the third pixel value interval (i.e., based on gt ′ and gt are the endpoints), and gt ′ is the endpoint of the second pixel value interval, that is, the length of the second pixel value interval is less than the third pixel value interval.

[0107] For example, a pixel value may be selected from the second pixel value interval, and the loss value at the pixel position may be calculated based on the selected pixel value and pred. Accordingly, the model parameters of the image denoising model may be adjusted based on the loss value, so that the pixel value at the pixel position output by the image denoising model may tend to the second pixel value interval.

[0108] In one embodiment, the noise image may be a RAW image. In this case, each pixel position in the noise image corresponds to four channels RGGB (Red, Green, Green, Blue). Accordingly, the predicted images corresponding to the four channels can also be obtained. Furthermore, for each channel, if the pixel value of the pixel position in the corresponding predicted image is within the first pixel value interval, the loss value of the pixel position is calculated based on the pixel value of the pixel position in the corresponding predicted image and the second pixel value interval, and the loss value of the channel at the pixel position can be obtained. Furthermore, by combining the loss values ​​of the channels at the pixel position, the loss value at the pixel position can be obtained.

[0109] In one embodiment, if the pixel value of the pixel position in the predicted image is within the first pixel value interval, then based on the pixel value of the pixel position in the predicted image and the second pixel value interval, calculating the loss value at the pixel position (S103) includes:

[0110] Step S1031: If the pixel value of the pixel position in the predicted image is within the first pixel value interval, the loss value at the pixel position is calculated based on the difference between the pixel value of the pixel position in the predicted image and the pixel value of the pixel position in the corresponding second label image.

[0111] For example, see Figure 5A , if pred is between input and gt ′ Between pred and gt, it can be directly based on ′ The difference between is used to calculate the loss value at the pixel position. For example, the loss value at the pixel position can be calculated based on the L1 loss function, or the loss value at the pixel position can be calculated based on the L2 loss function.

[0112] For example, the loss value at the pixel position can be calculated based on formula (2):

[0113] loss_b=|pred-gt ′ | (2)

[0114] Among them, loss_b is the loss value at the pixel position, pred is the pixel value of the pixel position in the predicted image, and gt ′ is the pixel value of the pixel position in the corresponding second label image.

[0115] Based on the above processing, based on gt ′ The calculated loss value will cause the output pixel value to tend to gt ′ , which can avoid forcibly fitting the noisy image to an image without noise, and further, can avoid the overfitting phenomenon of the image denoising model to a certain extent. Then, by processing the image based on the obtained image denoising model, it is possible to retain more image details while removing noise, and thus improve the image quality of the denoised image.

[0116] In addition, if the pixel value of the pixel position in the predicted image is within the first pixel value interval, the loss value can be calculated based on the difference between the pixel value of the pixel position in the predicted image and the pixel value in the corresponding first label image, and the model parameters of the image denoising model can be adjusted. However, this method fits the pixel value of the pixel position in the predicted image to the pixel value of the pixel position in the corresponding first label image, which requires a relatively large adjustment of the model parameters of the image denoising model, and will also increase the requirements for the structural complexity of the image denoising model.

[0117] Therefore, in an embodiment of the present application, if the pixel value of the pixel position in the predicted image is within the first pixel value interval, that is, the pixel value of the pixel position in the predicted image is close to the pixel value in the corresponding second label image, then based on the difference between the pixel value of the pixel position in the predicted image and the pixel value in the corresponding second label image, the loss value at the pixel position is calculated, which can avoid the above situation and avoid large adjustments to the model parameters of the image denoising model, thereby reducing the requirements for the structural complexity of the image denoising model.

[0118] For step S104, the loss value of the noise image can be calculated based on the weighted sum of the loss values ​​at each pixel position. For example, the weight of each pixel position can be 1, in which case the loss value of the noise image is the sum of the loss values ​​at each pixel position. The weight of each pixel position can be set based on experience and is not specifically limited.

[0119] For step S105, based on the loss value of the noise image, the model parameters of the image denoising model to be trained are adjusted until the convergence condition is reached, that is, the pixel value of each pixel position in the predicted image output by the image denoising model is located in the third pixel value interval, so as to avoid forcibly fitting the noise image to an image without noise, and further, to a certain extent, the overfitting phenomenon of the image denoising model can be avoided. Furthermore, by processing the image based on the obtained image denoising model, more image details can be retained while removing the noise, and the image quality of the denoised image can be improved.

[0120] For example, the gradient descent method can be used to adjust the model parameters of the image denoising model to be trained.

[0121] In one embodiment, see Figure 6 , Figure 6 The second flow chart of the image denoising model training method provided in the embodiment of the present application, the image denoising model training method further includes:

[0122] Step S106: If the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval, a loss value at the pixel position is calculated based on the pixel value of the pixel position in the predicted image and the fifth pixel value interval.

[0123] Among them, the endpoints of the fourth pixel value interval include: the pixel value of the pixel position in the corresponding first label image and the pixel value of the pixel position in the noise image; one endpoint of the fifth pixel value interval is the pixel value of the pixel position in the corresponding first label image, and the fifth pixel value interval belongs to the third pixel value interval.

[0124] In the embodiments of the present application, Figure 5B As shown, Figure 5B This is a second schematic diagram of the interval of pixel values ​​in the predicted image provided by the embodiment of the present application. For each pixel position in each noise image, the pixel value of the pixel position in the corresponding second label image (can be expressed by gt ′ , which is between the pixel value of the pixel position in the noise image (which can be represented by input) and the pixel value of the pixel position in the corresponding first label image (which can be represented by gt). If the pixel value of the pixel position in the predicted image (which can be represented by pred) is outside the interval with input and gt as endpoints, the loss value at the pixel position can be calculated based on pred and the fifth pixel value interval. The fifth pixel value interval belongs to the third pixel value interval, and gt is the endpoint of the fifth pixel value interval, that is, the length of the fifth pixel value interval is less than the third pixel value interval.

[0125] For example, a pixel value may be selected from the fifth pixel value interval, and the loss value at the pixel position may be calculated based on the selected pixel value and pred. Accordingly, the model parameters of the image denoising model may be adjusted based on the loss value, so that the pixel value at the pixel position output by the image denoising model may tend to the fifth pixel value interval.

[0126] The loss value calculated based on the fifth pixel value interval will cause the output pixel value to be within the third pixel value interval, thereby avoiding forcibly fitting the noisy image to an image without noise, and further, avoiding the overfitting phenomenon of the image denoising model to a certain extent. Furthermore, by processing the image based on the obtained image denoising model, more image details can be retained while removing noise, thereby improving the image quality of the denoised image.

[0127] In one embodiment, if the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval, the loss value at the pixel position is calculated based on the pixel value of the pixel position in the predicted image and the fifth pixel value interval (S106), including:

[0128] Step S1061: If the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval, the loss value at the pixel position is calculated based on the difference between the pixel value of the pixel position in the predicted image and the pixel value of the pixel position in the corresponding first label image.

[0129] In the embodiments of the present application, Figure 5B As shown in FIG. 1 , if pred is outside the interval with input and gt as endpoints, the loss value at the pixel position can be directly calculated based on the difference between pred and gt. For example, the loss value at the pixel position can be calculated based on the L1 loss function, or the loss value at the pixel position can be calculated based on the L2 loss function.

[0130] For example, the loss value at the pixel position can be calculated based on formula (3):

[0131] loss_c=|pred-gt| (3)

[0132] Among them, loss_c is the loss value at the pixel position, pred is the pixel value of the pixel position in the predicted image, and gt is the pixel value of the pixel position in the corresponding first label image.

[0133] Based on the above processing, the loss value calculated based on gt will cause the output pixel value to tend towards gt, and then, cause the output pixel value to tend towards the third pixel value interval, which can avoid forcibly fitting the noisy image to an image without noise, and then, to a certain extent, it can avoid the overfitting phenomenon of the image denoising model. Then, by processing the image based on the obtained image denoising model, it is possible to remove noise while retaining more image details, and thus improve the image quality of the denoised image.

[0134] In addition, if the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval, although based on the difference between the pixel value of the pixel position in the predicted image and the pixel value in the corresponding second label image, the loss value can also be calculated and the model parameters of the image denoising model can be adjusted. However, this method fits the pixel value of the pixel position in the predicted image to the pixel value of the pixel position in the corresponding second label image, which requires a relatively large adjustment of the model parameters of the image denoising model, and will also increase the requirements for the structural complexity of the image denoising model.

[0135] Therefore, in the embodiment of the present application, if the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval, that is, the pixel value of the pixel position in the predicted image is close to the pixel value in the corresponding first label image, then based on the difference between the pixel value of the pixel position in the predicted image and the pixel value in the corresponding first label image, the loss value at the pixel position is calculated, which can avoid the above situation and avoid large adjustments to the model parameters of the image denoising model, thereby reducing the requirements for the structural complexity of the image denoising model.

[0136] In one embodiment, see Figure 7 , Figure 7 This is a third flow chart of the image denoising model training method provided in an embodiment of the present application. The image denoising model training method further includes:

[0137] Step S107: If the pixel value of the pixel position in the predicted image is within the third pixel value interval, the loss value at the pixel position is determined to be 0.

[0138] In the embodiments of the present application, Figure 5C As shown, Figure 5C This is a third schematic diagram of the interval of pixel values ​​in the predicted image provided by the embodiment of the present application. For each pixel position in each noise image, the pixel value of the pixel position in the corresponding second label image (which can be expressed by gt ′denoted by ), the pixel value of the pixel position in the noise image (which can be represented by input) and the pixel value of the pixel position in the corresponding first label image (which can be represented by gt). If the pixel value of the pixel position in the predicted image (which can be represented by pred) is between gt ′ and gt, indicating that the output of the pixel position in the noisy image has met the conditions, then the loss value at the pixel position is determined to be 0, that is, for the currently input noisy image, there is no need to adjust the model parameters of the image denoising model based on the loss value at the pixel position.

[0139] For example, the loss value at the pixel position can be calculated based on formula (4):

[0140] loss_a=0 (4)

[0141] Among them, loss_a is the loss value at the pixel position.

[0142] Based on the above processing, the loss value of the pixel position that meets the conditions can be determined to be 0, which makes the output pixel value tend to the third pixel value interval, thus avoiding forcibly fitting the noisy image to an image without noise, and further, avoiding the overfitting phenomenon of the image denoising model to a certain extent. Then, by processing the image based on the obtained image denoising model, more image details can be retained while removing noise, thus improving the image quality of the denoised image.

[0143] In one embodiment, each image in the sample data is an image in RAW format.

[0144] In an embodiment of the present application, the ISP (Image Signal Process) chip in the image acquisition device can perform a series of processing on the RAW (original) image (i.e., the image in RAW format) input by the sensor, including AEC (Automatic Exposure Control), AGC (Automatic Gain Control), AWB (Automatic White Balance), color correction, etc., to obtain an sRGB (standard Red Green Blue, universal color standard) image. Since the sRGB image is obtained after a series of processing of the RAW image, if the sRGB image is denoised after the sRGB image is obtained, the noise contained in the original RAW image may not be effectively removed. Moreover, compared with the RAW image, the sRGB image has lost a lot of image details during the processing process. Even after denoising, the image quality of the denoised sRGB image is not high. Therefore, in the embodiment of the present application, the image denoising model can be trained based on the RAW image, and then, the RAW image can be processed based on the trained image denoising model, which can retain more image details while removing noise, thereby improving the image quality of the denoised RAW image, improving the denoising effect of the image denoising model, and then, the image quality of the generated sRGB image can be improved.

[0145] The present application also provides an image denoising method, see Figure 8 , Figure 8 A flowchart of an image denoising method provided in an embodiment of the present application, the method may include:

[0146] Step 801: Obtain an image to be denoised.

[0147] Step 802: input the image to be denoised into a pre-trained image denoising model to obtain a denoised image.

[0148] The image denoising model is obtained by training based on any of the above-mentioned image denoising model training method steps.

[0149] In an embodiment of the present application, an image denoising model trained based on any of the above-mentioned image denoising model training method steps is used to denoise the image to be denoised.

[0150] Since the above-mentioned image denoising model training method will force the output pixel value to be within the third pixel value interval, it can avoid forcibly fitting the noisy image to an image without noise, and further, it can also avoid the overfitting phenomenon of the image denoising model to a certain extent. Accordingly, by processing the image based on the obtained image denoising model, it is possible to remove noise while retaining more image details, thereby improving the image quality of the denoised image.

[0151] The present application also provides an image denoising model training device, see Fig. 9 , Fig. 9 A schematic diagram of the structure of an image denoising model training device provided in an embodiment of the present application, the device comprising:

[0152] The sample data acquisition module 901 is used to acquire sample data.

[0153] The sample data includes: multiple noise images containing noise, first label images containing no noise corresponding to each noise image, and second label images corresponding to each noise image; wherein each second label image is: a weighted sum of the corresponding noise image and the first label image.

[0154] The predicted image acquisition module 902 is used to input each noisy image into the image denoising model to be trained, and obtain the image output by the image denoising model as the predicted image.

[0155] The first calculation module 903 is used to calculate the loss value at each pixel position in the noise image based on the pixel value of the pixel position in the predicted image and the second pixel value interval if the pixel value of the pixel position in the predicted image is within the first pixel value interval.

[0156] Among them, the endpoints of the first pixel value interval include: the pixel value of the pixel position in the corresponding second label image and the pixel value of the pixel position in the noise image; one endpoint of the second pixel value interval is the pixel value of the pixel position in the corresponding second label image, and the second pixel value interval belongs to the third pixel value interval; the endpoints of the third pixel value interval include: the pixel value of the pixel position in the corresponding first label image and the pixel value of the pixel position in the corresponding second label image.

[0157] The second calculation module 904 is used to calculate the loss value of the noise image based on the loss value at each pixel position.

[0158] The model parameter adjustment module 905 is used to adjust the model parameters of the image denoising model to be trained based on the loss value of the noisy image until the convergence condition is reached to obtain a trained image denoising model.

[0159] Based on the above processing, if the pixel value of the pixel position in the predicted image is within the first pixel value interval, that is, between the pixel value of the pixel position in the corresponding second label image and the pixel value of the pixel position in the noise image, then the loss value at the pixel position can be calculated based on the pixel value of the pixel position in the predicted image and the second pixel value interval. Accordingly, adjusting the model parameters of the image denoising model based on the loss value can make the pixel value of the pixel position output by the image denoising model tend to the second pixel value interval. Since the second pixel value interval belongs to the third pixel value interval with the pixel value of the pixel position in the corresponding first label image (i.e., the image without noise) and the pixel value in the corresponding second label image as endpoints, the trained image denoising model can effectively denoise the image.

[0160] In addition, the related art uses the pixel value of the pixel position in the corresponding first label image to calculate the loss value, so as to force the noisy image to be fitted to the image without noise. In the solution of the present application, the loss value calculated based on the second pixel value interval will cause the output pixel value to be within the third pixel value interval, thus avoiding the forced fitting of the noisy image to the image without noise, and further, avoiding the overfitting phenomenon of the image denoising model to a certain extent. Further, by processing the image based on the obtained image denoising model, the image quality of the denoised image can be improved.

[0161] In one embodiment, the first calculation module 903 is specifically used to calculate the loss value at the pixel position based on the difference between the pixel value of the pixel position in the predicted image and the pixel value of the pixel position in the corresponding second label image if the pixel value of the pixel position in the predicted image is within the first pixel value interval.

[0162] In one embodiment, the image denoising model training device further includes:

[0163] The third calculation module is used to calculate the loss value at the pixel position based on the pixel value of the pixel position in the predicted image and the fifth pixel value interval if the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval.

[0164] Among them, the endpoints of the fourth pixel value interval include: the pixel value of the pixel position in the corresponding first label image and the pixel value of the pixel position in the noise image; one endpoint of the fifth pixel value interval is the pixel value of the pixel position in the corresponding first label image, and the fifth pixel value interval belongs to the third pixel value interval.

[0165] In one embodiment, the third calculation module is specifically used to calculate the loss value at the pixel position based on the difference between the pixel value of the pixel position in the predicted image and the pixel value of the pixel position in the corresponding first label image if the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval.

[0166] In one embodiment, the image denoising model training device further includes:

[0167] The fourth calculation module is used to determine the loss value at the pixel position as 0 if the pixel value of the pixel position in the predicted image is within a third pixel value interval.

[0168] In one embodiment, each image in the sample data is an image in RAW format.

[0169] The present application also provides an image denoising device. Fig.10 , Fig.10 This is a schematic diagram of the structure of an image denoising device provided in an embodiment of the present application, the device comprising:

[0170] The module 1001 for acquiring the image to be denoised is used to acquire the image to be denoised.

[0171] The denoising module 1002 is used to input the image to be denoised into a pre-trained image denoising model to obtain a denoised image.

[0172] The image denoising model is obtained by training based on any of the above-mentioned image denoising model training method steps.

[0173] The present application also provides an electronic device, such as Fig.11 As shown, it includes a processor 1101, a communication interface 1102, a memory 1103 and a communication bus 1104, wherein the processor 1101, the communication interface 1102, and the memory 1103 communicate with each other through the communication bus 1104.

[0174] Memory 1103, used for storing computer programs;

[0175] The processor 1101 is used to implement the following steps when executing the program stored in the memory 1103:

[0176] Acquire sample data, wherein the sample data includes: a plurality of noise images containing noise, first label images containing no noise corresponding to each noise image, and second label images corresponding to each noise image; wherein each second label image is: a weighted sum of the corresponding noise image and the second label image.

[0177] Each noisy image is input into the image denoising model to be trained, and the image output by the image denoising model is obtained as the predicted image.

[0178] For each pixel position in the noise image, if the pixel value of the pixel position in the predicted image is within the first pixel value interval, the loss value at the pixel position is calculated based on the pixel value of the pixel position in the predicted image and the second pixel value interval. The endpoints of the first pixel value interval include: the pixel value of the pixel position in the corresponding second label image and the pixel value of the pixel position in the noise image; one endpoint of the second pixel value interval is the pixel value of the pixel position in the corresponding second label image, and the second pixel value interval belongs to the third pixel value interval; the endpoints of the third pixel value interval include: the pixel value of the pixel position in the corresponding first label image and the pixel value of the pixel position in the corresponding second label image.

[0179] Based on the loss value at each pixel position, the loss value of the noisy image is calculated.

[0180] Based on the loss value of the noisy image, the model parameters of the image denoising model to be trained are adjusted until the convergence condition is reached, thereby obtaining a trained image denoising model.

[0181] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0182] The communication interface is used for communication between the above electronic device and other devices.

[0183] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0184] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0185] In another embodiment provided in the present application, a computer-readable storage medium is also provided, which stores a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned image denoising model training methods or image denoising methods are implemented.

[0186] In another embodiment provided in the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any image denoising model training method or image denoising method in the above-mentioned embodiments.

[0187] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0188] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0189] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0190] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A method for training an image denoising model, characterized in that: The method comprises: Acquire sample data; wherein the sample data includes: a plurality of noise images containing noise, first label images containing no noise corresponding to each noise image, and second label images corresponding to each noise image; wherein each second label image is: a weighted sum of the corresponding noise image and the first label image; Input each noisy image into the image denoising model to be trained, and obtain an image output by the image denoising model as a predicted image; For each pixel position in the noise image, if the pixel value of the pixel position in the predicted image is within the first pixel value interval, then based on the pixel value of the pixel position in the predicted image and the second pixel value interval, calculate the loss value at the pixel position; Wherein, the endpoints of the first pixel value interval include: the pixel value of the pixel position in the corresponding second label image and the pixel value of the pixel position in the noise image; one endpoint of the second pixel value interval is the pixel value of the pixel position in the corresponding second label image, and the second pixel value interval belongs to the third pixel value interval; the endpoints of the third pixel value interval include: the pixel value of the pixel position in the corresponding first label image and the pixel value of the pixel position in the corresponding second label image; the loss value at the pixel position is calculated based on the difference between the pixel value of the pixel position in the predicted image and a pixel value selected from the second pixel value interval; Based on the loss value at each pixel position, the loss value of the noise image is calculated; Based on the loss value of the noisy image, the model parameters of the image denoising model to be trained are adjusted until the convergence condition is reached, thereby obtaining a trained image denoising model.

2. The method according to claim 1, characterized in that If the pixel value of the pixel position in the predicted image is within the first pixel value interval, then based on the pixel value of the pixel position in the predicted image and the second pixel value interval, calculating the loss value at the pixel position, comprises: If the pixel value of the pixel position in the predicted image is within the first pixel value interval, the loss value at the pixel position is calculated based on the difference between the pixel value of the pixel position in the predicted image and the pixel value of the pixel position in the corresponding second label image.

3. The method according to claim 1, characterized in that: The method further comprises: If the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval, the loss value at the pixel position is calculated based on the pixel value of the pixel position in the predicted image and the fifth pixel value interval; wherein the endpoints of the fourth pixel value interval include: the pixel value of the pixel position in the corresponding first label image and the pixel value of the pixel position in the noise image; one endpoint of the fifth pixel value interval is the pixel value of the pixel position in the corresponding first label image, and the fifth pixel value interval belongs to the third pixel value interval.

4. The method according to claim 3, characterized in that If the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval, calculating the loss value at the pixel position based on the pixel value of the pixel position in the predicted image and the fifth pixel value interval, comprises: If the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval, the loss value at the pixel position is calculated based on the difference between the pixel value of the pixel position in the predicted image and the pixel value of the pixel position in the corresponding first label image.

5. The method according to claim 1, characterized in that The method further comprises: If the pixel value of the pixel position in the predicted image is within the third pixel value interval, the loss value at the pixel position is determined to be 0.

6. The method according to any one of claims 1 to 5, characterized in that: Each image in the sample data is an image in RAW format.

7. An image denoising method, characterized in that: The method comprises: Obtain an image to be denoised; The image to be denoised is input into a pre-trained image denoising model to obtain a denoised image; wherein the image denoising model is trained based on any one of the method steps described in claims 1-6.

8. An image denoising model training device, characterized in that: The device comprises: A sample data acquisition module, used to acquire sample data; wherein the sample data includes: a plurality of noise images containing noise, first label images containing no noise corresponding to each noise image, and second label images corresponding to each noise image; wherein each second label image is: a weighted sum of the corresponding noise image and the first label image; A prediction image acquisition module, used for inputting each noise image into the image denoising model to be trained, and obtaining an image output by the image denoising model as a prediction image; A first calculation module is used for calculating, for each pixel position in the noise image, a loss value at the pixel position based on the pixel value of the pixel position in the predicted image and a second pixel value interval if the pixel value of the pixel position in the predicted image is within a first pixel value interval; Wherein, the endpoints of the first pixel value interval include: the pixel value of the pixel position in the corresponding second label image and the pixel value of the pixel position in the noise image; one endpoint of the second pixel value interval is the pixel value of the pixel position in the corresponding second label image, and the second pixel value interval belongs to the third pixel value interval; the endpoints of the third pixel value interval include: the pixel value of the pixel position in the corresponding first label image and the pixel value of the pixel position in the corresponding second label image; the loss value at the pixel position is calculated based on the difference between the pixel value of the pixel position in the predicted image and a pixel value selected from the second pixel value interval; A second calculation module, used for calculating the loss value of the noise image based on the loss value at each pixel position; The model parameter adjustment module is used to adjust the model parameters of the image denoising model to be trained based on the loss value of the noisy image until the convergence condition is reached to obtain a trained image denoising model.

9. The device according to claim 8, characterized in that The first calculation module is specifically used to calculate the loss value at the pixel position based on the difference between the pixel value of the pixel position in the predicted image and the pixel value of the pixel position in the corresponding second label image if the pixel value of the pixel position in the predicted image is within a first pixel value interval.

10. The device according to claim 8, characterized in that The device also includes: A third calculation module is used to calculate the loss value at the pixel position based on the pixel value of the pixel position in the predicted image and the fifth pixel value interval if the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval; wherein the endpoints of the fourth pixel value interval include: the pixel value of the pixel position in the corresponding first label image and the pixel value of the pixel position in the noise image; one endpoint of the fifth pixel value interval is the pixel value of the pixel position in the corresponding first label image, and the fifth pixel value interval belongs to the third pixel value interval.

11. The device according to claim 10, characterized in that The third calculation module is specifically used to calculate the loss value at the pixel position based on the difference between the pixel value of the pixel position in the predicted image and the pixel value of the pixel position in the corresponding first label image if the pixel value of the pixel position in the predicted image is outside the fourth pixel value interval.

12. The device according to claim 8, characterized in that The device also includes: The fourth calculation module is used to determine the loss value at the pixel position as 0 if the pixel value of the pixel position in the predicted image is within the third pixel value interval.

13. The device according to any one of claims 8 to 12, characterized in that: Each image in the sample data is an image in RAW format.

14. An image denoising device, characterized in that: The device comprises: A module for acquiring an image to be denoised, used for acquiring an image to be denoised; A denoising module is used to input the image to be denoised into a pre-trained image denoising model to obtain a denoised image; wherein the image denoising model is trained based on any one of the method steps described in claims 1-6.

15. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1-6 or 7 when executing a program stored in a memory.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1-6, or 7 are implemented.

Images