Image noise reduction method, and training method and device of noise reduction model

Abstract

The invention relates to an image noise reduction method, a noise reduction model training method and a noise reduction model training device. The method comprises the following steps: acquiring a to-be-denoised image; inputting pixel data in the image to be denoised into a decoder of a pre-trained denoising model, and outputting a denoised image through the decoder in the denoising model; wherein the training process of the noise reduction model comprises the steps of calculating a reference pixel value based on a plurality of images which are shot in advance and have different exposure values in the same scene, obtaining a noise-added pixel value in an encoder of a pre-training model by utilizing a predetermined noise distribution condition and the reference pixel value, and performing noise reduction on the noise-added pixel value; and obtaining a de-noised pixel value based on the noise-added pixel value in a decoder of the pre-training model, and iterating the decoder of the pre-training model based on a loss value between the de-noised pixel value and the reference pixel value to obtain a noise reduction model. By adopting the method, the noise in the image can be accurately removed.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image denoising method, a denoising model training method and a denoising device. Background Art

[0002] With the development of image processing technology, image processing technology has been widely used in various fields, such as image noise reduction, etc. Among them, the purpose of image noise reduction is to remove noise in the image, improve the quality of the image, and obtain a clearer image.

[0003] Traditional image denoising techniques mostly use filtering methods, but this will damage the overall clarity and sharpness of the image, resulting in unclear images. Some other solutions remove the noise model based on theoretical modeling. However, since the location and intensity of noise on the image are probabilistic, the error of removing it with a fixed noise value is too large. In addition, the use of neural network models can also remove noise from images. However, this type of neural network model is usually not an end-to-end network model, and requires additional feature processing and a large amount of additional data for training, resulting in the model being heavily dependent on the accuracy of the training data. Summary of the invention

[0004] Based on this, it is necessary to provide an image denoising method, a denoising model training method and a device to address the above technical problems.

[0005] In a first aspect, the present disclosure provides an image noise reduction method. The method comprises:

[0006] Obtain an image to be denoised;

[0007] The pixel data in the image to be denoised is input into the decoder of a pre-trained denoising model, and the denoised image is output via the decoder in the denoising model; wherein the training process of the denoising model includes: calculating a reference pixel value based on a plurality of images with different exposure values ​​of the same scene taken in advance, obtaining a noisy pixel value in the encoder of the pre-trained model using a predetermined noise distribution and the reference pixel value, obtaining a denoised pixel value based on the noisy pixel value in the decoder of the pre-trained model, and iterating the decoder of the pre-trained model based on a loss value between the denoised pixel value and the reference pixel value to obtain a denoising model.

[0008] In one embodiment, the method further comprises:

[0009] Obtain multiple images with different exposure values ​​taken in the same scene;

[0010] Based on the multiple images with different exposure values, a pixel mean value at each position is calculated, and a reference pixel value at each position is determined based on the pixel mean value at each position;

[0011] In an encoder of the pre-trained model, a noisy pixel value is obtained based on the predetermined noise distribution and the reference pixel value;

[0012] In the decoder of the pre-trained model, a denoised pixel value is obtained based on the noisy pixel value, and the decoder of the pre-trained model is iterated based on a loss value between the denoised pixel value and the reference pixel value to obtain a denoising model.

[0013] In one embodiment, obtaining the noisy pixel value based on the predetermined noise distribution and the reference pixel value includes:

[0014] Based on the reference pixel value at each position, a pixel variance at each position is calculated;

[0015] Based on the reference pixel value and the pixel variance, a linear regression model for calculating noise parameters is constructed, and the noise parameters are calculated using the linear regression model;

[0016] Performing a linear transformation on the noise parameters, and determining a noise pixel distribution interval based on a predetermined noise model, noise distribution, and the noise parameters after the linear transformation;

[0017] Sampling is performed in the noise pixel distribution interval to obtain a noisy pixel value.

[0018] In one embodiment, the noise model is determined in the following manner:

[0019] determining noise generated by the imaging device before and after an analog gain stage in an imaging stage;

[0020] A noise model is generated according to the actual number of photons obtained in the imaging stage, quantum efficiency, and noise generated in stages before and after the simulated gain stage, wherein the quantum efficiency represents the efficiency of converting photons into electrons.

[0021] In one embodiment, performing a linear transformation on the noise parameter and determining a noise pixel distribution interval based on a predetermined noise model, noise distribution and the noise parameter after the linear transformation comprises:

[0022] Based on the noise model and the noise distribution, generate a noise distribution model;

[0023] Based on the noise distribution model and the noise parameters after linear transformation, a noise pixel distribution interval is determined.

[0024] In one embodiment, obtaining a denoised pixel value based on the noisy pixel value in the decoder of the pre-trained model, iterating the decoder of the pre-trained model based on a loss value between the denoised pixel value and the reference pixel value to obtain a denoising model, comprises:

[0025] In response to the decoder of the pre-trained model performing back propagation, randomly sampling from a standard normal distribution interval to obtain a random sampling value;

[0026] Based on the random sampling value and the noise pixel distribution interval, the decoder of the pre-trained model is back-propagated using the re-parameterization technology to update the parameters in the decoder of the pre-trained model to obtain a denoising model.

[0027] In one of the embodiments, before constructing a linear regression model for calculating noise parameters based on the reference pixel value and the pixel variance and calculating the noise parameters using the linear regression model, the method further includes:

[0028] Parameters in a linear regression model are determined based on the target photon number obtained in the imaging stage, the actual photon number and the noise model.

[0029] In a second aspect, the present disclosure further provides a method for training a denoising model, the method comprising:

[0030] Obtain multiple images with different exposure values ​​taken in the same scene;

[0031] Based on the multiple images with different exposure values, a pixel mean value at each position is calculated, and a reference pixel value at each position is determined based on the pixel mean value at each position;

[0032] In an encoder of the pre-trained model, a noisy pixel value is obtained based on the predetermined noise distribution and the reference pixel value;

[0033] In the decoder of the pre-trained model, a denoised pixel value is obtained based on the noisy pixel value, and the decoder of the pre-trained model is iterated based on a loss value between the denoised pixel value and the reference pixel value to obtain a denoising model.

[0034] In a third aspect, the present disclosure further provides an image noise reduction device. The device comprises:

[0035] A data acquisition module, used for acquiring the image to be denoised;

[0036] A model processing module is used to input the pixel data in the image to be denoised into the decoder of a pre-trained denoising model, and output the denoised image via the decoder in the denoising model; wherein the training process of the denoising model includes: calculating a reference pixel value based on a plurality of images with different exposure values ​​of the same scene taken in advance, applying noise to the reference pixel value using a predetermined noise distribution in the encoder of the pre-trained model to obtain a noisy pixel value, obtaining a denoised pixel value based on the noisy pixel value in the decoder of the pre-trained model, and iterating the decoder of the pre-trained model based on a loss value between the denoised pixel value and the reference pixel value to obtain a denoising model.

[0037] In a fourth aspect, the present disclosure further provides a training device for a noise reduction model, the device comprising:

[0038] An image acquisition module is used to acquire multiple images with different exposure values ​​taken in the same scene;

[0039] A pixel processing module, configured to calculate a pixel mean value at each position based on the plurality of images with different exposure values, and determine a reference pixel value at each position based on the pixel mean value at each position;

[0040] A noise adding module, configured to obtain a noisy pixel value based on the predetermined noise distribution and the reference pixel value in an encoder of a pre-trained model;

[0041] The model training module is used to obtain a denoised pixel value based on the noisy pixel value in the decoder of the pre-trained model, and iterate the decoder of the pre-trained model based on the loss value between the denoised pixel value and the reference pixel value to obtain a denoising model.

[0042] In a fourth aspect, the present disclosure further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in any of the above method embodiments are implemented.

[0043] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0044] In a fifth aspect, the present disclosure further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0045] In the above-mentioned image denoising method, denoising model training method and device, the reference pixel value is calculated by using multiple images with different exposure values ​​in the same scene, which makes full use of the consistency information of the scene. Images in the same scene have similar content and structure, and different exposure values ​​cause the images to change in brightness and other aspects, which provides a rich data basis for accurately calculating the reference pixel value. This method can better capture the pixel features and noise characteristics in the scene, so that the subsequently trained denoising model has stronger pertinence to the noise in the scene. During the training process, the denoised pixel value is obtained in the encoder in combination with the predetermined noise distribution and the reference pixel value. Since it is based on the image data of a specific scene, the determined noise distribution can be more consistent with the actual noise situation of the scene. For example, if the noise of the scene is mainly caused by a certain distribution of noise (such as Poisson noise under low light, etc.) caused by light changes, this method can more accurately simulate and learn this noise, and improve the model's ability to handle the noise of the scene. During the training process, the denoising process is first simulated in the encoder, and then the denoising operation is performed in the decoder, and the decoder is iterated based on the loss value between the denoised pixel value and the reference pixel value. In addition, since the decoder is used for denoising, only the decoder can be trained without training the encoder, which can improve the speed of model training. By calculating the loss value between the denoised pixel value and the reference pixel value, and iteratively updating the decoder based on this, the decoder can continuously adjust its own parameters to reduce the loss value. This iterative optimization process based on the loss value can make the model gradually converge to a better state and improve the denoising performance of the model. For example, when the mean square error is used as the loss function, as the number of iterations increases, the model can make the denoised pixel value closer to the reference pixel value, thereby improving the noise removal effect. After obtaining the image to be denoised, its pixel data can be directly input into the decoder of the pre-trained denoising model to quickly obtain the denoised image. This application method is simple and efficient, and there is no need to re-perform a complex training process for each image to be denoised. Since the model has been effectively trained in a specific scenario, it can quickly and accurately denoise the input image, improving the efficiency and quality of image denoising. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A schematic diagram of an application environment of an image noise reduction method in an embodiment;

[0048] Figure 2 is a schematic diagram of a flow chart of an image noise reduction method in one embodiment;

[0049] Figure 3 A schematic diagram of a flow chart of a model training step in an embodiment;

[0050] Figure 4 Schematic diagram of the process of step S306 in one embodiment;

[0051] Figure 5 A schematic flow chart of the steps of determining a noise model in one embodiment;

[0052] Figure 6 A schematic diagram of a process of converting photons into digital values ​​in one embodiment;

[0053] Figure 7 This is a schematic diagram of the process of step S308 in one embodiment;

[0054] Figure 8 is a schematic block diagram of the structure of an image noise reduction device in one embodiment;

[0055] Fig. 9 is a schematic block diagram of the structure of a training device for a noise reduction model in one embodiment;

[0056] Fig.10 A schematic diagram of the internal structure of a computer device in one embodiment

[0057] Fig.11 Schematic diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present disclosure more clear, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0059] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0060] In this article, the term "and / or" is only a description of the association relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects before and after are in an "or" relationship.

[0061] The present disclosure provides an image noise reduction method, which can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 can obtain the image to be denoised. The terminal 102 can input the pixel data in the image to be denoised into the decoder of the pre-trained denoising model, and output the denoised image through the decoder in the denoising model; wherein the training process of the denoising model includes: the terminal 102 or the server 104 calculates the reference pixel value based on a plurality of images with different exposure values ​​of the same scene taken in advance, and obtains the noisy pixel value in the encoder of the pre-trained model by using the predetermined noise distribution and the reference pixel value, and obtains the denoised pixel value based on the denoised pixel value in the decoder of the pre-trained model, and iterates the decoder of the pre-trained model based on the loss value between the denoised pixel value and the reference pixel value to obtain the denoising model. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, etc. The server 104 can be implemented as an independent server or a server cluster composed of multiple servers. It should be noted that this solution can also be used for a terminal or a server alone.

[0062] In one embodiment, Figure 2 As shown, an image denoising method is provided, which is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate, and the following steps are included:

[0063] S202: Obtain an image to be denoised.

[0064] Among them, the image to be denoised refers to the original image containing noise. Image noise is an unwanted part of the image, which is interference information introduced by various factors such as the imaging system, transmission medium or image processing process. For example, during the shooting process of a digital camera, noise will be generated on the image due to the thermal effect of the sensor, uneven light, interference from electronic components and the like. In some embodiments of the present disclosure, the noise generated during the imaging process of the sensor is generally considered. The type of noise is usually Gaussian noise, and the probability density function of the noise obeys the Gaussian distribution (normal distribution). The characteristic of this noise is that it appears as a random change in brightness or color on the image, which looks like a layer of "salt and pepper" sprinkled on the image, but the distribution of salt and pepper conforms to the law of Gaussian distribution. For example, photos taken under low light conditions are prone to Gaussian noise due to the signal amplification of the sensor.

[0065] S204, inputting the pixel data in the image to be denoised into the decoder of the pre-trained denoising model, and outputting the denoised image through the decoder in the denoising model; wherein the training process of the denoising model includes: calculating the reference pixel value based on a plurality of images with different exposure values ​​of the same scene taken in advance, obtaining the noisy pixel value in the encoder of the pre-trained model using the predetermined noise distribution and the reference pixel value, obtaining the denoised pixel value based on the noisy pixel value in the decoder of the pre-trained model, and iterating the decoder of the pre-trained model based on the loss value between the denoised pixel value and the reference pixel value to obtain the denoising model.

[0066] Among them, the denoising model can be a trained variational autoencoder (VAE) model. The VAE model is widely used in the field of generative models. It can learn the potential distribution of data and generate new data by sampling latent variables. In addition, the VAE model contains an encoder and a decoder. The goal of VAE is to learn the potential representation of data and generate new data from the potential representation. The encoder is responsible for mapping the input data to the latent space (usually the mean and variance), and the decoder is responsible for reconstructing the original data from the sampling points of the latent space. In some embodiments of the present disclosure, during the training process, the encoder is responsible for denoising the noise-free image, and the decoder is responsible for denoising the noisy image. In the application process of the denoising model, since it is not necessary to denoise the image, only the image needs to be denoised, therefore, only the decoder needs to be used for processing, and the encoder does not need to be used for processing, therefore, there is no need to train the encoder during the training process.

[0067] Specifically, the reference pixel value is first calculated using multiple images of the same scene with different exposure values ​​taken in advance. The image content of the same scene is basically the same, but the pixel values ​​will be different due to different exposures. By processing and analyzing these images, a relatively accurate reference pixel value can be obtained, which can be used as a benchmark (i.e., noise-free pixel value) in the subsequent training process. For example, the reference pixel can be obtained by the averaging method or the median method. The average method is to add the values ​​of the pixel values ​​at the same position in the same scene in different exposure images and then take the average; the median method is to take the median of these pixel values ​​as the reference pixel value. The average method is suitable for situations where the noise is relatively uniform and there are no obvious outliers. It can smooth out some of the noise; the median method has good robustness to outliers such as salt and pepper noise. According to different application scenarios, technicians in this field can flexibly choose the corresponding method of calculating the reference pixel value. The reference pixel value is input into the encoder of the pre-trained model. At the same time, the actual noise addition process is simulated by using a predetermined noise distribution, such as a known noise that obeys a certain distribution (such as Gaussian distribution, Poisson distribution, etc.), combined with the reference pixel value, so as to obtain the noisy pixel value in the encoder. This step is to allow the model to learn the pixel feature representation in the presence of noise. After obtaining the noisy pixel value, it is input into the decoder of the pre-trained model. The decoder will process the noisy pixel value according to its own structure and parameters, and try to restore the original noise-free or low-noise pixel value, that is, to obtain the denoised pixel value. The loss value between the denoised pixel value and the reference pixel value is calculated. This loss value can measure the accuracy of the decoder restoration. Commonly used loss functions include mean square error (MSE), etc. Then, based on this loss value, the decoder of the pre-trained model is iteratively updated using an optimization algorithm (such as stochastic gradient descent, etc.). In each iteration, the parameters of the decoder are adjusted so that the loss value gradually decreases until certain stop conditions are met (such as loss value convergence, reaching the maximum number of iterations, etc.), and finally a denoising model is obtained. Through such a training process, the decoder can learn how to better restore denoised pixel values ​​close to reference pixel values ​​from noisy pixel values, thereby improving the model's denoising ability. After the denoising model is trained, the denoised image can be input into the decoder of the denoising model, and the denoised image can be output through the decoder.

[0068] In the above-mentioned image denoising method, the reference pixel value is calculated by using multiple images with different exposure values ​​in the same scene, which makes full use of the consistency information of the scene. Images in the same scene have similar content and structure, and different exposure values ​​cause the images to change in brightness and other aspects, which provides a rich data basis for accurately calculating the reference pixel value. This method can better capture the pixel features and noise characteristics in the scene, so that the subsequently trained denoising model has stronger pertinence to the noise in the scene. During the training process, the denoised pixel value is obtained in the encoder in combination with the predetermined noise distribution and the reference pixel value. Since it is based on the image data of a specific scene, the determined noise distribution can be more consistent with the actual noise situation of the scene. For example, if the noise of the scene is mainly caused by a certain distribution of noise (such as Poisson noise under low light, etc.) caused by light changes, this method can more accurately simulate and learn this noise, and improve the model's ability to handle the noise of the scene. During the training process, the denoising process is first simulated in the encoder, and then the denoising operation is performed in the decoder. The whole process does not require additional feature processing, and the decoder is iterated based on the loss value between the denoised pixel value and the reference pixel value. In addition, since the decoder is used for denoising, only the decoder can be trained without training the encoder, which can improve the speed of model training. By calculating the loss value between the denoised pixel value and the reference pixel value, and iteratively updating the decoder based on this, the decoder can continuously adjust its own parameters to reduce the loss value. This iterative optimization process based on the loss value can make the model gradually converge to a better state and improve the denoising performance of the model. For example, when the mean square error is used as the loss function, as the number of iterations increases, the model can make the denoised pixel value closer to the reference pixel value, thereby improving the noise removal effect. After obtaining the image to be denoised, its pixel data can be directly input into the decoder of the pre-trained denoising model to quickly obtain the denoised image. This application method is simple and efficient, and there is no need to re-perform a complex training process for each image to be denoised. Since the model has been effectively trained in a specific scenario, it can quickly and accurately denoise the input image, improving the efficiency and quality of image denoising.

[0069] In one embodiment, Figure 3 As shown, the method also includes:

[0070] S302, acquiring a plurality of images with different exposure values ​​captured in the same scene.

[0071] Specifically, you can change the ISO value of the camera. For example, in a dimly lit indoor scene, first set the ISO to a lower value (such as ISO 100) to shoot a darker image, and then gradually increase the ISO (such as ISO 200, ISO400, etc.) to shoot, so as to obtain a series of images with different exposure values. In addition, you can also adjust the aperture size. The aperture controls the amount of light entering the lens. Its value is expressed by the f-value. The smaller the f-value, the larger the aperture, the more light entering, and the higher the exposure value; conversely, the larger the f-value, the smaller the aperture, the less light entering, and the lower the exposure value. Under the premise of fixing the shutter speed and ISO, manually adjust the aperture size, such as from f / 8 to f / 5.6, f / 4, etc., and shoot in sequence to obtain images with different exposure conditions in the same scene.

[0072] S304, based on the multiple images with different exposure values, calculate the pixel mean of each position, and determine the reference pixel value of each position based on the pixel mean of each position. In addition, the pixel variance of each position can also be calculated based on the reference pixel value of each position.

[0073] Specifically, for multiple images with different exposure values, the pixel mean and pixel variance of each position in the image may be calculated.

[0074] In some exemplary embodiments, since the images are of different exposure values ​​in the same scene, before calculating the pixel mean and variance, it is necessary to ensure that the images are aligned in spatial position, that is, the objects and other contents in the images must correspond accurately, so that the subsequent statistics for each pixel at the same position are meaningful. The images with different exposure values ​​can be aligned using a feature point matching method or an optical flow-based method. After the image alignment is completed, statistics can be performed for each specific pixel position. For a two-dimensional digital image, the pixel position is usually represented by the coordinate (x, y), where x represents the horizontal coordinate and y represents the vertical coordinate. For example, the pixel position in the upper left corner of the image can be represented as (0, 0), and each pixel point to the right and downward has its corresponding coordinate position. We need to traverse each such coordinate position in the image to calculate its related statistics. For multiple images with different exposure values ​​that have been aligned (assuming that the number of images is N), the pixel values ​​at the corresponding coordinate position (x, y) of each image are read in sequence. For example, read the pixel value at position (10, 20) of the first image, the pixel value at position (10, 20) of the second image, and so on, until all the pixel values ​​at that position of N images are traversed. Sum the N pixel values ​​read at that position, and then divide by the number of images N. The result is the average pixel value at that position (x, y). It can be expressed as follows:

[0075]

[0076] in, Represents the pixel mean at the position with coordinates (x, y), P i，x，y represents the pixel value of the i-th image at the coordinate (x, y), and N is the total number of images.

[0077] Then, for the coordinate position (x, y), first calculate the pixel value P of each image at that position i，x，y , and the pixel mean calculated previously The difference is then squared and summed, that is Divide the sum of squared differences by the number of images N (when calculating sample variance, it is usually divided by N-1, but here we take dividing by N as an example to simplify the explanation), and the result is the pixel variance at the position (x, y). It can be expressed as follows:

[0078]

[0079] S306: Obtain a noisy pixel value in an encoder of the pre-trained model based on the predetermined noise distribution and the reference pixel value.

[0080] Specifically, we first need to clarify the noise distribution. Common noise distributions include Gaussian noise (normal distribution), Poisson noise, etc. Based on the noise distribution, we can use a random number generator to generate noise samples that conform to the distribution. Add the generated noise sample to the reference pixel value to get the noisy pixel value.

[0081] S308, obtaining a denoised pixel value based on the noisy pixel value in the decoder of the pre-trained model, and iterating the decoder of the pre-trained model based on a loss value between the denoised pixel value and the reference pixel value to obtain a denoising model.

[0082] Specifically, the decoder of the pre-trained model has a specific network structure and parameters. When a noisy pixel value is input, the decoder performs a series of linear and nonlinear transformations on the input noisy pixel value according to its internal convolutional layer, fully connected layer and other components and the corresponding weights and bias parameters. These transformations are intended to restore the original noise-free or low-noise pixel value, that is, the denoised pixel value. During the training process, the following steps are repeated: the noisy pixel value is input into the decoder to obtain the denoised pixel value. The loss value between the denoised pixel value and the reference pixel value is calculated. According to the selected optimization algorithm, the loss value is used to update the parameters of the decoder.

[0083] In one embodiment, Figure 4 As shown, the obtaining of the noisy pixel value based on the predetermined noise distribution and the reference pixel value includes:

[0084] S402: Calculate the pixel variance of each position based on the reference pixel value of each position.

[0085] Specifically, how to calculate the reference pixel value here can be referred to the above embodiment, which will not be repeated here.

[0086] S404: construct a linear regression model for calculating noise parameters based on the reference pixel value and the pixel variance, and obtain the noise parameters by calculating using the linear regression model.

[0087] Specifically, we can first determine the parameters that affect the noise, take the parameters that affect the noise as unknown quantities (dependent variables), take the mean and pixel variance as independent variables to construct a linear regression model for calculating the noise parameters, and use the mean and pixel variance to calculate the values ​​of the parameters in the linear regression model to obtain the noise parameters.

[0088] In some exemplary embodiments, for example, if the parameters affecting the noise are k and σ, the following linear regression model can be constructed:

[0089] Among them, X * is the pixel mean (reference pixel value), and Var(X) is the pixel variance.

[0090] S406: Perform a linear transformation on the noise parameters, and determine a noise pixel distribution interval based on a predetermined noise model, noise distribution, and the noise parameters after the linear transformation.

[0091] Specifically, cameras automatically adjust ISO settings based on scene lighting, so different noise levels must be considered when training denoising neural networks. A straightforward solution is to train a single network to cover a wide range of ISO settings, but this places an additional burden on the network itself as the noise variation in the training data becomes quite large. Inspired by the variance-stabilizing transformation, a linear transformation is proposed here to address this problem. Variance-Stabilizing Transformation is a data transformation technique. Its main purpose is to convert the variance of the data into a relatively stable value so that the variance of the data does not depend on the mean of the data or other factors. In many statistical analysis and machine learning tasks, the variance of the data may vary with the mean of the data or other variables. For example, in some data with Poisson distribution or binomial distribution, the variance and mean are correlated. Taking Poisson distribution as an example, its variance is equal to the mean. This correlation between variance and mean may bring difficulties to subsequent analysis and model training. The variance stabilization transformation transforms the original data through a specific mathematical function, so that the variance of the transformed data is no longer closely related to the mean, or at least the variance variation range is reduced to a more manageable interval. Therefore, the noise parameters can be linearly transformed in a manner similar to the variance stabilization change, and the distribution characteristics of the noise-free signal can be inferred based on the known information of the noise distribution (which conforms to the normal distribution) and the transformed noise parameters and the predetermined noise model. For example, if the noise is Poisson noise, after the transformation, by analyzing the statistical characteristics of the transformation (such as mean, variance, etc.), and using the inverse transformation relationship, the distribution interval of the noise pixel value can be inferred.

[0092] In some exemplary embodiments, the noise parameter after linear transformation may be: Among them, k and σ 2 is the noise parameter.

[0093] S408, sampling is performed in the noise pixel distribution interval to obtain a noisy pixel value.

[0094] Specifically, after the noise-free pixel distribution interval is determined, random sampling can be performed in the noise pixel distribution interval to obtain noise-added pixels.

[0095] In this embodiment, the noise parameters are calculated by calculating the pixel variance based on the reference pixel value at each position, and combining the two to build a linear regression model. This method makes full use of the statistical information of the pixel value, and can more accurately reflect the noise characteristics in the data compared to some simple estimation methods. For example, in image data, pixels at different positions may be affected by noise to different degrees. By calculating the variance of each position and combining the reference pixel value, this change can be captured more carefully, thereby obtaining a noise parameter that is more in line with the actual situation. The pixel variance at each position is calculated instead of using the overall variance estimation, which enables the estimation of noise to take into account the differences in the spatial positions of pixels. In actual scenes, different areas of an image may have different noise levels. For example, the edge area and the center area of ​​an image may have different noise distributions due to factors such as light propagation and sensor characteristics. This position-based variance calculation and noise parameter estimation can better adapt to this complex noise situation. When determining the noise pixel distribution interval, not only the predetermined noise model and noise distribution are considered, but also the noise parameters after linear transformation are combined. This comprehensive consideration of multiple factors can more accurately depict the distribution range of noise pixels. For example, if it is known that the noise follows a Poisson distribution, and after a suitable linear transformation is performed based on the noise parameters calculated previously, the characteristics of the Poisson distribution and the transformed parameters can be used to more accurately determine the distribution interval of the noise pixels, rather than relying solely on the general assumptions of the noise model. Sampling the determined noise pixel distribution interval to obtain the noisy pixel value ensures that the generated noisy pixel value conforms to the estimated noise distribution law. Compared with random or inappropriate noise addition methods, the noisy pixel values ​​obtained by sampling within a reasonable distribution interval can more realistically simulate the impact of actual noise on pixels. For example, in image denoising research, reasonable noisy pixel values ​​can be better used to train the model, allowing the model to learn more accurate noise features and denoising methods. Obtaining noisy pixel values ​​in the above reasonable manner for subsequent model training (such as in the encoder of the pre-trained model) can improve the model's ability to learn noise. The model can better adapt to different noise levels and distributions, thereby improving the performance of the model in denoising tasks.

[0096] In one embodiment, Figure 5 As shown, the noise model includes determining in the following manner:

[0097] S502, determining the noise generated by the imaging device in the stages before and after the analog gain stage in the imaging stage.

[0098] S504, generating a noise model according to the actual number of photons obtained in the imaging stage, the quantum efficiency, and the noise generated in the stages before and after the simulated gain stage, wherein the quantum efficiency represents the efficiency of converting photons into electrons.

[0099] Specifically, Figure 6 As shown in the figure, the process of converting photons into digital values ​​by the camera sensor is shown, and the noise sources that may be introduced in each stage are marked. It can include the following stages: electron-voltage conversion stage, analog amplification stage, and analog-to-digital conversion stage. In the electron-voltage conversion stage, the concept of quantum efficiency (quantum efficiency, represented by α) is introduced in the process of converting photons into electrons. Quantum efficiency describes the efficiency of converting photons into electrons. After photons are converted into electrons, the next step of processing can be carried out. Electrons are converted into voltage (v). In this conversion process, dark noise (dark noise, represented by nd) is introduced. Dark noise is the noise generated by random thermal motion of electrons when there is no photon irradiation. Then the voltage signal is amplified by an analog amplifier, and analog gain (Analoggain, represented by g) is introduced in this process. Analog gain is used to enhance signal strength. The amplified voltage signal is converted into analog-to-digital conversion to convert the analog voltage into a digital value (x). In this process, read and quantization noise (read and quantization noise, represented by nr) is introduced. Read and quantization noise is the noise generated when converting analog signals into digital signals. From the above process, it can be seen that in the process of converting photons into digital values, dark noise will be generated before the analog gain stage. Read and quantization noise will be generated after the analog gain stage, so the dark noise and read and quantization noise can be determined. The noise model is generated based on the dark noise and read and quantization noise, as well as the actual number of photons and quantum efficiency obtained during the imaging stage.

[0100] In some exemplary embodiments, the ideal noise-free model can be expressed as follows:

[0101] x * =gαu *

[0102] Among them, x * represents a noise-free pixel, u * is the expected number of photons hitting the pixel area, α is the quantum efficiency, and g is the simulated gain.

[0103] The noise model can be expressed as follows:

[0104] x=g(αu+n d )+n r

[0105] Where u represents the actual number of photons collected, α is the quantum efficiency, g is the simulated gain, and n d is the noise after the analog gain stage (dark noise), nr Read and quantization noise. In addition, it should be noted that are Gaussian noise before and after applying analog gain, both of which satisfy normal (Gaussian) distribution. x represents the pixel value with noise. Usually, u obeys u * The Poisson distribution of

[0106] In addition, after the noise model is obtained, a linear regression model for calculating the noise parameter is constructed based on the reference pixel value and the pixel variance. Before the noise parameter is calculated using the linear regression model, the target photon number (u * ), the actual number of photons and the noise model, and determine the parameters in the linear regression model.

[0107] Specifically, combining the above content (noise model), the target photon number (u * ), the actual number of photons (u), and analysis is performed to determine the parameters that affect the noise in the linear regression model.

[0108] In some exemplary embodiments, x * =gαu * 、x=g(αu+n d )+n r , Combined This means that x obeys (gα) times The Poisson distribution has a mean of 0 and a variance of The sum of the normal distributions of represents the Poisson distribution, represents a normal distribution. Therefore, we can determine gα and is the parameter that affects the noise. In order to obtain the specific data of the parameter that affects the noise, Using k and σ 2 As well as the pixel mean and pixel variance to construct a linear regression model, and obtain k and σ 2 (For details, please refer to the above embodiments, which will not be repeated here).

[0109] In one embodiment, performing linear transformation on the noise parameters and determining the noise pixel distribution interval based on a predetermined noise model, noise distribution and the noise parameters after linear transformation includes:

[0110] A noise distribution model is generated based on the noise model and the noise distribution.

[0111] Specifically, the noise distribution usually refers to whether the noise conforms to the normal (Gaussian) distribution. After the noise model is determined, the noise distribution model can be generated by combining the noise distribution.

[0112] Based on the noise distribution model and the noise parameters after linear transformation, a noise pixel distribution interval is determined.

[0113] Specifically, when the noise distribution model is The noise parameters after linear transformation are: Since the noise distribution model conforms to Poisson distribution and normal distribution, the Poisson distribution and normal distribution are used to Simplify and deduce, and approximate the Poisson distribution to the normal distribution, so Approximately Then Analyze, where k = gα, therefore,

[0114] To simplify further, first look at because, After approximation It can be seen as in u * The fluctuation of the basis is approximately (Here is a derivation based on distribution approximation and variable relationship).

[0115] Look again This part undergoes a series of merging and approximation during the derivation process (based on the properties of the normal distribution and the parameter relationship defined above), and can finally be expressed as related form, and the overall distribution is similar to the previous The parts together constitute approximately The distribution of N[f(x * ), f(x * )] The derivation process here involves a lot of approximation and transformation techniques in probability theory and statistics, mainly to express the complex noise model and the transformed distribution in a form that is easier to analyze and understand.

[0116] With k = gα, Considering the Poisson distribution as a normal distribution with fixed mean and variance, and simplifying the above process, we can get:

[0117]

[0118] N[f(x * ), f(x * )] can be the noise pixel distribution interval.

[0119] In one embodiment, Figure 7 As shown, the denoised pixel value is obtained based on the noisy pixel value in the decoder of the pre-trained model, and the decoder of the pre-trained model is iterated based on the loss value between the denoised pixel value and the reference pixel value to obtain a denoising model, including:

[0120] S602, in response to the decoder of the pre-trained model performing back propagation, randomly sampling from a standard normal distribution interval to obtain a random sampling value.

[0121] S604, based on the random sampling value and the noise pixel distribution interval, back-propagation is performed on the decoder of the pre-trained model using a re-parameterization technique to update the parameters in the decoder of the pre-trained model to obtain a denoising model.

[0122] Among them, the standard normal distribution refers to the interval (0, 1). Reparameterization is a commonly used technique in machine learning and deep learning, mainly used to solve problems in the training process of certain models, especially when random variables and non-differentiable operations are involved.

[0123] Specifically, when training the variational autoencoder model, forward propagation can be performed first, that is, the noise-free image is denoised using the encoder, and then the image denoised by the encoder is denoised. Since the noise-free pixels are sampled from the noise-free pixel distribution interval. Therefore, problems will be encountered during the back propagation process. Back propagation is an important mechanism for calculating gradients in neural networks to update model parameters. However, the sampling operation is a random process, and its results are not differentiable. This means that when calculating the gradient, the error cannot be back-propagated from the output to the input through the conventional chain rule to update the model parameters, resulting in the training process of the entire VAE model cannot proceed normally, that is, the back propagation is interrupted. In order to solve this problem, the "reparameterization" technology is introduced. Specifically, random sampling can be performed from the standard normal distribution interval. Then, based on random sampling and the noise pixel distribution interval, back propagation training is performed using the reparameterization technology.

[0124] In some exemplary embodiments, random sampling can be first performed from a standard normal distribution interval to obtain a random sampling value ε, and then the random sampling value and the noise pixel distribution interval N[f(x * ), f(x * )], thus obtaining:

[0125] In the reparameterized expression, when back propagation is performed, since ε is sampled from the standard normal distribution, it can be regarded as a constant during the back propagation process. This is because when calculating the gradient, the gradient for the constant is (except in special cases, such as in some techniques involving sampling, but here it can be simply understood as a constant). In this way, the interference of the sampling operation on the back propagation is avoided, so that the entire VAE model can calculate the gradient normally and perform back propagation, thereby realizing the update and training of the model parameters. Through the reparameterization technology, without changing the essential function of the model (i.e., sampling from a certain distribution to generate or process data), the problem of interruption of back propagation is solved, so that the VAE model can be effectively trained and optimized.

[0126] In one embodiment, the present disclosure also provides a method for training a noise reduction model, including:

[0127] Obtain multiple images with different exposure values ​​taken in the same scene;

[0128] Based on the multiple images with different exposure values, a pixel mean value at each position is calculated, and a reference pixel value at each position is determined based on the pixel mean value at each position;

[0129] In an encoder of the pre-trained model, a noisy pixel value is obtained based on the predetermined noise distribution and the reference pixel value;

[0130] In the decoder of the pre-trained model, a denoised pixel value is obtained based on the noisy pixel value, and the decoder of the pre-trained model is iterated based on a loss value between the denoised pixel value and the reference pixel value to obtain a denoising model.

[0131] For the specific implementation and limitation of this embodiment, please refer to the above embodiment, which will not be repeated here.

[0132] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0133] Based on the same inventive concept, the embodiment of the present disclosure also provides an image denoising device for implementing the above-mentioned image denoising method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more image denoising device embodiments provided below can refer to the limitations of the image denoising method above, and will not be repeated here.

[0134] In one embodiment, Figure 8 As shown, an image denoising device 700 is provided, comprising a data acquisition module 702 and a model processing module 704, wherein:

[0135] A data acquisition module 702 is used to acquire an image to be denoised;

[0136] The model processing module 704 is used to input the pixel data in the image to be denoised into the decoder of the pre-trained denoising model, and output the denoised image through the decoder in the denoising model; wherein the training process of the denoising model includes: calculating the reference pixel value based on a plurality of images with different exposure values ​​of the same scene taken in advance, obtaining the noisy pixel value in the encoder of the pre-trained model using the predetermined noise distribution and the reference pixel value, obtaining the denoised pixel value based on the noisy pixel value in the decoder of the pre-trained model, and iterating the decoder of the pre-trained model based on the loss value between the denoised pixel value and the reference pixel value to obtain the denoising model.

[0137] In one embodiment of the device, the device further comprises:

[0138] An image acquisition module is used to acquire multiple images with different exposure values ​​taken in the same scene;

[0139] A pixel processing module, configured to calculate a pixel mean value at each position based on the plurality of images with different exposure values, and determine a reference pixel value at each position based on the pixel mean value at each position;

[0140] A noise adding module, configured to obtain a noisy pixel value based on the predetermined noise distribution and the reference pixel value in an encoder of a pre-trained model;

[0141] The model training module is used to obtain a denoised pixel value based on the noisy pixel value in the decoder of the pre-trained model, and iterate the decoder of the pre-trained model based on the loss value between the denoised pixel value and the reference pixel value to obtain a denoising model.

[0142] In one embodiment of the device, the noise adding module includes:

[0143] The variance calculation module is used to calculate the pixel variance of each position based on the reference pixel value of each position.

[0144] The noise parameter calculation module is used to construct a linear regression model for calculating the noise parameter based on the reference pixel value and the pixel variance, and to calculate the noise parameter using the linear regression model.

[0145] The distribution interval determination module is used to perform linear transformation on the noise parameters and determine the noise pixel distribution interval based on a predetermined noise model, noise distribution and the noise parameters after linear transformation.

[0146] The sampling module is used to perform sampling in the noise pixel distribution interval to obtain the noise pixel value.

[0147] In one embodiment of the device, the device also includes: a noise model generation module, which is used to determine the noise generated by the imaging device in the stages before and after the analog gain stage in the imaging stage; and generate a noise model based on the actual number of photons obtained in the imaging stage, the quantum efficiency, and the noise generated in the stages before and after the analog gain stage, wherein the quantum efficiency characterizes the efficiency of converting photons into electrons.

[0148] In one embodiment of the device, the distribution interval determination module is further used to generate a noise distribution model based on the noise model and the noise distribution; and determine the noise pixel distribution interval based on the noise distribution model and the noise parameters after linear transformation.

[0149] In one embodiment of the device, the model training module is also used to perform back propagation in response to the decoder of the pre-trained model, perform random sampling from a standard normal distribution interval, and obtain random sampling values; based on the random sampling values ​​and the noise pixel distribution interval, use re-parameterization technology to perform back propagation on the decoder of the pre-trained model, update the parameters in the decoder of the pre-trained model, and obtain a denoising model.

[0150] In one embodiment of the device, the device further comprises: a parameter determination module for determining parameters in a linear regression model based on the target photon number obtained in the imaging stage, the actual photon number and the noise model.

[0151] Each module in the above-mentioned image noise reduction device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0152] In one embodiment, Fig. 9 As shown, the present disclosure also provides a training device 800 for a noise reduction model, the device comprising:

[0153] An image acquisition module 802 is used to acquire multiple images with different exposure values ​​taken in the same scene;

[0154] The pixel processing module 804 is used to calculate the pixel mean value of each position based on the multiple images with different exposure values, and determine the reference pixel value of each position based on the pixel mean value of each position;

[0155] A noise adding module 806, configured to obtain a noisy pixel value based on the predetermined noise distribution and the reference pixel value in the encoder of the pre-trained model;

[0156] The model training module 808 is used to obtain a denoised pixel value based on the noisy pixel value in the decoder of the pre-trained model, and iterate the decoder of the pre-trained model based on the loss value between the denoised pixel value and the reference pixel value to obtain a denoising model.

[0157] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store images to be denoised. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image denoising method or a denoising model training method is implemented.

[0158] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.11As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an image denoising method or a denoising model training method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0159] Those skilled in the art will understand that Fig.10 The structure shown in or 11 is merely a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0160] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0161] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0162] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0163] It should be noted that the images to be denoised and the multiple images with different exposure values ​​involved in the present disclosure are all information and data authorized by the user or fully authorized by all parties.

[0164] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided by the present disclosure may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided by the present disclosure may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited thereto.

[0165] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0166] The above-described embodiments only express several implementation methods of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present disclosure, and these all belong to the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the attached claims.

Claims

1. An image denoising method, characterized in that: The method comprises: Obtain an image to be denoised; The pixel data in the image to be denoised is input into the decoder of a pre-trained denoising model, and the denoised image is output via the decoder in the denoising model; wherein the training process of the denoising model includes: calculating a reference pixel value based on a plurality of images with different exposure values ​​of the same scene taken in advance, obtaining a noisy pixel value in the encoder of the pre-trained model using a predetermined noise distribution and the reference pixel value, obtaining a denoised pixel value based on the noisy pixel value in the decoder of the pre-trained model, and iterating the decoder of the pre-trained model based on a loss value between the denoised pixel value and the reference pixel value to obtain a denoising model.

2. The method according to claim 1, characterized in that: The method further comprises: Obtain multiple images with different exposure values ​​taken in the same scene; Based on the multiple images with different exposure values, a pixel mean value at each position is calculated, and a reference pixel value at each position is determined based on the pixel mean value at each position; In an encoder of the pre-trained model, a noisy pixel value is obtained based on the predetermined noise distribution and the reference pixel value; In the decoder of the pre-trained model, a denoised pixel value is obtained based on the noisy pixel value, and the decoder of the pre-trained model is iterated based on a loss value between the denoised pixel value and the reference pixel value to obtain a denoising model.

3. The method according to claim 2, characterized in that The obtaining of the noisy pixel value based on the predetermined noise distribution and the reference pixel value comprises: Based on the reference pixel value at each position, a pixel variance at each position is calculated; Based on the reference pixel value and the pixel variance, a linear regression model for calculating noise parameters is constructed, and the noise parameters are calculated using the linear regression model; Performing a linear transformation on the noise parameters, and determining a noise pixel distribution interval based on a predetermined noise model, noise distribution, and the noise parameters after the linear transformation; Sampling is performed in the noise pixel distribution interval to obtain a noisy pixel value.

4. The method according to claim 3, characterized in that The noise model is determined by: determining noise generated by the imaging device before and after an analog gain stage in an imaging stage; A noise model is generated according to the actual number of photons obtained in the imaging stage, quantum efficiency, and noise generated in stages before and after the simulated gain stage, wherein the quantum efficiency represents the efficiency of converting photons into electrons.

5. The method according to claim 3, characterized in that: The linear transformation of the noise parameters and determining the noise pixel distribution interval based on a predetermined noise model, noise distribution and the noise parameters after the linear transformation include: Based on the noise model and the noise distribution, generate a noise distribution model; Based on the noise distribution model and the noise parameters after linear transformation, a noise pixel distribution interval is determined.

6. The method according to claim 3, characterized in that The method of obtaining a denoised pixel value based on the denoised pixel value in the decoder of the pre-trained model, and iterating the decoder of the pre-trained model based on a loss value between the denoised pixel value and the reference pixel value to obtain a denoising model comprises: In response to the decoder of the pre-trained model performing back propagation, randomly sampling from a standard normal distribution interval to obtain a random sampling value; Based on the random sampling value and the noise pixel distribution interval, the decoder of the pre-trained model is back-propagated using the re-parameterization technology to update the parameters in the decoder of the pre-trained model to obtain a denoising model.

7. The method according to claim 4, characterized in that Before constructing a linear regression model for calculating noise parameters based on the reference pixel value and the pixel variance and calculating the noise parameters using the linear regression model, the method further includes: Parameters in a linear regression model are determined based on the target photon number obtained in the imaging stage, the actual photon number and the noise model.

8. A method for training a denoising model, characterized in that: The method comprises: Obtain multiple images with different exposure values ​​taken in the same scene; Based on the multiple images with different exposure values, a pixel mean value at each position is calculated, and a reference pixel value at each position is determined based on the pixel mean value at each position; In an encoder of the pre-trained model, a noisy pixel value is obtained based on the predetermined noise distribution and the reference pixel value; In the decoder of the pre-trained model, a denoised pixel value is obtained based on the noisy pixel value, and the decoder of the pre-trained model is iterated based on a loss value between the denoised pixel value and the reference pixel value to obtain a denoising model.

9. An image noise reduction device, characterized in that: The device comprises: A data acquisition module, used for acquiring the image to be denoised; A model processing module is used to input the pixel data in the image to be denoised into the decoder of a pre-trained denoising model, and output the denoised image via the decoder in the denoising model; wherein the training process of the denoising model includes: calculating a reference pixel value based on a plurality of images with different exposure values ​​of the same scene taken in advance, applying noise to the reference pixel value using a predetermined noise distribution in the encoder of the pre-trained model to obtain a noisy pixel value, obtaining a denoised pixel value based on the noisy pixel value in the decoder of the pre-trained model, and iterating the decoder of the pre-trained model based on a loss value between the denoised pixel value and the reference pixel value to obtain a denoising model.

10. A training device for a noise reduction model, characterized in that: The device comprises: An image acquisition module is used to acquire multiple images with different exposure values ​​taken in the same scene; A pixel processing module, configured to calculate a pixel mean value at each position based on the plurality of images with different exposure values, and determine a reference pixel value at each position based on the pixel mean value at each position; A noise adding module, configured to obtain a noisy pixel value based on the predetermined noise distribution and the reference pixel value in an encoder of a pre-trained model; The model training module is used to obtain a denoised pixel value based on the noisy pixel value in the decoder of the pre-trained model, and iterate the decoder of the pre-trained model based on the loss value between the denoised pixel value and the reference pixel value to obtain a denoising model.

Images