Image noise reduction method and device, electronic equipment and storage medium

Abstract

The invention relates to the field of image processing, and provides an image noise reduction method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the noise transformation of a first noise image obtained through the processing of a target image sensor, and obtaining a second noise image obeying the noise distribution of a reference image sensor; inputting the second noise image into a noise reduction model for noise reduction processing to obtain a noise reduction image corresponding to the first noise image; the noise reduction model is an image noise reduction model adapted to the reference image sensor. According to the invention, the data of the target image sensor is mapped to the noise distribution of the reference image sensor through noise transformation, and the noise characteristic difference of different image sensors is fully considered, so that the data of different image sensors can be processed by using the noise reduction model adaptive to the reference image sensor. The general image noise reduction effect suitable for various different image sensors is achieved through a model with a small scale.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image noise reduction method, device, electronic device and storage medium. Background Art

[0002] At present, image denoising technology based on deep learning is developing rapidly. The focus of this type of image denoising method is on how to construct a training data set. This is because deep learning technology requires massive amounts of data to drive it.

[0003] In the real world, however, it is very difficult, time-consuming, and labor-intensive to collect a large number of clean-noisy training data pairs. In addition, the noise performance of each image sensor is inconsistent. If training data from multiple image sensors is collected, the workload and difficulty will be doubled. Therefore, in the field of image processing, synthetic data is usually used to generate training data sets.

[0004] Although synthetic data solutions are mature, they only have good noise reduction effects for a certain type of image sensor. In order to save development costs, many application scenarios usually require a universal model for image noise reduction, that is, a model that can have good noise reduction effects for many different image sensors.

[0005] To achieve this goal, the existing approach is to collect noise data from multiple image sensors that need to be processed to obtain a training data set and then train the model. However, there is a problem with this. Due to differences in hardware design and modules, different image sensors have huge differences in noise distribution. Using such data to train a model will ultimately produce a mediocre effect.

[0006] In order to improve the model effect, existing methods usually use more complex models. However, in scenarios with limited hardware and high performance requirements, the application of large models cannot be supported, so the only option is to trade off the effect.

[0007] In summary, the urgent problem that needs to be solved at present is how to achieve a universal image denoising effect applicable to a variety of different image sensors through a smaller model. Summary of the invention

[0008] The present application provides an image denoising method, device, electronic device and storage medium to solve the problem in the prior art of how to achieve a universal image denoising effect applicable to a variety of different image sensors through a smaller-scale model.

[0009] The present application provides an image noise reduction method, comprising: Performing noise transformation on a first noise image obtained by processing the target image sensor to obtain a second noise image that obeys the noise distribution of the reference image sensor; The second noise image is input into a noise reduction model for noise reduction processing to obtain a noise reduction image corresponding to the first noise image; the noise reduction model is an image noise reduction model adapted for the reference image sensor.

[0010] According to an image denoising method provided by the present application, the first noise image processed by the target image sensor is subjected to noise transformation to obtain a second noise image that obeys the noise distribution of the reference image sensor, including: Acquire a readout noise corresponding to the target image sensor and a readout noise corresponding to the reference image sensor; Removing readout noise corresponding to the target image sensor in the first noisy image to obtain first noisy image data that obeys the Poisson noise distribution of the target image sensor; converting first noisy image data that obeys the Poisson noise distribution of the target image sensor into second noisy image data that obeys the Poisson noise distribution of the reference image sensor; The readout noise corresponding to the reference image sensor is added to the second noisy image data to obtain the second noisy image.

[0011] According to an image denoising method provided by the present application, the reference image sensor corresponds to a plurality of different readout noises; the second noise image is input into a noise reduction model for noise reduction processing to obtain a noise reduction image corresponding to the first noise image, comprising: Inputting a plurality of second noise images to which a plurality of different readout noises corresponding to the reference image sensor are added respectively into the noise reduction model for noise reduction processing, thereby obtaining a plurality of intermediate noise reduction images output by the noise reduction model; Performing mean processing on the plurality of intermediate denoised images to obtain a denoised image corresponding to the first noisy image.

[0012] According to an image noise reduction method provided by the present application, the readout noise corresponding to the target image sensor is at least one black frame processed by the target image sensor under the sensitivity parameter of the first noise image; and / or,

[0013] The readout noise corresponding to the reference image sensor is at least one black frame processed by the reference image sensor under the sensitivity parameter of the first noise image.

[0014] According to an image denoising method provided by the present application, the first noisy image data obeying the Poisson noise distribution of the target image sensor is converted into the second noisy image data obeying the Poisson noise distribution of the reference image sensor, comprising: Acquire an incident noise parameter corresponding to the target image sensor and an incident noise parameter corresponding to the reference image sensor under the sensitivity parameter of the first noise image; Dividing the incident noise parameter corresponding to the reference image sensor under the sensitivity parameter of the first noise image by the incident noise parameter corresponding to the target image sensor to obtain a conversion coefficient representing a conversion relationship between the Poisson noise distribution of the target image sensor and the Poisson noise distribution of the reference image sensor; The first noisy image data is converted into the second noisy image data using the conversion coefficient.

[0015] According to an image denoising method provided by the present application, the training process of the denoising model includes: Acquire a clean reference image and an incident noise parameter and a readout noise of the reference image sensor under a test sensitivity; Using the incident noise parameter of the reference image sensor at the test sensitivity, synthesizing the readout noise of each pixel in the clean reference image to obtain the incident noise of the clean reference image; Adding the readout noise of the reference image sensor at the test sensitivity and the incident noise of the clean reference image to the clean reference image to obtain a noisy reference image corresponding to the clean reference image; The initial denoising model is optimized and trained using a training data pair consisting of the clean reference image and a noisy reference image corresponding to the clean reference image to obtain an optimized denoising model.

[0016] According to an image noise reduction method provided by the present application, the image sensor includes any one of the reference image sensor and the target image sensor, and the noise parameter includes any one of a readout noise and an incident noise parameter; The process of acquiring the noise parameters of the image sensor includes: Determine a first calibrated sensitivity and a second calibrated sensitivity adjacent to a target sensitivity parameter, wherein the target sensitivity parameter is between the first calibrated sensitivity and the second calibrated sensitivity, and the target sensitivity parameter includes a sensitivity parameter of a first noise image or a test sensitivity selected during noise reduction model training; Acquire a first calibrated noise parameter of the image sensor at the first calibrated sensitivity and a second calibrated noise parameter of the image sensor at the second calibrated sensitivity; The noise parameter of the image sensor at the target sensitivity parameter is determined according to a first calibrated noise parameter of the image sensor at the first calibrated sensitivity and a second calibrated noise parameter of the image sensor at the second calibrated sensitivity.

[0017] The present application provides an image noise reduction device, comprising: A noise transformation module, used for performing noise transformation on a first noise image obtained by processing the target image sensor to obtain a second noise image that complies with the noise distribution of the reference image sensor; An image denoising module is used to input the second noisy image into a denoising model for denoising to obtain a denoised image corresponding to the first noisy image; the denoising model is an image denoising model adapted for the reference image sensor.

[0018] The present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the image denoising method described above is implemented.

[0019] The present application also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the image denoising method described in any one of the above is implemented.

[0020] The image denoising method, device, electronic device and storage medium provided by the present application perform noise transformation on the first noise image obtained by processing the target image sensor to obtain a second noise image that obeys the noise distribution of the reference image sensor; the second noise image is input into the noise reduction model for noise reduction processing to obtain a noise reduction image corresponding to the first noise image; the noise reduction model is an image noise reduction model adapted for the reference image sensor. The present application maps the data of the target image sensor to the noise distribution of the reference image sensor through noise transformation, fully considering the differences in noise characteristics of different image sensors, so that the noise reduction model adapted for the reference image sensor can process the data of different image sensors, and achieve a universal image noise reduction effect applicable to a variety of different image sensors through a smaller-scale model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 It is a flowchart of the image denoising method provided by this application.

[0023] Figure 2 It is a structural schematic diagram of the image noise reduction device provided by this application.

[0024] Figure 3It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] It should be noted that in the description of the embodiments of the present application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The orientation or position relationship indicated by the terms "upper", "lower" and the like is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. Unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or it can be a connection between the two elements. For those skilled in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0027] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0028] Combine the following Figure 1-Figure 3 The present invention describes an image noise reduction method, device, electronic device and storage medium.

[0029] Specifically, the present application provides an image noise reduction method, referring to Figure 1 , Figure 1 It is a flowchart of the image denoising method provided by this application.

[0030] The image noise reduction method provided in the embodiment of the present application includes: Step 100: Perform noise transformation on a first noise image obtained by processing a target image sensor to obtain a second noise image that complies with the noise distribution of a reference image sensor.

[0031] In the embodiment of the present application, noise transformation can be understood as a process of converting or adjusting the noise characteristics of an image that obeys a target image sensor to the noise characteristics that obey a reference image sensor.

[0032] In image processing, different sensors or devices may have different noise distributions and characteristics. Through noise transformation, the original noise characteristics of an image can be adjusted to make the image conform to the noise characteristics of the new image sensor, so that the first noise image is consistent with other noise images that obey the reference image sensor in noise distribution or noise characteristics, thereby making it easier to perform unified processing in subsequent data processing or noise reduction processes to ensure the consistency of the noise or noise characteristics of the image data.

[0033] When step 100 is implemented, a first noise image processed by a target image sensor may be obtained, wherein the target image sensor refers to a sensor involved in an image that needs to be processed by image noise transformation, and the first noise image may be acquired by the target image sensor. In this embodiment, the target image sensor may not have an adapted noise reduction model, and the number of the target image sensors may be single or multiple.

[0034] The first noise image is a noise image that complies with the noise characteristics or noise distribution of the target image sensor. Then, the first noise image processed by the target image sensor is subjected to noise transformation to obtain a second noise image that complies with the noise distribution of the reference image sensor.

[0035] The first noise image and the second noise image may both be noise images in the raw domain. In other examples, the noise images may also be noise images in other domains, such as the RGB domain.

[0036] For example, the reference image sensor and each target image sensor may be subjected to noise calibration to obtain noise calibration parameters, and the noise calibration parameters may be used to perform noise transformation on the first noise image captured by the target image sensor to obtain a second noise image that obeys the noise distribution of the reference image sensor. In some examples, the above noise calibration process may be performed with reference to the prior art.

[0037] In this way, through the new processing method of noise transformation, the noise images obtained by various sensors are converted into a unified noise image that obeys the reference image sensor, which facilitates the unified processing of training data and actual denoised image data, and ensures the consistency of noise distribution.

[0038] It is also convenient to use the image denoising model adapted for the reference image sensor to process the image captured by the target image sensor in the future, thereby achieving a universal denoising processing capability among different sensors.

[0039] Step 200: Input the second noise image into a noise reduction model for noise reduction processing to obtain a noise reduction image corresponding to the first noise image.

[0040] In an embodiment of the present application, the denoising model is an image denoising model adapted to a reference image sensor, wherein the image denoising model adapted to the reference image sensor may be a denoising model that has been trained based on the reference image sensor.

[0041] The denoising model can be obtained by training with a training data pair, the training data pair including a noise sample image / noisy reference image and its corresponding original image / clean reference image. In this embodiment, the training data pair used in the training of the denoising model includes a clean reference image and a noisy reference image that obeys the noise distribution of the reference image sensor, thereby using the training data pair to train an image denoising model adapted to the reference image sensor.

[0042] After acquiring the second noisy image, the second noisy image is input into the denoising model for denoising, and a denoised image corresponding to the first noisy image is obtained. The denoising process can be understood as reducing the noise introduced into the image by the sensor, acquisition device and / or transmission process. Noise usually causes blurring or distortion of image details, and the denoising process can restore the clarity and details of the image, thereby improving the visual quality of the image.

[0043] The image denoising method provided in the embodiment of the present application performs noise transformation on the first noise image obtained by processing the target image sensor to obtain a second noise image that obeys the noise distribution of the reference image sensor; the second noise image is input into the noise reduction model for noise reduction processing to obtain a noise reduction image corresponding to the first noise image; the noise reduction model is an image noise reduction model adapted for the reference image sensor. The present application maps the data of different image sensors to the same noise distribution through noise transformation, that is, the data of the target image sensor can be mapped to the noise distribution of the reference image sensor, fully considering the differences in noise characteristics of different image sensors, so that the same noise reduction model (using the noise reduction model adapted for the reference image sensor) can process data from different image sensors, and achieve a general image noise reduction effect applicable to a variety of different image sensors through a smaller-scale specialized training model.

[0044] Based on the above embodiment, the step of performing noise transformation on the first noise image obtained by processing the target image sensor to obtain a second noise image that obeys the noise distribution of the reference image sensor includes: Step 110, obtaining a readout noise corresponding to the target image sensor and a readout noise corresponding to the reference image sensor; Step 111, removing the readout noise corresponding to the target image sensor in the first noise image to obtain first noisy image data obeying the Poisson noise distribution of the target image sensor; Step 112, converting the first noisy image data that obeys the Poisson noise distribution of the target image sensor into the second noisy image data that obeys the Poisson noise distribution of the reference image sensor; Step 113: Add the readout noise corresponding to the reference image sensor to the second noisy image data to obtain the second noisy image.

[0045] It should be noted that for any image sensor, its noise model includes incident noise and readout noise, that is, Ntotal=Nshot+Nread, where Ntotal is the noise model of the image sensor, Nshot is the incident noise of the image sensor, and Nread is the readout noise of the image sensor.

[0046] For example, at least one of the incident noise and the readout noise may be a noise calibration parameter obtained through noise calibration.

[0047] Alternatively, for example, at least one of the incident noise and the readout noise may be obtained by noise calibration parameter processing.

[0048] Alternatively, the incident noise may be obtained by processing a noise calibration parameter, and the readout noise may be one of the noise calibration parameters obtained by the noise calibration.

[0049] In this embodiment, the readout noise corresponding to the target sensor, i.e., the first readout noise, and the readout noise corresponding to the reference image sensor, i.e., the second readout noise, can be read out from the device memory, or the readout noise can also be obtained by looking up a table.

[0050] For example, the sensitivity parameter of the first noise image that currently needs to undergo noise transformation can be obtained, and then the first readout noise corresponding to the target image sensor under the sensitivity parameter of the first noise image and the second readout noise corresponding to the reference image sensor under the sensitivity parameter can be obtained by table lookup.

[0051] The above sensitivity parameter refers to the ISO value, which is a parameter used to measure the sensitivity of the image sensor to light. Different ISO values ​​may correspond to different readout noises of the image sensor.

[0052] The above-mentioned incident noise is noise caused by incident light, and the incident noise follows a Poisson distribution. The incident noise can be estimated by collecting bright image data, such as raw domain image data, and analyzing the variance of its brightness value. The noise calibration parameters related to the incident noise can include key parameters similar to the incident noise parameters.

[0053] The above-mentioned incident noise parameter is a key parameter of the Poisson distribution and can be obtained through noise calibration. It can be the slope k value of the curve formed by the pixel value sequence formed by all pixels of the same brightness in the image collected by the image sensor and the variance of the sequence.

[0054] The slope k value can be used to characterize the linear relationship between the brightness value and the noise intensity of the image, and based on this, the image conversion relationship between the Poisson noise distributions of different image sensors can also be obtained.

[0055] The read noise is the noise caused by the read circuit of the sensor. In some schemes, the read noise conforms to the Gaussian distribution. In other optional schemes of the present application, considering that the read noise is relatively complex, directly using the Gaussian distribution is too simple and cannot well reflect the actual noise distribution. Therefore, in this scheme, the read noise is not modeled using Gaussian noise, but the collected black frame is directly used as the read noise. Therefore, by directly using the collected black frame data as the read noise, it can be closer to the actual noise distribution than the Gaussian distribution, reflecting the complexity of the read noise.

[0056] It should be noted that the above readout noise is at least one black frame acquired. For example, the readout noise may be multiple black frames acquired by an image sensor at different sensitivities.

[0057] The black frame is an image taken in the absence of light, so it only contains the readout noise of the image sensor and does not contain the noise caused by the incident light. That is, in the black frame, due to the absence of light, the Poisson noise caused by the incident light is greatly reduced, while the electronic noise caused by the readout circuit dominates.

[0058] In one embodiment, it is described above that the acquired black frame can be directly used as the readout noise. Correspondingly, the readout noise corresponding to the target image sensor acquired above can be at least one black frame processed by the target image sensor under the sensitivity parameters of the first noise image.

[0059] and / or,

[0060] The readout noise corresponding to the reference image sensor may be at least one black frame obtained by processing the reference image sensor under the sensitivity parameter of the first noise image.

[0061] In the above embodiment, the black frame acquired by the image sensor is used as the readout noise or noise calibration parameter to participate in the image data noise transformation between different image sensors, which can improve the accuracy of the readout noise and data processing. Compared with the Gaussian distribution, it can be closer to the real noise distribution and reflect the complexity of the readout noise. In the case of using black frames to process the noise in the training data, the noise reduction effect of the noise reduction model can also be indirectly improved.

[0062] In order to better illustrate the noise conversion process in the above steps, the above process is described below by taking sensorY as the target image sensor and sensorX as the reference image sensor.

[0063] When the first noise image inputY of the target image sensor sensorY needs to be noise-converted or denoised, the readout noise black_Y corresponding to the target image sensor sensorY and the readout noise black_X corresponding to the reference image sensor sensorX can be obtained. Then, the readout noise black_Y corresponding to the target image sensor sensorY in the first noise image inputY is removed to obtain the first noisy image data that obeys the Poisson noise distribution of the target image sensor sensorY.

[0064] It can be understood that, after the above processing, the first noisy image data is image data acquired based on the target image sensor sensorY, from which incident noise is removed and which only contains the readout noise of the target image sensor sensorY.

[0065] For example, when the first readout noise corresponding to the target image sensor is obtained, the first noisy image can be processed by a specific algorithm or filter to remove the first readout noise therein, so as to reduce or eliminate the influence introduced by the first readout noise and obtain the first noisy image data.

[0066] Subsequently, the first noisy image data that complies with the Poisson noise distribution of the target image sensor sensorY may be further converted into second noisy image data that complies with the Poisson noise distribution of the reference image sensor sensorX.

[0067] The second noisy image data is obtained by converting the first noisy image data, and can be understood as image data obtained by removing incident noise and including readout noise of the reference image sensor. The second noisy image data is actually obtained by collecting based on the target image sensor sensorY.

[0068] In one embodiment, a conversion coefficient may be obtained; and then, the first noisy image data is converted into the second noisy image data using the conversion coefficient.

[0069] It should be noted that the above conversion coefficient represents the conversion relationship between the Poisson noise distribution of the target image sensor and the Poisson noise distribution of the reference image sensor, and the above conversion coefficient can be obtained by noise calibration or calculation of the incident noise parameters of each image sensor.

[0070] It should be noted that the above-mentioned incident noise parameter may be related to the sensitivity parameter of the noisy image, that is, different sensitivity parameters may correspond to different incident noise parameters of each image sensor.

[0071] The first noisy image has had its readout noise removed. If a conversion coefficient is used to convert its incident noise into second noisy image data that obeys the Poisson noise distribution of a new reference image sensor, even though the noisy image data is still acquired based on the target image sensor, its incident noise already conforms to the noise characteristics of the reference image sensor in the distribution of the image noise model.

[0072] Therefore, this embodiment provides a conversion scheme for the incident noise for converting the first noise image into a second noise image that obeys the new image sensor noise distribution through noise transformation, provides a technical basis for the unified processing of noise data, and ensures the consistency of the image in obeying the noise distribution of the reference image sensor.

[0073] In one embodiment, the process of obtaining the conversion coefficient may include: A first incident noise parameter corresponding to the target image sensor and a second incident noise parameter corresponding to the reference image sensor are acquired, and then parameter operation is performed on the first incident noise parameter and the second incident noise parameter to obtain a conversion coefficient.

[0074] The above parameter operation may be a division operation, that is, the second incident noise parameter may be divided by the first incident noise parameter to obtain a conversion coefficient.

[0075] In one embodiment, the conversion relationship between the Poisson noise distributions of different sensors can also be calculated in advance, that is, the conversion coefficient between the target image sensor and the reference image sensor can be calculated in advance. When performing the Poisson noise distribution conversion later, the conversion coefficient required for the current conversion can be determined by looking up a table based on the given target image sensor and reference image sensor, and even in combination with the sensitivity parameters.

[0076] Corresponding to the process of converting the first noisy image data obeying the Poisson noise distribution of the target image sensor into the second noisy image data obeying the Poisson noise distribution of the reference image sensor in step 112, the process may include: An incident noise parameter (ie, a first incident noise parameter) corresponding to the target image sensor and an incident noise parameter (ie, a second incident noise parameter) corresponding to the reference image sensor under the sensitivity parameter of the first noise image are obtained.

[0077] The incident noise parameter corresponding to the reference image sensor under the sensitivity parameter of the first noise image is divided by the incident noise parameter corresponding to the target image sensor to obtain a conversion coefficient representing the conversion relationship between the Poisson noise distribution of the target image sensor and the Poisson noise distribution of the reference image sensor.

[0078] The first noisy image data is converted into the second noisy image data using the conversion coefficient.

[0079] The following description will be continued with an example using the target image sensor sensorY and the reference image sensor sensorX.

[0080] An incident noise parameter k_Y (first incident noise parameter) corresponding to the target image sensor sensorY and an incident noise parameter k_X (second incident noise parameter) corresponding to the reference image sensor sensorX under the sensitivity parameter of the first noise image inputY may be obtained.

[0081] The sensitivity parameter refers to the ISO value, which is a parameter used to measure the sensitivity of the image sensor to light. The first incident noise parameter and the second incident noise parameter may be key incident noise parameters related to the incident noise in the noise model mentioned above.

[0082] In some embodiments, the incident noise parameter and readout noise mentioned above may be used as noise parameters included in each image sensor.

[0083] On this basis, the whole scheme obtains the noise parameters corresponding to the reference image sensor and the target image sensor respectively, for example, the implementation process of obtaining the noise parameters of each image sensor under the sensitivity parameters of the first noise image may include: When the first noise image inputY of the target image sensor sensorY needs to be denoised or noise transformed, the sensitivity (i.e., ISO value) corresponding to the first noise image inputY is obtained, and the noise parameters corresponding to the ISO value are determined, including the incident noise parameter k_Y and the black frame black_Y, where k_Y may be the slope of the Poisson distribution obeyed by the incident noise of sensorY, and the black frame black_Y is the readout noise of sensorY.

[0084] For the ISO value, noise parameters corresponding to the reference image sensor sensorX may also be obtained, including an incident noise parameter k_X and a black frame black_X, wherein k_X is the slope of the Poisson distribution obeyed by the incident noise of sensorX, and the black frame black_X is the readout noise of sensorX.

[0085] In some embodiments, the first incident noise parameter may be understood as the slope of the Poisson distribution obeyed by the incident noise of the target image sensor; the second incident noise parameter may be understood as the slope of the Poisson distribution obeyed by the incident noise of the reference image sensor.

[0086] By dividing the incident noise parameter k_X corresponding to the reference image sensor sensorX under the sensitivity parameter of the first noise image inputY by the incident noise parameter k_Y corresponding to the target image sensor sensorY, a conversion coefficient representing the conversion relationship between the Poisson noise distribution of the target image sensor sensorY and the Poisson noise distribution of the reference image sensor sensorX can be obtained, such as (k_X / k_Y), and then the first noisy image data is converted into the second noisy image data using the conversion coefficient.

[0087] In one embodiment, the process of converting the first noisy image data into the second noisy image data using the conversion coefficient may include: The product of the first noisy image data and the conversion coefficient is obtained, where the product is second noisy image data that complies with the Poisson noise distribution of the reference image sensor.

[0088] In this embodiment, a conversion coefficient is used as a process parameter for mapping the noise data of the reference image sensor to the noise distribution of the reference image sensor. The use of the conversion coefficient can better characterize the conversion relationship from the noise distribution of the reference image sensor to the noise distribution of the target image sensor in the incident noise distribution, and help to quickly achieve noise transformation on the incident noise.

[0089] Finally, the readout noise corresponding to the reference image sensor sensorX is added to the second noisy image data to obtain a second noisy image.

[0090] In order to better illustrate the above noise conversion process, the following still describes the process of converting the first noise image inputY that obeys the sensorY noise distribution into the second noise image inputY_new that obeys the sensorX noise distribution in combination with the above example.

[0091] Based on this, the above noise transformation process can be expressed according to the following formula: inputY_new=(inputY–black_Y)×(k_X) / (k_Y)+black_X; Wherein, inputY_new is the second noise image, inputY is the first noise image, black_Y is the readout noise corresponding to the image sensor sensorY, for example, it can be a black frame or any frame of multiple black frames acquired by the image sensor sensorY under the sensitivity parameter of the first noise image inputY, black_X is the readout noise corresponding to the image sensor sensorX, for example, it can be a black frame or any frame of multiple black frames acquired by the image sensor sensorX under the sensitivity parameter of the first noise image inputY. (k_X) / (k_Y) can be a noise conversion coefficient, where (k_X) is the incident noise parameter corresponding to the image sensor sensorX, and (k_Y) is the incident noise parameter corresponding to the image sensor sensorY.

[0092] Combining the above noise transformation process and noise transformation formula, the overall noise transformation principle is as follows: The first noisy image inputY of the image sensor sensorY is subtracted from the readout noise black_Y corresponding to the image sensor Y. This operation can be understood as removing the noise inherent in the image sensor sensorY itself, so that the remaining image data only contains the incident noise, that is, (inputY–black_Y) is the first noisy image data that obeys the Poisson noise distribution of inputY.

[0093] Then, (k_X) / (k_Y) represents the conversion of two Poisson distributions, that is, the conversion coefficient. The operation is to convert the first noisy image data into the second noisy image data using the conversion coefficient, where k_Y is the incident noise parameter / first incident noise parameter corresponding to the target image sensor, and k_X is the incident noise parameter / second incident noise parameter corresponding to the reference image sensor.

[0094] The second incident noise parameter can be divided by the first incident noise parameter to obtain a conversion coefficient, and then the conversion coefficient is multiplied by the first noisy image data obtained that obeys the Poisson noise distribution of the image sensor sensorY. The product is the second noisy image obtained that obeys the Poisson noise distribution of the image sensor sensorX.

[0095] Finally, the second noisy image that obeys the Poisson noise distribution of the image sensor sensorX is added with the readout noise black_X of sensorX to obtain the second noisy image inputY_new, so that the noisy image can be restored to the noise model with the image sensor sensorX added, that is, the noisy image that obeys the readout noise of the image sensor sensorX and the incident noise distribution.

[0096] This operation uses the incident noise parameters and the readout noise to ensure that the resulting second noise image not only conforms to the characteristics of sensorX in terms of brightness value, but also conforms to sensorX in terms of noise.

[0097] The embodiment of the present application provides an optional implementation scheme for noise transformation, which generally utilizes the readout noise of each image sensor, and can be stripped from the noisy image to obtain noisy image data with the readout noise removed and only obeying the incident noise distribution, and then using the incident noise parameters of each image sensor, the conversion relationship of the incident noise distribution when different sensors perform noise transformation is obtained, and the noisy image obeying the noise model distribution of the target image sensor is converted into a noisy image obeying the incident noise and readout noise model distribution of the reference image sensor, thereby mapping the data of different image sensors to the same noise distribution (the noise distribution of the reference image sensor). This method takes into account the differences in noise characteristics of different image sensors, so that the same noise reduction model, that is, the image noise reduction model adapted to the reference image sensor, can process data from different image sensors, and has good versatility.

[0098] In some examples, the noise transformation process is described above. For the noise parameter calibration involved therein, the following process can be referred to to perform noise calibration on each image sensor (including the target image sensor and the reference image sensor) that may be involved in the noise reduction process.

[0099] Before noise calibration, several color cards, light boxes, and image sensors required for image acquisition can be prepared in advance. The color card is, for example, a 24-color card, and the light box is, for example, a D65 light box. The color card can be fixed in the light box, the illumination of the light box is maintained at about 400 lux, the illumination is kept uniform, the lens is fixed, and the distance between the lens and the color card is adjusted so that the captured color card covers about 1 / 2 of the entire imaging screen.

[0100] In addition, the calibrated sensitivity to be collected can also be set. For example, multiple calibrated sensitivities can be set between ISO100 and ISO3200, and the calibrated sensitivity can be adjusted until the maximum value of the analog gain Again of the image sensor is reached. In some examples, it can also be the sensitivity value corresponding to the maximum value of the analog gain Again and the digital gain Dgain set to 1.

[0101] When collecting each calibrated sensitivity, it is necessary to adjust the exposure time so as to keep the brightness of the image collected at each calibrated sensitivity consistent or substantially consistent.

[0102] For each calibrated sensitivity ISO value, N frames of bright image data can be collected as data for calibrating incident noise. At the same time, for each calibrated sensitivity ISO value, the external light source needs to be turned off and the lens is completely covered, and N frames of black image data are collected as black frames / readout noise of the image sensor. For example, the above image data can be image data in the raw domain, and N can be an integer between 30-100, such as N=60.

[0103] By averaging N frames of bright image data, a frame of clean image data can be obtained, and then a series of brightness values ​​can be selected from the clean image data.

[0104] Assuming that the bit width of the image data is 10 bits, the value is taken at intervals of 50 from 100 to 1023. With the brightness value as brightness, all pixel points with the pixel value of brightness are found in the image data, and their positions are recorded. The pixel values ​​at these positions can be obtained in the N frames of bright image data, thereby obtaining a pixel value sequence.

[0105] With brightness as the mean, the variance var of the sequence can be calculated. In this way, each brightness value brightness can get a corresponding variance value. With brightness as the horizontal coordinate and the variance var of the sequence as the vertical coordinate, a curve var=k×brightness+b can be fitted, where the slope k is the key parameter of the Poisson distribution and can be used as the incident noise parameter.

[0106] After noise calibration of all involved image sensors, a k value (incident noise parameter) and corresponding N black frames (readout noise) at each calibrated ISO value can be collected for each image sensor. Noise parameters can be obtained, noise reduction models can be trained, and noise transformation can be performed accordingly.

[0107] The above embodiments can illustrate how to perform noise conversion and subsequent noise reduction processing when the image sensor corresponds to a single readout noise, or when the image sensor corresponds to a single readout noise under the sensitivity parameters of the first noise image.

[0108] Based on the above embodiment, it is considered that the reference image sensor corresponds to a plurality of different readout noises, for example, a plurality of black frames collected by the reference image sensor correspond to a plurality of different readout noises.

[0109] In the above step 200, the process of inputting the second noise image into the noise reduction model for noise reduction processing to obtain the noise reduction image corresponding to the first noise image may include: Step 210, inputting a plurality of second noise images to which a plurality of different readout noises corresponding to the reference image sensor are added respectively into the noise reduction model for noise reduction processing, and obtaining a plurality of intermediate noise reduction images output by the noise reduction model; Step 211, performing mean processing on the plurality of intermediate denoised images to obtain a denoised image corresponding to the first noisy image.

[0110] It should be noted that, considering that the readout noise may be randomly selected from multiple readout noises when it is acquired once, the randomly selected black frame may not be used when training the noise reduction model adapted to the reference image sensor, which may cause the noise reduction result to be inconsistent with the actual effect.

[0111] To this end, different readout noises / black frames corresponding to the reference image can be randomly selected to obtain multiple second noise images with different readout noises / black frames added. That is, there can be multiple second noise images, and different second readout noises can be added to the multiple second noise images.

[0112] Then, the noise reduction model adapted to the reference image sensor is used to reduce the noise of multiple second noise images to obtain multiple intermediate noise reduction images, and then the multiple intermediate noise reduction images as the noise reduction results are averaged, which can effectively average out the random errors in the noise reduction results, thereby effectively solving the problem that the black frames adapted for training are inconsistent with the randomly selected readout noise in the actual noise reduction process, resulting in the difference between the noise reduction results and the actual effects, thereby improving the final noise reduction effect and helping to improve the stability and accuracy of the final result.

[0113] The embodiment of the present application generates a series of slightly different input data to the noise reduction model by selecting different black frames, which is equivalent to providing the noise reduction model with multiple slightly different observation perspectives, introducing data diversity, and thus improving the accuracy of image noise reduction.

[0114] Based on the above embodiment, the training process of the denoising model includes: Step 120, obtaining a clean reference image and an incident noise parameter and a readout noise of the reference image sensor under a test sensitivity; Step 121, using the incident noise parameter of the reference image sensor at the test sensitivity, synthesize the readout noise of each pixel in the clean reference image to obtain the incident noise of the clean reference image; Step 122, adding the readout noise of the reference image sensor at the test sensitivity and the incident noise of the clean reference image to the clean reference image to obtain a noisy reference image corresponding to the clean reference image; Step 123, optimizing and training the initial denoising model using the training data pair consisting of the clean reference image and the noisy reference image corresponding to the clean reference image, to obtain an optimized denoising model.

[0115] In the embodiment of the present application, the clean reference image refers to clean training data without noise or with less noise. For example, the clean reference image is a clean image obtained through real acquisition, namely GT (Ground Truth). Further, the clean reference image can be clean raw data.

[0116] It is understood that raw data refers to unprocessed image data directly obtained from a digital camera or other image acquisition device. The image data contains the original image information captured by the sensor without any form of compression or processing, retaining all the details and original pixel values ​​captured by the sensor. Clean raw data refers to raw image data that has been calibrated, preprocessed, or some types of noise removed.

[0117] In addition, the incident noise parameter and the readout noise of the reference image sensor under the test sensitivity can also be obtained. The test sensitivity can be obtained randomly, and the test sensitivity can be recorded as iso_current.

[0118] It should be noted that after noise calibration is performed on the reference image sensor to obtain noise calibration parameters, the incident noise parameter k_current and readout noise black_current corresponding to the reference image sensor at any test sensitivity iso_current calibrated by the reference image sensor can be obtained through the collected noise calibration parameters.

[0119] Furthermore, when the readout noise of each pixel in the clean reference image is synthesized using the incident noise parameter of the reference image sensor at the test sensitivity to obtain the incident noise of the clean reference image, the variance of each pixel can be obtained through the incident noise parameter k_current×GT.

[0120] Assume that the pixel position in the clean reference image GT is (i, j), and the variance of the point is k_current×GT(i, j), where GT(i, j) is the brightness value of the pixel. Then, a value is randomly obtained using the Gaussian distribution with the variance, which is the incident noise (shot_noise) of the point.

[0121] The incident noise of the clean reference image GT can be synthesized based on the readout noise of all pixels. Then, the readout noise black_current corresponding to the reference image sensor under the test sensitivity iso_current is added to obtain the noise noise of the clean reference image GT. This process can be expressed by the following formula, namely: noise=shot_noise+black_current; Based on this, the noisy reference image input corresponding to the clean reference image can be obtained by adding the noise of the clean reference image GT to the clean reference image GT, that is, it satisfies the following formula: input=GT+noise; In this way, multiple training data pairs (input, GT) can be obtained, for example, multiple training data pairs adapted to the reference image sensor at multiple arbitrary test sensitivities for training the network.

[0122] After determining a plurality of training data pairs (input, GT), a denoising model based on a neural network is trained according to the training data pairs to obtain a denoising model.

[0123] For example, a suitable denoising model architecture can be selected according to task requirements. At the same time, this solution can use any supervised deep learning method to train the denoising model or denoising model. There are no specific requirements for the network training method and the setting of the loss function, as long as the overall training data used is the training data pair adapted to the reference image sensor.

[0124] For example, you can design the model's network structure, number of layers, activation function, etc. During the training process, you can optimize the model parameters by minimizing the difference between the model's predicted value on the training data and the actual clean image (loss function). After training, you can use the test set to evaluate the model's performance.

[0125] The embodiment of the present application only uses the noise data of one image sensor to synthesize the training data pair for training, that is, a small amount of data and a smaller-scale model are used for training to obtain a special training model adapted for the reference image sensor as the actual general denoising model, which is then combined with the noise transformation method to achieve a general denoising effect adapted to different image sensors.

[0126] Compared to collecting large amounts of data from multiple sensors and training them using complex models, training with data from a single image sensor requires less cost and resources, while also reducing the complexity of data collection and processing.

[0127] On the other hand, by using a denoising model adapted to the reference image sensor and using a noise transformation method to convert the noise images of multiple target sensors into noise images that obey the noise distribution of the reference image sensor, it can be applied to the denoising tasks of multiple different image sensors, thereby achieving a wider range of applications in different application scenarios. This kind of specially trained model as a general denoising model has stronger adaptability and can handle new image sensors or changes in image sensor parameters that may appear in the future. Although only the data of one image sensor is used for training, the effect of the general denoising model is close to that of the specially trained model, which means that similar denoising effects can be achieved in practical applications while reducing the dependence on specific sensor data. Therefore, the general denoising model trained using data from a single image sensor has certain advantages in saving costs and resources, simplifying processes, improving versatility and scalability, achieving effects close to those of specially trained models, and reducing model complexity.

[0128] It should be noted that the above embodiments mention processes such as noise transformation and noise reduction model training, all of which involve noise parameters including incident noise parameters and readout noise, which correspond to the reference image sensor or the target image sensor.

[0129] In order to explain the present solution more clearly, the following briefly introduces the acquisition or collection calibration process of the above-mentioned noise parameters.

[0130] First of all, it should be noted that the image sensor involved in the following includes any one of the above-mentioned reference image sensor and the above-mentioned target image sensor, and the noise parameter includes any one of the above-mentioned readout noise and incident noise parameters.

[0131] On this basis, in the noise transformation stage or in the noise reduction model training stage, it is necessary to obtain noise parameters under different sensitivities, and the implementation process is similar, that is, in these processes, the sensitivity parameter of the first noise image or the test sensitivity selected during the noise reduction model training can be set as the target sensitivity. Therefore, the process of obtaining the noise parameters of the image sensor under the target sensitivity includes: Step 130, determining a first calibrated sensitivity and a second calibrated sensitivity adjacent to a target sensitivity parameter, wherein the target sensitivity parameter is located between the first calibrated sensitivity and the second calibrated sensitivity; Step 131, obtaining a first calibration noise parameter of the image sensor at the first calibration sensitivity and a second calibration noise parameter of the image sensor at the second calibration sensitivity; Step 132: Determine the noise parameter of the image sensor at the target sensitivity parameter according to the first calibrated noise parameter of the image sensor at the first calibrated sensitivity and the second calibrated noise parameter of the image sensor at the second calibrated sensitivity.

[0132] That is, this embodiment provides how to obtain the noise parameters of the reference image sensor at the test sensitivity in the noise reduction model training stage, and how to obtain the noise parameters of the reference image sensor and the target image sensor at the sensitivity parameters of the first noise image in the noise transformation stage.

[0133] Taking iso_current as the target sensitivity parameter for example, two calibrated sensitivities during noise calibration can be found around the target sensitivity parameter iso_current. For example, the two closest calibrated sensitivities are the first calibrated sensitivity iso_last and the second calibrated sensitivity iso_next, that is, the target sensitivity parameter iso_current is between the first calibrated sensitivity iso_last and the second calibrated sensitivity iso_next.

[0134] For the first calibrated sensitivity iso_last and the second calibrated sensitivity iso_next, their calibrated noise parameters during noise calibration of the image sensor can be obtained respectively. The first calibrated noise parameters corresponding to the first calibrated sensitivity iso_last include incident noise parameters k_last and readout noise black_last; the second calibrated noise parameters corresponding to the second calibrated sensitivity iso_next include incident noise parameters k_next and readout noise black_next.

[0135] Further according to the first calibrated noise parameter of the image sensor at the first calibrated sensitivity and the second calibrated noise parameter of the image sensor at the second calibrated sensitivity, the noise parameter of the image sensor at the target sensitivity parameter can be determined.

[0136] For example, the noise parameter corresponding to the target sensitivity parameter iso_current can be obtained by weighted summation. Taking the incident noise parameter k_current corresponding to the target sensitivity parameter iso_current as an example, its value is k_current=k_last×w+k_next×(1-w); Among them, w is the weight, and its calculation formula is: w=(iso_current-iso_last) / (iso_next–iso_last); Taking the readout noise / black frame corresponding to the target sensitivity parameter iso_current as an example, the readout noise / black frame corresponding to the target sensitivity parameter iso_current is: black_current=black_last×w+black_next×(1-w); Among them, the weight w is the same as above, black_last is an optional frame among the N black frames corresponding to iso_last or any one of the N read noises corresponding to iso_last, and black_next is an optional frame among the N black frames corresponding to iso_next or any one of the N read noises corresponding to iso_next.

[0137] Thus, the incident noise parameter k_current of the image sensor under the target sensitivity parameter and the readout noise black_current of the image sensor under the target sensitivity parameter can be obtained.

[0138] This embodiment provides a process for obtaining the noise parameters of an image sensor in the noise transformation stage or the noise reduction model training stage, which can effectively model the noise characteristics of the image sensor at different ISO sensitivities, thereby improving the quality and accuracy of image noise reduction.

[0139] Figure 2 is a schematic diagram of the structure of the image noise reduction device provided by the present application, referring to Figure 2 , an embodiment of the present application provides an image denoising device, including a noise transformation module 201 and an image denoising module 202 .

[0140] A noise transformation module 201 is used to perform noise transformation on a first noise image obtained by processing the target image sensor to obtain a second noise image that complies with the noise distribution of the reference image sensor; The image denoising module 202 is used to input the second noisy image into a denoising model for denoising to obtain a denoised image corresponding to the first noisy image; the denoising model is an image denoising model adapted for the reference image sensor.

[0141] In one embodiment, the noise conversion module 201 is specifically used for: Acquire the readout noise corresponding to the target image sensor and the readout noise corresponding to the reference image sensor; remove the readout noise corresponding to the target image sensor in the first noise image to obtain first noisy image data that obeys the Poisson noise distribution of the target image sensor; convert the first noisy image data that obeys the Poisson noise distribution of the target image sensor into second noisy image data that obeys the Poisson noise distribution of the reference image sensor; add the readout noise corresponding to the reference image sensor to the second noisy image data to obtain the second noise image.

[0142] In one embodiment, the reference image sensor corresponds to a plurality of different readout noises, and the image noise reduction module 202 is specifically configured to: A plurality of second noise images to which a plurality of different readout noises corresponding to the reference image sensor are added are input into the noise reduction model for noise reduction processing to obtain a plurality of intermediate noise reduction images output by the noise reduction model; and a mean value processing is performed on the plurality of intermediate noise reduction images to obtain a noise reduction image corresponding to the first noise image.

[0143] In one embodiment, the readout noise corresponding to the target image sensor is at least one black frame processed by the target image sensor under the sensitivity parameter of the first noise image; and / or,

[0144] The readout noise corresponding to the reference image sensor is at least one black frame processed by the reference image sensor under the sensitivity parameter of the first noise image.

[0145] In one embodiment, the noise conversion module 201 is specifically used for: Obtain incident noise parameters corresponding to the target image sensor and incident noise parameters corresponding to the reference image sensor under the sensitivity parameters of the first noise image; divide the incident noise parameters corresponding to the reference image sensor under the sensitivity parameters of the first noise image by the incident noise parameters corresponding to the target image sensor to obtain a conversion coefficient characterizing a conversion relationship between a Poisson noise distribution of the target image sensor and a Poisson noise distribution of the reference image sensor; and use the conversion coefficient to convert the first noisy image data into the second noisy image data.

[0146] In one embodiment, the training process of the denoising model includes: A clean reference image and incident noise parameters and readout noise of the reference image sensor at a test sensitivity are obtained; the readout noise of each pixel in the clean reference image is synthesized using the incident noise parameters of the reference image sensor at the test sensitivity to obtain the incident noise of the clean reference image; the readout noise of the reference image sensor at the test sensitivity and the incident noise of the clean reference image are added to the clean reference image to obtain a noisy reference image corresponding to the clean reference image; an initial denoising model is optimized and trained using a training data pair consisting of the clean reference image and the noisy reference image corresponding to the clean reference image to obtain an optimized denoising model.

[0147] In one embodiment, the image sensor includes any one of the reference image sensor and the target image sensor, and the noise parameter includes any one of a readout noise and an incident noise parameter; and the process of acquiring the noise parameter of the image sensor includes: Determine a first calibrated sensitivity and a second calibrated sensitivity adjacent to a target sensitivity parameter, wherein the target sensitivity parameter is located between the first calibrated sensitivity and the second calibrated sensitivity, and the target sensitivity parameter includes a sensitivity parameter of a first noise image or a test sensitivity selected during noise reduction model training; obtain a first calibrated noise parameter of the image sensor at the first calibrated sensitivity and a second calibrated noise parameter of the image sensor at the second calibrated sensitivity; determine the noise parameter of the image sensor at the target sensitivity parameter according to the first calibrated noise parameter of the image sensor at the first calibrated sensitivity and the second calibrated noise parameter of the image sensor at the second calibrated sensitivity.

[0148] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor (processor) 310 , a communication interface (CommunicationsInterface) 320 , a memory (memory) 330 and a communication bus 340 .

[0149] The processor 310 , the communication interface 320 , and the memory 330 communicate with each other via the communication bus 340 .

[0150] The processor 310 may call the logic instructions in the memory 330 to execute all the steps of the above-mentioned image noise reduction method embodiment. For example, the method may include: Performing noise transformation on a first noise image obtained by processing the target image sensor to obtain a second noise image that obeys the noise distribution of the reference image sensor; The second noise image is input into a noise reduction model for noise reduction processing to obtain a noise reduction image corresponding to the first noise image; the noise reduction model is an image noise reduction model adapted for the reference image sensor.

[0151] In addition, the logic instructions in the memory 330 described above may be implemented in the form of software functional units and when sold or used as independent products, may be stored in a computer-readable storage medium.

[0152] Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or the part that contributes to the prior art, or the part of the technical solution. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0153] The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0154] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform all the steps of the image denoising method provided by the above methods.

[0155] The method may at least include: Performing noise transformation on a first noise image obtained by processing the target image sensor to obtain a second noise image that obeys the noise distribution of the reference image sensor; The second noise image is input into a noise reduction model for noise reduction processing to obtain a noise reduction image corresponding to the first noise image; the noise reduction model is an image noise reduction model adapted for the reference image sensor.

[0156] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image denoising method, characterized in that: include: Performing noise transformation on a first noise image obtained by processing the target image sensor to obtain a second noise image that obeys the noise distribution of the reference image sensor; Inputting the second noisy image into a denoising model for denoising, to obtain a denoised image corresponding to the first noisy image; The denoising model is an image denoising model adapted to the reference image sensor.

2. The image denoising method according to claim 1, characterized in that: The step of performing noise transformation on the first noise image obtained by processing the target image sensor to obtain a second noise image that complies with the noise distribution of the reference image sensor includes: Acquire a readout noise corresponding to the target image sensor and a readout noise corresponding to the reference image sensor; Removing readout noise corresponding to the target image sensor in the first noisy image to obtain first noisy image data that obeys the Poisson noise distribution of the target image sensor; converting first noisy image data that obeys the Poisson noise distribution of the target image sensor into second noisy image data that obeys the Poisson noise distribution of the reference image sensor; The readout noise corresponding to the reference image sensor is added to the second noisy image data to obtain the second noisy image.

3. The image denoising method according to claim 2, characterized in that: The reference image sensor corresponds to a plurality of different readout noises; and the inputting the second noise image into the noise reduction model for noise reduction processing to obtain a noise reduction image corresponding to the first noise image includes: Inputting a plurality of second noise images to which a plurality of different readout noises corresponding to the reference image sensor are added respectively into the noise reduction model for noise reduction processing, thereby obtaining a plurality of intermediate noise reduction images output by the noise reduction model; Performing mean processing on the plurality of intermediate denoised images to obtain a denoised image corresponding to the first noisy image.

4. The image denoising method according to claim 2, characterized in that: The readout noise corresponding to the target image sensor is at least one black frame obtained by processing the target image sensor under the sensitivity parameter of the first noise image; and / or, The readout noise corresponding to the reference image sensor is at least one black frame processed by the reference image sensor under the sensitivity parameter of the first noise image.

5. The image denoising method according to claim 2, characterized in that: The step of converting the first noisy image data that obeys the Poisson noise distribution of the target image sensor into the second noisy image data that obeys the Poisson noise distribution of the reference image sensor comprises: Acquire an incident noise parameter corresponding to the target image sensor and an incident noise parameter corresponding to the reference image sensor under the sensitivity parameter of the first noise image; Dividing the incident noise parameter corresponding to the reference image sensor under the sensitivity parameter of the first noise image by the incident noise parameter corresponding to the target image sensor to obtain a conversion coefficient representing a conversion relationship between the Poisson noise distribution of the target image sensor and the Poisson noise distribution of the reference image sensor; The first noisy image data is converted into the second noisy image data using the conversion coefficient.

6. The image denoising method according to claim 1, characterized in that: The training process of the denoising model includes: Acquire a clean reference image and an incident noise parameter and a readout noise of the reference image sensor under a test sensitivity; Using the incident noise parameter of the reference image sensor at the test sensitivity, synthesizing the readout noise of each pixel in the clean reference image to obtain the incident noise of the clean reference image; Adding the readout noise of the reference image sensor at the test sensitivity and the incident noise of the clean reference image to the clean reference image to obtain a noisy reference image corresponding to the clean reference image; The initial denoising model is optimized and trained using a training data pair consisting of the clean reference image and a noisy reference image corresponding to the clean reference image to obtain an optimized denoising model.

7. The image denoising method according to any one of claims 2 to 6, characterized in that: The image sensor includes any one of the reference image sensor and the target image sensor, and the noise parameter includes any one of a readout noise and an incident noise parameter; The process of acquiring the noise parameters of the image sensor includes: Determine a first calibrated sensitivity and a second calibrated sensitivity adjacent to a target sensitivity parameter, wherein the target sensitivity parameter is between the first calibrated sensitivity and the second calibrated sensitivity, and the target sensitivity parameter includes a sensitivity parameter of a first noise image or a test sensitivity selected during noise reduction model training; Acquire a first calibrated noise parameter of the image sensor at the first calibrated sensitivity and a second calibrated noise parameter of the image sensor at the second calibrated sensitivity; The noise parameter of the image sensor at the target sensitivity parameter is determined according to a first calibrated noise parameter of the image sensor at the first calibrated sensitivity and a second calibrated noise parameter of the image sensor at the second calibrated sensitivity.

8. An image noise reduction device, characterized in that: include: A noise transformation module, used for performing noise transformation on a first noise image obtained by processing the target image sensor to obtain a second noise image that complies with the noise distribution of the reference image sensor; An image denoising module, configured to input the second noisy image into a denoising model for denoising, so as to obtain a denoised image corresponding to the first noisy image; The denoising model is an image denoising model adapted to the reference image sensor.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the image noise reduction method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image noise reduction method according to any one of claims 1 to 7 is implemented.

Images