Image noise adding method and device, equipment and storage medium

Abstract

The invention relates to the technical field of image processing, and particularly discloses an image noise adding method and device, equipment and a storage medium. The computer equipment comprises a memory and a processor, the memory and the processor are in communication connection with each other, computer instructions are stored in the memory, and the processor executes the image noise adding method by executing the computer instructions. According to the method, the difference between the constructed noisy graph and the real noisy graph can be reduced, so that the noisy graph is more real.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image noise adding method, device, equipment and storage medium. Background Art

[0002] Image sensors based on CMOS (Complementary Metal Oxide Semiconductor) technology have the characteristics of low power consumption and low cost. However, since each photosensitive unit of CMOS needs to be equipped with an analog amplifier, the read noise characteristics of each amplifier are difficult to maintain consistency. Therefore, compared with CCD image sensors (Charge coupled Device, which can be called CCD image sensors) with only one amplifier placed on the edge of the chip, CMOS sensors exhibit greater noise.

[0003] At present, in the image or video denoising processing solution based on artificial intelligence (AI) of neural network, it is possible to achieve better results than traditional denoising algorithms under high sensitivity and low signal-to-noise ratio. Specifically, a noisy image is generated by adding noise to a clean image, and then the noisy image is used as the input of the neural network model, and the clean image is used as a label to train the neural network model to obtain a trained neural network model, which has image denoising function. Therefore, the more accurate the noise model, the better the denoising effect of the trained network. However, in the process of denoising the image, it is necessary to model the noise of the dark black frame image. Therefore, when training the denoising neural network, it is necessary to add noise to the clean, noise-free image in order to improve the denoising effect of the network during denoising. Summary of the invention

[0004] In view of this, the present invention provides an image denoising method, device, equipment and storage medium to solve the problem of poor denoising effect of noisy images in the neural network denoising process in the related art.

[0005] In a first aspect, the present invention provides an image denoising method, the method comprising:

[0006] Acquire a target image, wherein the target image is an image after Poisson noise is added;

[0007] Collecting a plurality of black frame images, and calculating the mean and standard deviation of each pixel in the plurality of black frame images according to the plurality of black frame images;

[0008] According to the mean and standard deviation of each pixel point, a random number corresponding to the probability distribution is obtained by a broken line function fitting method;

[0009] Generate a normal distribution random number according to the random number of the probability distribution, and determine the read noise by the normal distribution random number and the gain;

[0010] The read noise is added to the pixel values ​​of the target image to generate a noisy image.

[0011] The image denoising method provided by the present invention calculates the mean and standard deviation of each image pixel point based on multiple black frame images, and then uses a broken line function fitting method to obtain probability distribution random numbers of each mean and standard deviation. Because the mean and standard deviation of the readout noise of different pixel points have the same normal distribution, the normal distribution random number is determined according to the probability distribution random number obtained by fitting, and then the readout noise of the target image is determined using the normal distribution random number and the gain. Finally, the readout noise is added to the original target image to obtain a noisy image. The method realizes the denoising of a clean image.

[0012] In addition, the readout noise determined based on the normally distributed random number can reduce the difference between the analog amplifier readout noise of each photosensitive unit, making the noisy image after adding noise through the present invention more realistic, and then a training set can be constructed using the noisy image constructed by the present invention, so as to help improve the denoising performance of the trained neural network model.

[0013] In a second aspect, the present invention provides an image noise adding device, the device comprising:

[0014] An acquisition module, used for acquiring a target image, wherein the target image is an image after Poisson noise is added;

[0015] A calculation module, used for collecting a plurality of black frame images, and calculating the mean and standard deviation of each pixel in the plurality of black frame images according to the plurality of black frame images;

[0016] A processing module, used to obtain a uniformly distributed random number within a preset range by a broken line function fitting method according to the mean and standard deviation of each pixel point;

[0017] A determination module, configured to generate a normally distributed random number according to the uniformly distributed random number, and determine a readout noise by using the normally distributed random number and a gain;

[0018] The noise adding module is used to add the read noise to the pixel value of the target image to generate a noisy image.

[0019] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the image denoising method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0020] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the image denoising method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 is a flowchart of an image noise adding method according to an embodiment of the present invention;

[0023] Figure 2 is a flow chart of another image denoising method according to an embodiment of the present invention;

[0024] Figure 3 is a flowchart of another image denoising method according to an embodiment of the present invention;

[0025] Figure 4a is a schematic diagram of a histogram of a mean according to an embodiment of the present invention;

[0026] Figure 4b is a schematic diagram of a histogram of a standard deviation according to an embodiment of the present invention;

[0027] Figure 5a is a schematic diagram of a histogram of means and fitting results in a logarithmic coordinate according to an embodiment of the present invention;

[0028] Figure 5b is a schematic diagram of a histogram of standard deviation and a fitting result in a logarithmic coordinate according to an embodiment of the present invention;

[0029] Figure 6 is a flowchart of another image denoising method according to an embodiment of the present invention;

[0030] Figure 7 is a structural block diagram of an image noise adding device according to an embodiment of the present invention;

[0031] Figure 8 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0033] In the related art, a neural network model (for example, a denoising network) with image denoising function can be trained by using a clean image as a label and a noisy image as the input of the neural network model; wherein the noisy image is obtained by adding noise to the clean image. A key point in adding noise to a clean image is to establish a noise model that is close to the real image noise. For example, random numbers that satisfy the Poisson distribution can be generated to simulate the shot noise of photons and random numbers that satisfy the Gaussian distribution can be generated to simulate the readout noise that is unrelated to the light signal. However, due to the technical limitations of the related art, such as the fact that all pixels of the image in the noise model obey the same normal distribution when reading out the noise, and the fact that the readout noise between pixels on the image sensor of the CMOS process is different, how to reasonably add readout noise to the image has become a technical pain point in this field.

[0034] In order to solve the above problem, this method uses the noise model to assume that the readout noise of all pixels obeys the same normal distribution. Aiming at the characteristics of different readout noises between CMOS pixels, a readout noise model with different pixel distributions is proposed. The readout noise determined by this noise model is closer to the actual situation than the same distribution of pixels, and the difference caused by the readout noise of the analog amplifier is reduced, thereby improving the denoising effect of the network.

[0035] According to an embodiment of the present invention, an embodiment of a method for adding image noise is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] The embodiment of the present invention is used to implement adding Poisson noise and readout noise to a clean original image, so as to improve the degree of closeness between the constructed noisy image and the real noisy image.

[0037] The present invention provides an image noise adding method in an embodiment, which can be used in a computer device. Figure 1 is a flow chart of a method for adding image noise according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0038] Step S101: Acquire a target image, where the target image is an image with Poisson noise added.

[0039] Specifically, one implementation method is to first obtain a clean original image, that is, a clean image (raw image), and perform the following operations on each pixel point on the clean raw image to obtain the target image.

[0040] Subtract the black level from the pixel value of each pixel and then divide it by αg A , take the obtained value as the mean of Poisson distribution, then generate Poisson distribution random numbers based on the mean, and multiply the generated Poisson distribution random numbers by αg A , plus the black level, we can get the pixel value after adding Poisson noise.

[0041] The method of generating a Poisson distribution random number can be, for example, generated by the library function random.poisson(lambda) in numpy (an open source scientific computing library); wherein α represents a coefficient related to the photoelectric conversion efficiency, g A Indicates the preset analog signal gain (known parameter).

[0042] For adding Poisson noise, the noise can be added according to the noise adding method of the ordinary noise model, for example, according to the noise model

[0043] X=αgU+gN 1 +N 2 +δ (1)

[0044]

[0045] Where X represents the random variable of pixel value, EX represents its mathematical expectation; the first term αgU represents the Poisson noise of photons, U is a Poisson distribution random variable with mean (EX-δ) / αg, P(λ) represents the Poisson distribution with mean λ, in formula (1), λ=(EX-δ) / αg; the second term gN 1 and the third term N 2 represents the read noise, and N 1 and N 2 Obeys normal distribution, N(0,σ 2 ) represents a normal distribution with a mean of 0 and a standard deviation of σ; g is the gain, α is a coefficient related to the photoelectric conversion efficiency; δ is the black level.

[0046] Step S102: collecting a plurality of black frame images, and calculating the mean and standard deviation of each pixel in the plurality of black frame images according to the plurality of black frame images.

[0047] Each black frame image is an image or an image in a video taken in a completely dark environment. Generally, the pixel value of each pixel on the black frame image should be close to the black level.

[0048] Take a black frame image captured by a camera with an image sensor model of imx485 as an example. In this embodiment, a video or multiple images are captured in a completely dark environment. Assume that the video consists of N completely black images, where N represents the number of video frames, for example, N=100.

[0049] In this embodiment, steps S102 to S104 are used to determine the readout noise, where the readout noise is different from the traditional noise model. In this solution, not all pixel values ​​of the entire image obey the normal distribution with the same parameters, because the parameters (mean, variance) of the normal distribution of different pixels also need to be represented by random variables, such as formula (2):

[0050] N 1 :N(M,S 2 ) (2)

[0051] Since the value of gain g is usually large, N in formula (1) 2 The influence of is relatively small, so N is ignored in this embodiment. 2 In formula (2), the random variables M and S represent the mean and standard deviation of the pixel value X, respectively. The standard deviation S is the arithmetic square root of the variance. Assume that the random variable X i , it can be expressed by formula (3):

[0052] X i =αgU i +gn i +δ (3)

[0053]

[0054] Among them, X i Represents the pixel value of pixel i, μ i and σ i are samples of random variables M and S respectively. Since the mean value μ of the readout noise at different pixels i and standard deviation σ i Different, but obey the same distribution, that is, the distribution of M and S, so it is necessary to calibrate different sensors. The specific calibration method is steps S103 and S104.

[0055] The following introduces the calibration method steps S103 and S104, ie, the non-parametric estimation method for the M and S distributions.

[0056] Step S103: According to the mean and standard deviation of each pixel point, a uniformly distributed random number within a preset range is obtained by a broken line function fitting method.

[0057] According to several frames of collected matte black frame images, the mean and standard deviation of each pixel are calculated, and then the histograms of the mean and standard deviation are respectively statistically calculated. According to the histograms of the mean and standard deviation, a uniformly distributed random number within a preset range is obtained by a broken line function fitting method.

[0058] The polyline function fitting method is used to convert the histogram of the mean and standard deviation of each pixel into a probability density function of linear coordinates, that is, to fit the histogram into a piecewise linear function representation, and then determine the uniformly distributed random number based on the linear function representation. The random number is a random number uniformly distributed within the calibration range.

[0059] Step S104: Generate a normally distributed random number according to the uniformly distributed random number, and determine the readout noise by using the normally distributed random number and gain.

[0060] One implementation method is to generate a random number with a uniform distribution, such as a random number with a mean value μ i and standard deviation σ i , according to these mean values ​​μ i and standard deviation σ i The code generates n pairs of μ i and σ i , n is the number of pixels, and then these n pairs μ i and σ i As the parameters of the normal distribution, a normal distribution random number generator is used to generate n random numbers. For example, the numpy.random.normal function can be used to generate normal distribution random numbers. Finally, the product of the normal distribution random number and the gain g is calculated to obtain the readout noise.

[0061] Among them, the read noise is a kind of noise that satisfies the Gaussian distribution and is independent of the light signal. In this embodiment, the read noise can be determined through the above step S104, and the read noise is used to be added to the image to which Poisson noise has been added. The image to which Poisson noise has been added is the target image. The specific process of adding Poisson noise can be referred to the above step S101, which will not be repeated here in this embodiment.

[0062] Step S105: adding the read noise to the pixel values ​​of the target image to generate a noisy image.

[0063] Specifically, the readout noise is added to the pixel values ​​of the previous image with Poisson noise (ie, the target image), thereby completing the noise addition to the clean raw image.

[0064] The image denoising method provided by the present invention calculates the mean and standard deviation of each image pixel point based on multiple black frame images, and then uses a broken line function fitting method to obtain probability distribution random numbers of each mean and standard deviation. Because the mean and standard deviation of the readout noise of different pixel points have the same normal distribution, the normal distribution random number is determined according to the probability distribution random number obtained by fitting, and finally the readout noise of the target image is determined using the normal distribution random number and the gain, and finally the readout noise is added to the original target image to obtain a noisy image. The method realizes the denoising of a clean image.

[0065] In addition, the readout noise determined based on the normally distributed random number can reduce the difference between the analog amplifier readout noise of each photosensitive unit, making the noisy image after adding noise through the present invention more realistic, and then a training set can be constructed using the noisy image constructed by the present invention, so as to help improve the denoising performance of the trained neural network model.

[0066] In some alternative embodiments, see Figure 2 , the above step S103 specifically includes:

[0067] Step S1031 , according to the mean and standard deviation of each pixel point, a histogram of the mean and the histogram of the standard deviation of each pixel point are statistically calculated.

[0068] As in the aforementioned example, for 100 black frame images, the mean value corresponding to each pixel point is the mean value of the 100 pixel values, and the standard deviation corresponding to each pixel point is the standard deviation of the 100 pixel values.

[0069] After finding the mean and standard deviation of each pixel, the histogram of the mean and the histogram of the standard deviation are calculated respectively. Figure 4a and Figure 4b As shown. Among them, Figure 4a is the histogram of the mean, Figure 4b is a histogram of the standard deviation.

[0070] Step S1032, performing linear coordinate transformation on the histogram of the mean value of each pixel point and the histogram of the standard deviation to obtain a probability density function that approximates the mean value and the standard deviation of each pixel point.

[0071] Specifically, Figure 3 As shown, the method includes:

[0072] Step S1032-1, using a broken line function to fit the histogram, converting the vertical coordinate in the histogram into a logarithmic coordinate.

[0073] For example, convert the ordinate of the histogram of the mean and standard deviation to a logarithmic coordinate with base e. Figure 5a and Figure 5bAs shown, the method adopted in this embodiment is: a broken line function is used to fit the histogram under logarithmic coordinates. The histogram is fitted using n broken lines, and the coordinates of n+1 vertices are calibrated by fitting the broken line function, wherein the fitted broken line function is represented by a piecewise linear function. Then, according to the calibrated n+1 vertex coordinates, for example (x 1 ,y 1 ), (x 2 ,y 2 )…(x N+1 ,y N+1 ), converting the vertical coordinate in the histogram into the logarithmic coordinate.

[0074] Step S1032-2, converting the logarithmic coordinates into linear coordinates to obtain a piecewise logarithmic function expression of the histogram, and obtaining a probability density function that approximates the mean and standard deviation of each pixel point according to the expression.

[0075] The above n+1 vertex coordinates are used to fit the broken line function, which can be represented by a piecewise linear function, as shown in expression (4):

[0076]

[0077] Where k is the slope, k n =(y n+1 -y n ) / (x n+1 -x n ), k n It represents the slope of each line segment on the fitting result line in the above logarithmic coordinate system.

[0078] exist Figure 5a The jagged line indicates the aforementioned Figure 4a The schematic diagram of the mean histogram after conversion to a logarithmic coordinate system. The non-serrated line represents the fitted histogram. Figure 5b The jagged line in the figure is the aforementioned Figure 5a The figure is a schematic diagram of the standard deviation histogram after being converted to a logarithmic coordinate system; the line chart without jagged edges is the histogram generated after fitting.

[0079] Finally, the broken line function in logarithmic coordinates is converted to linear coordinates to obtain a probability density function, which is expressed as the following relation (5):

[0080]

[0081] Wherein, f(x) represents a probability density function. In this embodiment, according to the logarithmic function expression of the histogram of the mean and standard deviation in step S1032-2, the cumulative distribution function of the mean and standard deviation corresponding to each pixel point can be obtained respectively.

[0082] Step S1033, based on the probability density function approximated by the mean and standard deviation of each pixel point, a cumulative distribution function corresponding to the probability density function is calculated.

[0083] One implementation method is to perform an integral operation on the probability density function of the approximate mean and standard deviation of each pixel point to obtain the cumulative distribution function of the mean and the cumulative distribution function of the standard deviation.

[0084] Specifically, based on the probability density function of a random variable, a method for generating a random number is to find the inverse function of its cumulative distribution function and then substitute a uniformly distributed random number. As shown in formula (5), the cumulative distribution function corresponding to the probability density function (i.e., the integral of the probability density function) is calculated as follows:

[0085]

[0086] In this formula, represents the cumulative distribution function F(x).

[0087] Step S1034, generating random numbers uniformly distributed within a preset range according to the calibrated vertex coordinates.

[0088] In the above formula (6), when z 1 = 0, and z n satisfy When , the inverse function of the cumulative distribution function is as shown in the following formula (7):

[0089]

[0090] When using this method to generate random numbers, the x obtained by the above calibration is 1 :x N+1 ,y 1 :y N+1 , z can be calculated 1 :z N+1 , and then generate the preset range [0,z N+1 ] and then substitute the random number into equation (7) to generate a random number with the corresponding probability distribution.

[0091] Step S1035 , generating random numbers of the probability distribution according to the inverse function of the cumulative distribution function and the uniformly distributed random numbers.

[0092] It should be noted that when the above method is used to generate a black frame image, the parameter can be fitted to the mean histogram of the black frame pixel values ​​based on formula (4): Fitting parameters to the histogram of standard deviation Then, we can use formula (7) to generate samples of mean and standard deviation, such as mean μ i and standard deviation σ i Finally, according to the above formula (3), a black frame image with readout noise can be generated using a normal distribution random number generator.

[0093] In this embodiment, firstly, the histogram of the mean and standard deviation of each pixel is counted according to multiple black frame images, and then the histogram in the logarithmic coordinate system is fitted by a piecewise linear function to obtain the linear fitting result represented by the piecewise logarithmic function, that is, the probability density function of the mean and standard deviation of each pixel point is obtained, and then the corresponding cumulative distribution function is determined according to the probability density function, and finally the probability distribution random number is obtained according to the cumulative distribution function. Due to the amplification characteristic parameters of the amplifiers in different COMS sensors, the random variables M and S (representing the mean and standard deviation of the pixel value) have the same normal distribution, so the probability density function is determined based on the mean and standard deviation of the normal distribution, and the corresponding cumulative distribution function can obtain a relatively stable random number, thereby reducing the difference between the analog amplifier readout noise of each photosensitive unit, making the readout noise determined by this method more real and accurate, and providing a basis for the noise model training set constructed later.

[0094] In addition, the above-mentioned construction and modeling of black frame noise corresponds to the generation of readout noise.

[0095] Optionally, in some other optional implementations, the above step S1035, generating a normally distributed random number according to a random number of a probability distribution, includes:

[0096] Generate N pairs of means and standard deviations according to the random numbers of the probability distribution, where N is the number of pixels, N≥1 and N is a positive integer; use the N pairs of means and standard deviations as parameters of the normal distribution, and use the random function of the numpy library to generate N normal distribution random numbers.

[0097] The random function of the numpy library may be the numpy.random.normal() function.

[0098] Specifically, in order to generate the readout noise, according to the calibration method of steps S1031 to S1035, the random number (such as the mean μ i and standard deviation σ i ) code, generating N pairs of μ i and σ i , and then these N pairs of μ i and σ i As the parameters of the normal distribution, use a normal distribution random number generator to generate n random numbers. For example, you can use the numpy.random.normal() function to generate normal distribution random numbers.

[0099] After obtaining the above-mentioned normal distribution random number, the read noise is obtained by multiplying the normal distribution random number by the gain g. Finally, the read noise is added to the pixel value of the target image with Poisson noise added previously, thus completing the process of adding noise to the clean raw image.

[0100] In this embodiment, based on the probability distribution random number, the normal distribution random number is generated by the random function of the numpy library, and the read noise to be added can be calculated using the normal distribution random number, and the read noise can effectively reduce the difference between the analog amplifier read noise of the photosensitive unit, so as to obtain the noise model training set, which is helpful to improve the denoising performance of the trained neural network model.

[0101] In this embodiment, a method for adding noise to an image is provided, which can be used in a computer device. Figure 6 is a flow chart of an image denoising method according to an embodiment of the present invention. Figure 6 As shown, the process includes the following steps:

[0102] Step 1: Collect multiple black frame images. For example, collect 100 black frame images without light.

[0103] Step 2: According to the black frame image, calculate the mean and standard deviation of each pixel, and calculate the histogram of the mean and standard deviation of each pixel.

[0104] For the specific process, please refer to the aforementioned step S102, which will not be described in detail in this embodiment.

[0105] Step 3: Use a piecewise exponential function to fit the above histogram of the mean and standard deviation. The specific fitting method is to convert the ordinate of the histogram into a logarithmic coordinate fitting with base e, and use a piecewise linear function to fit the histogram under the logarithmic coordinate to obtain the probability density function of the mean and the probability density function of the standard deviation.

[0106] For the specific process, please refer to step S103 of the aforementioned embodiment, and the specific sub-steps S1031 to S103.

[0107] Step 4: Based on the probability density function of the mean value of each pixel and the probability density function of the standard deviation of each pixel, calculate their corresponding cumulative distribution functions.

[0108] Specifically, it can be obtained by using the cumulative distribution function corresponding to the probability density function shown in the above formula (5) (calculating the integral of the probability density function).

[0109] Step 5: Based on the cumulative distribution function of the mean and the cumulative distribution function of the standard deviation, calculate the uniformly distributed random number within the calibrated vertex coordinate range, then obtain the normally distributed random number based on the random number, and finally multiply the normally distributed random number by the gain to obtain the readout noise.

[0110] Step 6: Add the readout noise obtained in the previous step to the target image (the image after Poisson noise is added) to obtain a noisy image, thus completing the denoising of the clean image.

[0111] This embodiment can be used to implement the function of adding Poisson noise and readout noise to a clean original image, thereby improving the closeness of the constructed noisy image to the real noisy image. In addition, this method overcomes the difference between the analog amplifier readout noise of different photosensitive units, generates readout noise using uniformly distributed random numbers, and implements a more realistic noise model overall. The noisy image generated using a more realistic noise model is closer to the actual shooting situation of the camera, so that the denoising performance of the trained denoising network is better.

[0112] In this embodiment, an image noise adding device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0113] This embodiment provides an image noise adding device, such as Figure 7 As shown, including:

[0114] The acquisition module 701 is used to acquire a target image, where the target image is an image with Poisson noise added.

[0115] The calculation module 702 is used to collect a plurality of black frame images, and calculate the mean and standard deviation of each pixel in the plurality of black frame images according to the plurality of black frame images.

[0116] The processing module 703 is used to obtain a random number uniformly distributed within a preset range through a broken line function fitting method according to the mean and standard deviation of each pixel point.

[0117] The determination module 704 is configured to generate a normally distributed random number according to the uniformly distributed random number, and determine the readout noise by using the normally distributed random number and the gain.

[0118] The noise adding module 705 is used to add the read noise to the pixel values ​​of the target image to generate a noisy image.

[0119] In an optional embodiment, the processing module 703 is specifically used to calculate the histogram of the mean and the histogram of the standard deviation of each pixel point based on the mean and standard deviation of each pixel point; perform linear coordinate transformation on the histogram of the mean and the histogram of the standard deviation of each pixel point to obtain a probability density function that approximates the mean and standard deviation of each pixel point; calculate the cumulative distribution function corresponding to the probability density function based on the probability density function that approximates the mean and standard deviation of each pixel point; generate random numbers that are uniformly distributed within a preset range based on the calibrated vertex coordinates; and generate random numbers with a probability distribution based on the inverse function of the cumulative distribution function and the uniformly distributed random numbers.

[0120] In another optional embodiment, the processing module 703 is specifically used to fit the histogram using a broken line function, convert the vertical coordinate in the histogram into a logarithmic coordinate; convert the logarithmic coordinate into a linear coordinate to obtain a piecewise logarithmic function expression of the histogram, and obtain a probability density function that approximates the mean and standard deviation of each pixel point based on the expression.

[0121] In another optional embodiment, the processing module 703 is specifically used to fit the histogram using n-segment broken lines, and calibrate the coordinates of n+1 vertices by fitting the broken line function, wherein the fitted broken line function is represented by a piecewise linear function; according to the calibrated n+1 vertex coordinates, the vertical coordinate in the histogram is converted into a logarithmic coordinate.

[0122] In yet another optional implementation, the processing module 703 is further configured to perform an integral operation on the probability density function of the approximate mean and standard deviation of each pixel point to obtain a cumulative distribution function of the mean and a cumulative distribution function of the standard deviation.

[0123] In another optional implementation, the processing module 703 is further specifically used to generate N pairs of means and standard deviations based on random numbers of probability distribution, where N is the number of pixels, and N≥1; using the N pairs of means and standard deviations as parameters of normal distribution, and using the random function of the numpy library to generate N normally distributed random numbers.

[0124] In yet another optional implementation, the determination module 704 is specifically configured to calculate the product of the normally distributed random number and the gain to obtain the readout noise.

[0125] The further functional description of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0126] The image noise adding device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0127] The embodiment of the present invention also provides a computer device having the above Figure 7 The image noise adding device shown.

[0128] See also Figure 8 , Figure 8 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 8 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0129] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0130] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the image denoising method shown in the above embodiment.

[0131] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0132] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0133] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0134] An embodiment of the present invention also provides a computer-readable storage medium, and the above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented by downloading through a network and originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware.

[0135] The storage medium may be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid state drive, etc.; further, the storage medium may also include a combination of the above-mentioned types of memories. It is understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, processor or hardware, the method shown in the above embodiment is implemented.

[0136] The foregoing content can be better understood in accordance with the following terms:

[0137] Clause A1. A method for adding noise to an image, the method comprising:

[0138] Acquire a target image, wherein the target image is an image after Poisson noise is added;

[0139] Collecting a plurality of black frame images, and calculating the mean and standard deviation of each pixel in the plurality of black frame images according to the plurality of black frame images;

[0140] According to the mean and standard deviation of each pixel point, a random number corresponding to the probability distribution is obtained by a broken line function fitting method;

[0141] Generate a normal distribution random number according to the random number of the probability distribution, and determine the read noise by the normal distribution random number and the gain;

[0142] The read noise is added to the pixel values ​​of the target image to generate a noisy image.

[0143] Clause A2. The method according to claim A1, wherein the random number corresponding to the probability distribution is obtained by a broken line function fitting method based on the mean and standard deviation of each pixel point, comprising:

[0144] According to the mean and standard deviation of each pixel point, a histogram of the mean and the histogram of the standard deviation of each pixel point are calculated;

[0145] Performing linear coordinate transformation on the histogram of the mean value of each pixel point and the histogram of the standard deviation to obtain a probability density function that approximates the mean value and the standard deviation of each pixel point;

[0146] According to the probability density function approximated by the mean and standard deviation of each pixel point, a cumulative distribution function corresponding to the probability density function is calculated;

[0147] Generate a random number evenly distributed within a preset range according to the calibrated vertex coordinates;

[0148] The random number of the probability distribution is generated according to the inverse function of the cumulative distribution function and the random number of the uniform distribution.

[0149] Clause A3. The method according to clause A2, wherein the histogram of the mean value of each pixel point and the histogram of the standard deviation are subjected to linear coordinate transformation to obtain a probability density function that approximates the mean value and the standard deviation of each pixel point, including:

[0150] Fitting the histogram using a broken line function, converting the ordinate in the histogram into a logarithmic coordinate;

[0151] The logarithmic coordinates are converted into linear coordinates to obtain a piecewise logarithmic function expression of the histogram, and a probability density function approximating the mean and standard deviation of each pixel point is obtained according to the expression.

[0152] Clause A4. The method according to clause A3, wherein fitting the histogram using a broken line function and converting the ordinate in the histogram into a logarithmic coordinate comprises:

[0153] Fitting the histogram using n segments of broken lines, calibrating the coordinates of n+1 vertices by fitting the broken line function, wherein the fitted broken line function is represented by a piecewise linear function;

[0154] According to the calibrated n+1 vertex coordinates, the ordinate in the histogram is converted into the logarithmic coordinate.

[0155] Clause A5. The method according to clause A2, wherein the cumulative distribution function corresponding to the probability density function is calculated based on the probability density function approximated by the mean and standard deviation of each pixel point, and comprises:

[0156] An integral operation is performed on the probability density functions of the approximate mean and standard deviation of each pixel point to obtain a cumulative distribution function of the mean and a cumulative distribution function of the standard deviation.

[0157] Clause A6. The method according to any one of clauses A1 to A5, wherein the step of generating a normally distributed random number based on the random number of the probability distribution comprises:

[0158] Generate N pairs of means and standard deviations according to the random numbers of the probability distribution, where N is the number of pixels and N≥1; use the N pairs of means and standard deviations as parameters of the normal distribution, and use the random function of the numpy library to generate N random numbers of the normal distribution.

[0159] Item A7. The method according to Item A6, wherein determining the read noise using the normally distributed random number and the gain comprises: calculating the product of the normally distributed random number and the gain to obtain the read noise.

[0160] Clause A8. An image noise adding device, the device comprising:

[0161] An acquisition module, used for acquiring a target image, wherein the target image is an image after Poisson noise is added;

[0162] A calculation module, used for collecting a plurality of black frame images, and calculating the mean and standard deviation of each pixel in the plurality of black frame images according to the plurality of black frame images;

[0163] A processing module, used to obtain a uniformly distributed random number within a preset range by a broken line function fitting method according to the mean and standard deviation of each pixel point;

[0164] A determination module, configured to generate a normally distributed random number according to the uniformly distributed random number, and determine a readout noise by using the normally distributed random number and a gain;

[0165] The noise adding module is used to add the read noise to the pixel value of the target image to generate a noisy image.

[0166] Item A9. A computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the image denoising method described in any one of Items A1 to A7 by executing the computer instructions.

[0167] Item A10. A computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions are used to cause a computer to execute the image denoising method described in any one of Items A1 to A7.

[0168] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for adding noise to an image, It is characterized in that The method comprises: Acquire a target image, wherein the target image is an image after Poisson noise is added; Collecting a plurality of black frame images, and calculating the mean and standard deviation of each pixel in the plurality of black frame images according to the plurality of black frame images; According to the mean and standard deviation of each pixel point, a random number corresponding to the probability distribution is obtained by a broken line function fitting method; Generate a normal distribution random number according to the random number of the probability distribution, and determine the read noise by the normal distribution random number and the gain; The read noise is added to the pixel values ​​of the target image to generate a noisy image.

2. The method according to claim 1, It is characterized in that The method of obtaining a random number corresponding to the probability distribution by a broken line function fitting method according to the mean and standard deviation of each pixel point includes: According to the mean and standard deviation of each pixel point, a histogram of the mean and the histogram of the standard deviation of each pixel point are calculated; Performing linear coordinate transformation on the histogram of the mean value of each pixel point and the histogram of the standard deviation to obtain a probability density function that approximates the mean value and the standard deviation of each pixel point; According to the probability density function approximated by the mean and standard deviation of each pixel point, a cumulative distribution function corresponding to the probability density function is calculated; Generate a random number evenly distributed within a preset range according to the calibrated vertex coordinates; The random number of the probability distribution is generated according to the inverse function of the cumulative distribution function and the random number of the uniform distribution.

3. The method according to claim 2, It is characterized in that The linear coordinate transformation is performed on the histogram of the mean value of each pixel point and the histogram of the standard deviation to obtain a probability density function that approximates the mean value and the standard deviation of each pixel point, including: Fitting the histogram using a broken line function, converting the ordinate in the histogram into a logarithmic coordinate; The logarithmic coordinates are converted into linear coordinates to obtain a piecewise logarithmic function expression of the histogram, and a probability density function approximating the mean and standard deviation of each pixel point is obtained according to the expression.

4. The method according to claim 3, It is characterized in that The step of fitting the histogram using a broken line function and converting the ordinate in the histogram into a logarithmic coordinate comprises: Fitting the histogram using n segments of broken lines, calibrating the coordinates of n+1 vertices by fitting the broken line function, wherein the fitted broken line function is represented by a piecewise linear function; According to the calibrated n+1 vertex coordinates, the ordinate in the histogram is converted into the logarithmic coordinate.

5. The method according to claim 2, It is characterized in that The method of calculating a cumulative distribution function corresponding to the probability density function based on the probability density function approximated by the mean and standard deviation of each pixel point includes: An integral operation is performed on the probability density functions of the approximate mean and standard deviation of each pixel point to obtain a cumulative distribution function of the mean and a cumulative distribution function of the standard deviation.

6. The method according to any one of claims 1 to 5, It is characterized in that The step of generating a normally distributed random number according to the random number of the probability distribution comprises: Generate N pairs of means and standard deviations according to the random numbers of the probability distribution, where N is the number of pixels and N≥1; The N pairs of means and standard deviations are used as parameters of a normal distribution, and the random function of the numpy library is used to generate N normally distributed random numbers.

7. The method according to claim 6, It is characterized in that The determining of the read noise by using the normal distribution random number and the gain comprises: The product of the normally distributed random number and the gain is calculated to obtain the read noise.

8. An image noise adding device, It is characterized in that The device comprises: An acquisition module, used for acquiring a target image, wherein the target image is an image after Poisson noise is added; A calculation module, used for collecting a plurality of black frame images, and calculating the mean and standard deviation of each pixel in the plurality of black frame images according to the plurality of black frame images; A processing module, used to obtain a uniformly distributed random number within a preset range by a broken line function fitting method according to the mean and standard deviation of each pixel point; A determination module, configured to generate a normally distributed random number according to the uniformly distributed random number, and determine a readout noise by using the normally distributed random number and a gain; The noise adding module is used to add the read noise to the pixel value of the target image to generate a noisy image.

9. A computer device, It is characterized in that include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the image denoising method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the image denoising method according to any one of claims 1 to 7.

Images