High dynamic range image noise reduction method and apparatus

Abstract

The present application provides an image noise reduction method, an apparatus, an electronic device, and a computer-readable storage medium, the method comprising the steps of: performing variance stabilization transformation on an image to be noise-reduced based on a predetermined noise model parameter to obtain a first intermediate image; performing noise reduction processing on the first intermediate image based on a predetermined signal-to-noise ratio variation curve and a predetermined noise reduction algorithm to obtain a second intermediate image; performing an inverse transformation of the variance stabilization transformation on the second intermediate image based on the noise model parameter to obtain a noise-reduced image, the signal-to-noise ratio variation curve representing noise variance difference information corresponding to the image to be noise-reduced under different exposure gains. The image noise reduction method improves the noise reduction effect on a high dynamic range image.

Description

[Technical Field]

[0001] The present application relates to the field of image processing, and in particular to an image noise reduction method, apparatus, electronic device and computer-readable storage medium. [Background technology]

[0002] With the development of intelligent driving technology, CMOS (Complementary Metal Oxide Semiconductor) image sensors for automotive applications have emerged. High-dynamic-range images captured by CMOS image sensors contain multiple exposure information, offering high light sensitivity and a wide dynamic range. While the dynamic range of images captured by conventional image sensors is typically around 70 dB, the dynamic range of images captured by CMOS image sensors can reach over 100 dB, even reaching 180 dB (exceeding the human eye's ability to perceive the environment). The advantage of CMOS image sensors is that they can accurately reproduce the shape and color of traffic lights in brightly lit areas, while also reproducing details in dark areas under poor lighting conditions. To restore the high-dynamic-range information from images captured by CMOS image sensors, the image processing module must perform noise reduction processing on the image information output from the CMOS image sensor to improve the signal-to-noise ratio.

[0003] Currently, when performing noise reduction processing on high dynamic range images, noise reduction processing is usually performed directly using a linear noise model. However, because the noise characteristics in high dynamic range images do not follow a linear model, there is a problem that noise reduction becomes uneven and the noise reduction effect is reduced. Summary of the Invention

[0004] An object of the embodiments of the present application is to provide an image noise reduction method, apparatus, electronic device, and computer-readable storage medium for improving the noise reduction effect on high dynamic range images.

[0005] In a first aspect, an image noise reduction method according to the present application includes the steps of performing a variance stabilizing transform on an image to be subjected to noise reduction based on predetermined noise model parameters to obtain a first intermediate image, performing noise reduction processing on the first intermediate image based on a predetermined signal-to-noise ratio variation curve and a predetermined noise reduction algorithm to obtain a second intermediate image, and performing an inverse transform of the variance stabilizing transform on the second intermediate image based on the noise model parameters to obtain a noise-reduced image, wherein the signal-to-noise ratio variation curve represents difference information of noise variance corresponding to the image to be subjected to noise reduction at different exposure gains.

[0006] In the embodiment of the present application, when noise reduction is performed on a high dynamic range image using the above method, the noise reduction is performed on the high dynamic range image based on a signal-to-noise ratio variation curve representing the difference information of noise variance corresponding to the image to be noise reduced at different exposure gains and a noise reduction algorithm, and in the process, the estimation of the noise variance distribution is guaranteed to be accurate at each luminance (i.e., at different exposure gains), thereby improving the noise reduction effect.In addition, compared to a noise reduction method that converts a high dynamic range image into multiple frames of low dynamic range images, then performs noise reduction on each of the multiple frames of low dynamic range images and reconverts them into a high dynamic range image, the image noise reduction method according to the embodiment of the present application directly reduces noise on the high dynamic range image, thereby reducing the resource consumption caused by processing multiple frames of low dynamic range images.

[0007] In any embodiment, before the step of performing a variance stabilizing transform on the image to be noise reduced based on predetermined noise model parameters to obtain a first intermediate image, the method further includes the steps of obtaining a plurality of sets of sample images acquired by an image sensor at different exposure gains, and determining noise model parameters based on the plurality of sets of sample images and a predetermined noise model, wherein the image sensor is the image sensor that acquires the image to be noise reduced, and each set of sample images corresponds to one exposure gain.

[0008] In an embodiment of the present application, an image sensor that acquires an image to be subjected to noise reduction acquires multiple sets of sample images at different exposure gains, and noise model parameters are determined by fitting them to the multiple sets of sample images, thereby avoiding variations in noise model parameters depending on the model number of the image sensor and improving the effectiveness of noise reduction for the next image to be subjected to noise reduction.

[0009] In any embodiment, the plurality of sets of sample images are obtained by photographing a gray board in a pre-set environment with the image sensor at different exposure gains.

[0010] In any embodiment, the step of determining noise model parameters based on the plurality of sets of sample images and a predetermined noise model includes the steps of: performing noise reduction on each set of sample images to obtain each set of noise-reduced sample images; determining a noise variance for each set of sample images based on each set of sample images and the noise-reduced sample images corresponding to each set; and determining first and second parameters corresponding to each set of sample images based on each set of sample images and the noise variance corresponding to each set; determining first hyperparameters and second hyperparameters of the noise model based on the photosensitivity of each set of sample images and the first and second parameters corresponding to each set of sample images; and determining the noise model parameters based on the reference photosensitivity of the image to be noise-reduced, the first hyperparameters, and the second hyperparameters.

[0011] In an embodiment of the present application, first, first and second parameters corresponding to each set of sample images are determined based on each set of sample images and the corresponding noise variance. Then, first and second hyperparameters of a noise model are determined based on the photosensitivity of each set of sample images and the first and second parameters corresponding to each set. Determining the first and second hyperparameters corresponds to determining a functional relationship between the photosensitivity and the first and second parameters. Therefore, when an image to be subjected to noise reduction is acquired using an image sensor, the noise model parameters corresponding to the image to be subjected to noise reduction are determined by combining the reference photosensitivity corresponding to the image to be subjected to noise reduction with the first and second hyperparameters. Using the above method, the determined noise model parameters are better suited to the image to be subjected to noise reduction.

[0012] In any embodiment, the reference sensitivity of the image to be noise-reduced is the sensitivity corresponding to the longest exposure frame in the image to be noise-reduced.

[0013] In the embodiment of the present application, the high dynamic range image has multiple single exposures with different exposure gains, and the longest exposure frame is the exposure frame with the longest exposure time and the optimal signal-to-noise ratio. The sensitivity corresponding to the longest exposure frame in the image to be noise reduced is used as the reference sensitivity, and the noise model parameters are determined based on the reference sensitivity, thereby improving the noise reduction effect when performing the next image noise reduction.

[0014] In any embodiment, before the step of performing noise reduction processing on the first intermediate image based on a predetermined signal-to-noise ratio variation curve and a predetermined noise reduction algorithm to obtain a second intermediate image, the method includes the steps of obtaining exposure information of the image to be noise reduced, determining each exposure ratio using an exposure gain and exposure time corresponding to each individual exposure, and determining the signal-to-noise ratio variation curve using the exposure ratio corresponding to each individual exposure, wherein the exposure information includes an exposure gain and exposure time corresponding to each of a plurality of individual exposures.

[0015] In the embodiment of the present application, the signal-to-noise ratio variation curve is determined by analyzing the exposure ratio corresponding to each single exposure in the image to be noise reduced, and then the signal-to-noise ratio variation curve is combined with a predetermined noise reduction algorithm to perform noise reduction on the image to be noise reduced, which can effectively ensure that the estimation of the noise variance distribution under different exposure gains is accurate, so that the details of the noise-reduced image can be maintained at each brightness level, and the noise reduction effect can be improved.

[0016] In any embodiment, the step of determining a respective exposure ratio using an exposure gain and an exposure time corresponding to each of the single exposures comprises: determining an exposure ratio corresponding to each single exposure based on the following formula: i is the exposure ratio corresponding to the i-th single exposure, and exp iis the exposure time corresponding to the i-th single exposure, and gain i is the exposure gain corresponding to the i-th single exposure, EXP is the exposure time of the reference exposure, and GAIN is the exposure gain corresponding to the reference exposure.

[0017]

number

[0018] In a second aspect, the present application provides an image noise reduction apparatus, the apparatus including: a first determination module that performs a variance stabilizing transformation on an image to be noise reduced based on predetermined noise model parameters to obtain a first intermediate image; a second determination module that performs noise reduction processing on the first intermediate image based on a predetermined signal-to-noise ratio variation curve and a predetermined noise reduction algorithm to obtain a second intermediate image; and a third determination module that performs an inverse transformation of the variance stabilizing transformation on the second intermediate image based on the noise model parameters to obtain a noise-reduced image, wherein the signal-to-noise ratio variation curve represents difference information of noise variances corresponding to the image to be noise reduced under different exposure gains.

[0019] In any embodiment, the apparatus further includes a fourth determination module that obtains a plurality of sets of sample images acquired by an image sensor at different exposure gains, and determines noise model parameters based on the plurality of sets of sample images and a predetermined noise model, where the image sensor is an image sensor that acquires the image to be noise reduced, and each set of sample images corresponds to one exposure gain.

[0020] In any embodiment, the plurality of sets of sample images are obtained by photographing a gray board in a pre-set environment with the image sensor at different exposure gains.

[0021] In any embodiment, specifically, the fourth determination module performs noise reduction on each set of sample images to obtain each set of noise-reduced sample images; for each set of sample images, determine the noise variance of each set of sample images based on each set of sample images and the noise-reduced sample images corresponding to each set; further determine first parameters and second parameters corresponding to each set of sample images based on each set of sample images and the noise variance corresponding to each set; determine first hyperparameters and second hyperparameters of the noise model based on the photosensitivity of each set of sample images and the first parameters and second parameters corresponding to each set of sample images; and determine the noise model parameters based on the reference photosensitivity of the image to be noise reduced, the first hyperparameters, and the second hyperparameters.

[0022] In any embodiment, the reference sensitivity of the image to be noise-reduced is the sensitivity corresponding to the longest exposure frame in the image to be noise-reduced.

[0023] In any embodiment, the apparatus further includes a fifth determination module that obtains exposure information of the image to be noise reduced, determines each exposure ratio according to an exposure gain and exposure time corresponding to each individual exposure, and determines a variation curve of the signal-to-noise ratio according to the exposure ratio corresponding to each individual exposure, wherein the exposure information includes an exposure gain and exposure time corresponding to each of a plurality of individual exposures.

[0024] In any embodiment, specifically, the fifth determining module determines the exposure ratio corresponding to each single exposure respectively according to the following formula:

[0025]

number

[0026] In any embodiment, specifically, the second determination module determines the similarity between each pixel point of the first intermediate image and its adjacent pixel point, corrects the similarity according to the variation curve of the signal-to-noise ratio to obtain the similarity of each pixel point after the correction, and performs noise reduction for each pixel point according to the similarity of each pixel point after the correction and the predetermined noise reduction algorithm to obtain the second intermediate image.

[0027] In a third aspect, the present application provides an electronic device including a processor and a memory storing program commands executable by the processor, the program commands being capable of executing the method of any one of the above embodiments.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium storing computer program instructions that, when called and executed by a computer, perform the method of any one of the above embodiments. [Brief explanation of the drawings]

[0029] In order to more clearly explain the technical solutions of the embodiments of the present application, the following will briefly explain the drawings necessary for explaining the embodiments of the present application. The following drawings only illustrate some embodiments of the present application and are not intended to limit the scope. Those skilled in the art can obtain other related drawings based on these drawings without using inventive ability. [Figure 1] 1 is a flowchart of an image noise reduction method according to an embodiment of the present application. [Figure 2] 1 is a flowchart for determining noise model parameters according to an embodiment of the present application. [Figure 3] 1 is a flowchart of a specific embodiment of step A2 according to an embodiment of the present application. [Figure 4] 1 is a flowchart for determining a variation curve of a signal-to-noise ratio according to an embodiment of the present application; [Figure 5] 10 is a flowchart for determining a second intermediate image according to an embodiment of the present application. [Figure 6a] FIG. 6a is a schematic diagram of noise distribution determined by the image noise reduction method according to an embodiment of the present application. [Figure 6b] FIG. 6b is a schematic diagram of noise distribution determined by the image noise reduction method according to an embodiment of the present application. [Figure 6c] FIG. 6c is a schematic diagram of noise distribution determined by the image noise reduction method according to an embodiment of the present application. [Figure 6d] FIG. 6d is a schematic diagram of noise distribution determined by the image noise reduction method according to an embodiment of the present application. [Figure 7] 1 is a structural block diagram of an image noise reduction device according to an embodiment of the present application; [Figure 8]FIG. 1 is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0030] Hereinafter, the technical solutions according to the embodiments of the present application will be described with reference to the drawings in the embodiments of the present application.

[0031] An object of the embodiments of the present application is to provide an image noise reduction method, apparatus, electronic device, and computer-readable storage medium for improving the noise reduction effect on high dynamic range images.

[0032] 1 is a flowchart of an image noise reduction method according to an embodiment of the present application. As shown in FIG. 1, the image noise reduction method includes the following steps:

[0033] Step 101: A variance stabilizing transformation is performed on the image to be subjected to noise reduction based on predetermined noise model parameters to obtain a first intermediate image.

[0034] Step 102: Perform noise reduction processing on the first intermediate image based on a predetermined signal-to-noise ratio variation curve and a predetermined noise reduction algorithm to obtain a second intermediate image.

[0035] Step 103: Perform an inverse transformation of the variance stabilizing transformation on the second intermediate image based on the noise model parameters to obtain a noise-reduced image.

[0036] In the embodiments of the present application, the image to be noise-reduced is a high dynamic range image. The signal-to-noise ratio variation curve represents the difference information of noise variance corresponding to the image to be noise-reduced at different exposure gains. Because the high dynamic range image contains luminance information at multiple different exposure gains, noise occurs at different exposure gains. Performing noise reduction using only a predetermined noise reduction algorithm results in uneven noise reduction. Therefore, noise reduction processing is performed on a first intermediate image, and then noise reduction processing is performed on the first intermediate image by combining the signal-to-noise ratio variation curve and the noise reduction algorithm, thereby improving the accuracy of noise reduction. Finally, the second intermediate image obtained after noise reduction is subjected to an inverse variance stabilization transform, restoring the second intermediate image from the nonlinear domain where variance is stabilized to the linear domain, thereby obtaining a noise-reduced image.

[0037]

[0013] When noise reduction is performed on a high dynamic range image using the above method, the noise reduction is performed on the high dynamic range image based on a signal-to-noise ratio variation curve representing the difference information of noise variance corresponding to the image to be noise reduced at different exposure gains and a noise reduction algorithm, and in the process, the estimation of the noise variance distribution is guaranteed to be accurate for each luminance (i.e., at different exposure gains), thereby improving the noise reduction effect. Furthermore, compared to noise reduction methods that convert a high dynamic range image into multiple frames of low dynamic range images, then perform noise reduction on each of the multiple frames of low dynamic range images and reconvert them into a high dynamic range image, the image noise reduction method according to the embodiment of the present application directly reduces noise on the high dynamic range image, thereby reducing resource consumption associated with processing multiple frames of low dynamic range images.

[0038] The above steps will now be described in detail.

[0039] Step 101: A variance stabilizing transformation is performed on the image to be subjected to noise reduction based on predetermined noise model parameters to obtain a first intermediate image.

[0040] In the embodiment of the present application, first, a variance stabilizing transformation is performed on the image to be noise reduced based on predetermined noise model parameters, and the image to be noise reduced is transformed from a linear domain to a nonlinear domain in which the variance is stabilized, and a first intermediate image is obtained for subsequent image noise reduction processing.

[0041] Specifically, the predetermined noise model may be a Poisson-Gaussian noise model, and a variance stabilizing transformation is performed on the image to be noise reduced according to the noise model parameters of the Poisson-Gaussian noise model to determine the first intermediate image. The variance stabilizing transformation on the image to be noise reduced can be determined by the following formula:

[0042]

number

[0043] The image noise reduction method according to the embodiment of the present application is applicable to an image noise reduction device in which noise model parameters are preset. When an image to be subjected to noise reduction is input to the image noise reduction device, the image noise reduction device directly performs the above-described processing on the image to be subjected to noise reduction using the noise model parameters to obtain a first intermediate image.

[0044] A method for determining the noise model parameters will be described below.

[0045] As shown in FIG. 2, before step 101, the image noise reduction method according to the embodiment of the present application may include the following steps:

[0046] Step A1: Obtain a plurality of sets of sample images captured by an image sensor at different exposure gains.

[0047] Step A2: Determine noise model parameters based on a plurality of sets of sample images and a predetermined noise model.

[0048] In an embodiment of the present application, in order to improve the accuracy of the noise model parameters, the image sensor that acquires the multiple sets of sample images at different exposure gains and the image sensor that acquires the image to be noise reduced are the same image sensor.

[0049] Specifically, the plurality of sets of sample images are obtained by photographing a gray board in a preset environment with an image sensor at different exposure gains.

[0050] A method for obtaining multiple sets of sample images will now be described with reference to a specific example.

[0051] (1) In a standard laboratory, a uniform gray plate 1.5 m long and 1 m wide is placed. The observation surface of the image sensor is parallel to the gray plate, the gray plate is the only object in the field of view, and the viewing angle covers 80% of the area of ​​the gray plate.

[0052] (2) A flat light source is placed at a distance of 1 to 2 m from the gray board, with its surface parallel to the surface of the gray board.

[0053] (3) The automatic exposure module of the image sensor is turned on, and the brightness of the flat light source is adjusted so that the exposure gain of the automatic exposure module is 1.

[0054] (4) 20 to 50 frames of images of the gray board are continuously captured to obtain a set of sample images with an exposure gain of 1x.

[0055] (5) Return to step (3) and capture sample images with different exposure gains, such as 2x, 4x, 8x, and 16x, by adjusting the brightness of the flat light source, to obtain multiple sets of sample images.

[0056] In step (3), the brightness of the flat light source is adjusted, and the auto exposure module uses an auto exposure control (AEC) algorithm to set the exposure gain to 1. The auto exposure control algorithm is configured to ensure that the sample image output from the image sensor is within an appropriate brightness range. For example, if the brightness of the image sensor is expressed in 8 bits, the auto exposure control algorithm is configured to ensure that the sample image output from the image sensor is within a brightness range of approximately 100, 128, and 130.

[0057] The above example is merely one embodiment of the present application, and the embodiment of the present application is not particularly limited with respect to the size of the gray plate and the coverage rate of the viewing angle.

[0058] After obtaining the sets of sample images, noise model parameters are determined according to the sets of sample images and a predetermined noise model.

[0059] In one optional embodiment, as shown in FIG. 3, step A2 may include the following steps:

[0060] Step A21: Noise reduction is performed on each set of sample images to obtain each set of noise-reduced sample images.

[0061] Step A22: For each set of sample images, determine the noise variance of each set of sample images according to each set of sample images and the noise-reduced sample images corresponding to each set, and further determine the first parameter and the second parameter corresponding to each set of sample images according to each set of sample images and the noise variance corresponding to each set.

[0062] Step A23: Determine the first hyperparameter and the second hyperparameter of the noise model according to the photosensitivity of each set of sample images and the first parameter and the second parameter corresponding to each set of sample images.

[0063] Step A24: Determine noise model parameters according to the reference photosensitivity of the image to be noise-reduced, the first hyperparameter, and the second hyperparameter.

[0064] In the embodiment of the present application, noise reduction is first performed on each set of sample image data at each exposure gain. Specifically, each set of noise-reduced sample images can be obtained through frame matching and time-domain superposition. Then, the noise variance is determined by subtracting the sample image from the corresponding set of noise-reduced sample images.

[0065] Taking the noise model as an example, the Poisson-Gaussian noise model, the Poisson-Gaussian noise model can be expressed as follows:

[0066]

number

[0067] Each set of sample images corresponds to a certain light sensitivity, which is linearly related to the first parameter a, i.e., satisfies the following equation:

[0068] a=k×ISO Among them, k is the first hyperparameter and ISO is the light sensitivity.

[0069] The photosensitivity is linearly related to the square of the second parameter b, i.e., satisfies the following equation:

[0070] b=p×ISO×ISO Among them, p is the second hyperparameter and ISO is the light sensitivity.

[0071] Therefore, by substituting the photosensitivity of each set of sample images and the first and second parameters corresponding to each set of sample images into the above two equations, the first hyperparameter and the second hyperparameter are calculated.

[0072] After determining the first hyperparameter and the second hyperparameter, if it is necessary to determine the noise model parameters a and b of the image to be noise reduced, the noise model parameters a and b of the image to be noise reduced can be calculated by substituting the reference photosensitivity of the image to be noise reduced, the first hyperparameter, and the second hyperparameter into the above two equations.

[0073] In some embodiments, the reference sensitivity is the sensitivity corresponding to the longest exposure frame in the image to be noise reduced. The high dynamic range image includes multiple single exposures with different exposure gains, and the longest exposure frame is the exposure frame with the longest exposure time and the optimal signal-to-noise ratio. By using the sensitivity corresponding to the longest exposure frame in the image to be noise reduced as the reference sensitivity and determining noise model parameters based on the reference sensitivity, the noise reduction effect can be improved when performing subsequent image noise reduction.

[0074] Step 102: Perform noise reduction processing on the first intermediate image based on a predetermined signal-to-noise ratio variation curve and a predetermined noise reduction algorithm to obtain a second intermediate image.

[0075] In the embodiment of the present application, the predetermined noise reduction algorithm may be a noise reduction algorithm according to a Gaussian distribution, such as a non-local means filter algorithm, a median filter algorithm, or a Gaussian filter algorithm, although the present application does not particularly limit this.

[0076] The signal-to-noise ratio variation curve is determined by the characteristics of the image sensor that captures the image to be noise reduced, and in some embodiments, the signal-to-noise ratio variation curve may be preset in the image noise reduction device for a particular image sensor.

[0077] In some other embodiments, as shown in FIG. 4, before step 102, a signal-to-noise ratio variation curve is determined as follows:

[0078] Step B1: Obtain exposure information for the image to be subjected to noise reduction.

[0079] Step B2: Determine the exposure ratio for each single exposure according to the exposure gain and exposure time corresponding to each single exposure.

[0080] Step B3: Determine the variation curve of the signal-to-noise ratio according to the exposure ratio corresponding to each single exposure.

[0081] In an embodiment of the present application, the image to be subjected to noise reduction includes a plurality of single exposures with different exposure gains, and the exposure information of the image to be subjected to noise reduction includes an exposure gain and an exposure time corresponding to each of the plurality of single exposures. The exposure gain and exposure time corresponding to each single exposure are set in advance before the image sensor acquires the image to be subjected to noise reduction. When determining the signal-to-noise ratio variation curve, the setting parameters of the image sensor can be directly obtained to determine the exposure gain and exposure time corresponding to each single exposure in the image to be subjected to noise reduction.

[0082] Specifically, the exposure ratio corresponding to each single exposure can be determined by the following formula:

[0083]

number

[0084] In some embodiments, it can be seen from the above that the signal-to-noise ratio of the longest exposure frame is optimal, so the exposure time corresponding to the longest exposure frame is set as the exposure time of the reference exposure, and the exposure gain corresponding to the longest exposure frame is set as the exposure gain corresponding to the reference exposure.

[0085] In the above step B3, after determining the exposure ratio corresponding to each single exposure, calculate the square root of the exposure ratio corresponding to each single exposure, and determine the variation curve of the signal-to-noise ratio according to the square root of the exposure ratio of each single exposure.

[0086] Specifically, if the exposure ratio corresponding to the first single exposure is Ratio1, the value F1 on the fluctuation curve of the signal-to-noise ratio generated by the first single exposure is expressed by the following equation.

[0087]

number

[0088]

number

[0089] Furthermore, as shown in FIG. 5, the above step 102 may include the following steps:

[0090] Step C1: Determine the similarity between each pixel point of the first intermediate image and its adjacent pixel points.

[0091] Step C2: The similarity is corrected according to the signal-to-noise ratio variation curve, and the corrected similarity of each pixel point is obtained.

[0092] Step C3: Based on the similarity of each pixel point after correction and a predetermined noise reduction algorithm, noise reduction is performed on each pixel point to obtain the second intermediate image.

[0093] The processing steps for each pixel point in the first intermediate image are the same. For convenience of explanation, the specific processing steps of steps C1 to C3 will be described below using one pixel point A in the first intermediate image as an example and the non-local means filter algorithm as the predetermined noise reduction algorithm.

[0094] In step C1, a plurality of pixel points surrounding pixel point A are determined as one pixel block, and a similarity D between the pixel block corresponding to pixel point A and the adjacent pixel blocks is calculated. The similarity D may be the average value of the difference in absolute value between the pixel block corresponding to pixel point A and the adjacent pixel blocks, or may be the Euclidean distance between the pixel block corresponding to pixel point A and the adjacent pixel blocks.

[0095] For example, pixel point A is represented by a 64-dimensional vector formed by the pixel values ​​of all pixel points within its surrounding 8x8 pixel blocks. In a 16x16 region centered on pixel point A, in addition to the 8x8 pixel blocks surrounding pixel point A, there are 80 neighboring pixel blocks. Each pixel block can be described as a 64-dimensional vector qi, where i is a number between 1 and 80, and these correspond to the 80 neighboring pixel blocks, respectively.

[0096] When the average value of the differences in absolute values ​​is used as the similarity between the pixel block corresponding to pixel point A and a plurality of adjacent pixel blocks, the similarity D can be expressed as follows.

[0097]

number

[0098] When the Euclidean distance is used as the similarity between the pixel block corresponding to pixel point A and a plurality of adjacent pixel blocks, the similarity D can be expressed as follows.

[0099]

number

[0100] In step C2, a value F on the signal-to-noise ratio variation curve generated by single exposure corresponding to pixel point A is determined using the exposure gain corresponding to the pixel point. Then, the corrected similarity D' is determined using the following formula:

[0101]

number

[0102] Step 103: Perform an inverse transformation of the variance stabilizing transformation on the second intermediate image based on the noise model parameters to obtain a noise-reduced image.

[0103] In the embodiment of the present application, the second intermediate image obtained by steps 101-102 is in the nonlinear domain, so that the second intermediate image in the nonlinear domain is transformed into the linear domain according to the noise model parameters to obtain a noise-reduced image.

[0104] Specifically, performing the inverse transformation of the variance stabilizing transformation on the second intermediate image can be determined by the following formula:

[0105]

number

[0106] As shown in Figures 6a to 6d, Figure 6a is an image linear noise model curve for an image to be noise reduced, Figure 6b is a signal-to-noise ratio variation curve, Figure 6c is an actual noise distribution for the image to be noise reduced, and Figure 6d is a noise distribution obtained by combining the image linear noise model curve for the image to be noise reduced corresponding to Figure 6a with the signal-to-noise ratio variation curve for Figure 6b. Comparing Figures 6c and 6d, it can be seen that the noise distribution obtained by combining the image linear noise model curve for the image to be noise reduced and the signal-to-noise ratio variation curve for the image to be noise reduced is very close to the actual noise distribution for the image to be noise reduced. Therefore, by using the image noise reduction method according to the embodiments of the present application, it is possible to improve the noise reduction effect.

[0107] Based on the same inventive idea, Fig. 7 is a structural block diagram of an image noise reduction device according to an embodiment of the present application. As shown in Fig. 7, the image noise reduction device 700 may include a first determination module 701, a second determination module 702, and a third determination module 703.

[0108] The first determination module 701 performs variance stabilization transformation on the image to be noise reduced based on predetermined noise model parameters to obtain a first intermediate image.

[0109] The second determination module 702 performs noise reduction processing on the first intermediate image based on a predetermined signal-to-noise ratio variation curve and a predetermined noise reduction algorithm to obtain a second intermediate image, where the signal-to-noise ratio variation curve represents difference information of noise variance corresponding to the image to be noise-reduced under different exposure gains.

[0110] A third determination module 703 performs an inverse transform of the variance stabilizing transform on the second intermediate image based on the noise model parameters to obtain a noise-reduced image.

[0111] In an optional embodiment, the apparatus further includes a fourth determination module that obtains a plurality of sets of sample images acquired by an image sensor at different exposure gains, and determines noise model parameters based on the plurality of sets of sample images and a predetermined noise model, where the image sensor is an image sensor that acquires the image to be noise reduced, and each set of sample images corresponds to one exposure gain.

[0112] In any embodiment, the plurality of sets of sample images are obtained by photographing a gray board in a pre-set environment with the image sensor at different exposure gains.

[0113] In any embodiment, specifically, the fourth determination module performs noise reduction on each set of sample images to obtain each set of noise-reduced sample images; for each set of sample images, determine the noise variance of each set of sample images based on each set of sample images and the noise-reduced sample images corresponding to each set; further determine first parameters and second parameters corresponding to each set of sample images based on each set of sample images and the noise variance corresponding to each set; determine first hyperparameters and second hyperparameters of the noise model based on the photosensitivity of each set of sample images and the first parameters and second parameters corresponding to each set of sample images; and determine the noise model parameters based on the reference photosensitivity of the image to be noise reduced, the first hyperparameters, and the second hyperparameters.

[0114] In any embodiment, the reference sensitivity of the image to be noise-reduced is the sensitivity corresponding to the longest exposure frame in the image to be noise-reduced.

[0115] In an optional embodiment, the apparatus further includes a fifth determination module that obtains exposure information of the image to be noise reduced, determines exposure ratios according to exposure gains and exposure times corresponding to each of the individual exposures, and determines the signal-to-noise ratio variation curve according to the exposure ratios corresponding to each of the individual exposures, where the exposure information includes exposure gains and exposure times corresponding to each of the individual exposures.

[0116] In any embodiment, specifically, the fifth determining module determines the exposure ratio corresponding to each single exposure respectively according to the following formula:

[0118]

number

[0119] In any embodiment, specifically, the second determination module 702 determines the similarity between each pixel point of the first intermediate image and its adjacent pixel points, corrects the similarity according to the variation curve of the signal-to-noise ratio to obtain the similarity of each pixel point after the correction, and performs noise reduction on each pixel point according to the similarity of each pixel point after the correction and the predetermined noise reduction algorithm to obtain the second intermediate image.

[0120] The image noise reduction device 700 corresponds to the above-mentioned image noise reduction method, and each functional module corresponds to each step of the above-mentioned image noise reduction method. Therefore, the embodiment of each functional module can refer to the embodiment of the image noise reduction method according to the above-mentioned example, and the description thereof will be omitted here.

[0121] 8, an embodiment of the present application further provides an electronic device 800. The electronic device 800, which can be responsible for performing the image dynamic range compression method described above, includes a processor 810 and a memory 820.

[0122] The memory 820 stores processor 810 executable instructions that, when executed by the processor 810, cause the image noise reduction method of the above embodiment to be performed.

[0123] The processor 810 can be connected to the memory 820 via a communication bus and can be connected via a communication module such as a wireless communication module, a Bluetooth (registered trademark) communication module, a Wi-Fi communication module, a 2G (second generation mobile communication system), a 3G (third generation mobile communication system), a 4G (fourth generation mobile communication system), or a 5G (fifth generation mobile communication system) communication module.

[0124] The processor 810 may be an integrated circuit chip having signal processing capabilities. The processor 810 may be a general-purpose processor including a CPU (Central Processing Unit), an NP (Network Processor), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or perform each method, step, and logic block diagram disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor.

[0125] The memory 820 may include, but is not limited to, RAM (Random Access Memory), ROM (Read Only Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), etc.

[0126] The electronic device 800 may further include its own general-purpose modules as required, the description of which will be omitted in the embodiments of the present application.

[0127] It should be noted that the embodiments of the present application further provide a computer-readable storage medium storing a computer program that, when executed by a computer, performs the steps of the image noise reduction method according to the above embodiments.

[0128] In the embodiments of the present application, the described devices and methods may be realized in other ways. The device embodiments described above are merely illustrative. For example, the division of the units described above is merely a logical functional division, and may be otherwise realized in actual practice. For example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not implemented. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be indirect couplings or communication connections via several interfaces, devices, or units, and may be electrical, mechanical, or other types of connections.

[0129] Furthermore, units described as separate components may or may not be physically separate. Elements shown as units may or may not be physical units, i.e., they may be located in the same location or distributed across multiple network units. Some or all of the units can be selected according to actual requirements to achieve the purpose of the proposed embodiment.

[0130] In addition, each functional module according to each embodiment of the present application may be an independent part formed by integration, or may be a stand-alone module, or may be an independent part formed by integration of two or more modules.

[0131] The functions may be implemented in the form of software functional modules and stored in a computer-readable storage medium when sold or used as an independent product. From this understanding, the technical solution of the present application itself, or a portion of the technical solution that contributes to the prior art, may be implemented in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of commands for causing a computer device (such as a personal computer, a server, or a network device) to execute all or part of the steps of the above-described methods in each embodiment of the present application. The storage medium may include various media capable of storing program code, such as a USB disk, a portable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0132] As used herein, relational terms such as first and second, etc., may be used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual relationship or order between such entities or actions.

[0133] The above is merely an example of the present application and does not limit the scope of protection of the present application. Those skilled in the art may make improvements or modifications to the present application. Any modifications, equivalent substitutions, or improvements made without departing from the spirit of the present application are within the scope of protection of the present application.

Claims

1. performing variance stabilizing transformation on the image to be noise reduced based on predetermined noise model parameters to obtain a first intermediate image; performing noise reduction processing on the first intermediate image based on a predetermined signal-to-noise ratio variation curve and a predetermined noise reduction algorithm to obtain a second intermediate image; and performing an inverse variance stabilizing transform on the second intermediate image based on the noise model parameters to obtain a noise-reduced image. The signal-to-noise ratio variation curve represents difference information of noise variance corresponding to the image to be subjected to noise reduction at different exposure gains; Before performing a variance stabilizing transformation on the image to be noise reduced based on predetermined noise model parameters to obtain a first intermediate image, obtaining a plurality of sets of sample images acquired by an image sensor at different exposure gains; determining noise model parameters based on the plurality of sets of sample images and a predetermined noise model; the image sensor is an image sensor that acquires the image to be subjected to noise reduction, and each set of sample images corresponds to one exposure gain; determining noise model parameters based on the plurality of sets of sample images and a predetermined noise model, performing noise reduction on each set of sample images to obtain a respective set of noise-reduced sample images; For each set of sample images, determining a noise variance of each set of sample images according to each set of sample images and the noise-reduced sample images corresponding to each set, and further determining a first parameter and a second parameter corresponding to each set of sample images according to each set of sample images and the noise variance corresponding to each set; determining a first hyperparameter and a second hyperparameter of the noise model according to the photosensitivity of each set of sample images and a first parameter and a second parameter respectively corresponding to each set of sample images; determining the noise model parameters based on a reference photosensitivity of the image to be subjected to noise reduction, the first hyperparameter, and the second hyperparameter; 1. A method for reducing image noise.

2. The plurality of sets of sample images are obtained by photographing a gray board in a preset environment with the image sensor at different exposure gains.

2. The method of claim 1 .

3. The reference sensitivity of the image to be subjected to noise reduction is the sensitivity corresponding to the longest exposure frame in the image to be subjected to noise reduction.

2. The method of claim 1 .

4. before the step of performing noise reduction processing on the first intermediate image based on a predetermined signal-to-noise ratio variation curve and a predetermined noise reduction algorithm to obtain a second intermediate image; obtaining exposure information of the image to be subjected to noise reduction; determining a respective exposure ratio by an exposure gain and an exposure time corresponding to each single exposure; determining a variation curve of the signal-to-noise ratio according to an exposure ratio corresponding to each single exposure; The exposure information includes an exposure gain and an exposure time corresponding to each of a plurality of single exposures.

2. The method of claim 1 .

5. determining a respective exposure ratio by an exposure gain and an exposure time corresponding to each single exposure, determining an exposure ratio corresponding to each of the single exposures based on the following formula: [Equation 1] Among them, Ratio ｉ is the exposure ratio corresponding to the i-th single exposure, and exp ｉ is the exposure time corresponding to the i-th single exposure, and gain ｉ is the exposure gain corresponding to the i-th single exposure, EXP is the exposure time of the reference exposure, and GAIN is the exposure gain corresponding to the reference exposure.

5. The method of claim 4.

6. The step of performing noise reduction processing on the first intermediate image based on a predetermined signal-to-noise ratio variation curve and a predetermined noise reduction algorithm to obtain a second intermediate image includes: determining a similarity between each pixel point of the first intermediate image and its neighboring pixel points; correcting the similarity using the signal-to-noise ratio variation curve to obtain a corrected similarity for each pixel point; and performing noise reduction on each pixel point based on the similarity of each pixel point after correction and the predetermined noise reduction algorithm to obtain the second intermediate image.

2. The method of claim 1 .

7. a first determination module for performing variance stabilization transformation on the image to be noise reduced based on predetermined noise model parameters to obtain a first intermediate image; a second determination module for performing noise reduction processing on the first intermediate image based on a predetermined signal-to-noise ratio variation curve and a predetermined noise reduction algorithm to obtain a second intermediate image; a third determination module for performing an inverse variance stabilizing transform on the second intermediate image based on the noise model parameters to obtain a noise-reduced image. The signal-to-noise ratio variation curve represents difference information of noise variance corresponding to the image to be subjected to noise reduction at different exposure gains; the first determination module performs variance stabilization transformation on the image to be noise reduced based on predetermined noise model parameters; before the step of obtaining a first intermediate image, obtain a plurality of sets of sample images captured by an image sensor with different exposure gains; and determine noise model parameters based on the plurality of sets of sample images and a predetermined noise model, the image sensor being the image sensor that captures the image to be noise reduced, and each set of sample images corresponds to one exposure gain; The first determination module: performing noise reduction on each set of sample images to obtain each set of noise-reduced sample images; For each set of sample images, determine a noise variance of each set of sample images according to each set of sample images and the noise-reduced sample images corresponding to each set; and further determine a first parameter and a second parameter corresponding to each set of sample images according to each set of sample images and the noise variance corresponding to each set; Determine a first hyperparameter and a second hyperparameter of the noise model according to the photosensitivity of each set of sample images and a first parameter and a second parameter respectively corresponding to each set of sample images; The noise model parameters are determined based on a reference photosensitivity of the image to be subjected to noise reduction, the first hyperparameter, and the second hyperparameter.

1. An image noise reduction device comprising:

8. a processor and a memory storing program commands executable by said processor; The processor invokes the program commands that are capable of carrying out the method according to any one of claims 1 to 6. An electronic device characterized by:

9. storing computer program commands which, when called and executed by a computer, carry out the method according to any one of claims 1 to 6; A computer-readable storage medium comprising:

Images