Image blind denoising method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a kind of image blind denoising method, device, electronic equipment and storage medium.The image blind denoising method includes: according to the image noise calibration result obtained by preimage noise calibration to the image acquisition device of to-be-de-noised image, determine the target noise parameter of to-be-de-noised image;To-be-de-noised image is carried out preliminary filtering processing, and the preliminary filtering image of to-be-de-noised image is obtained;According to target noise parameter and preliminary filtering image, determine the noise level estimation result of to-be-de-noised image;According to noise level estimation result, to-be-de-noised image is carried out final denoising processing, and the final blind denoising result of to-be-de-noised image is obtained.The embodiments of the present application realize noise level estimation by the method that image noise calibration and filtering are combined, reduce the overall complexity of blind denoising method, to reduce the deployment difficulty of blind denoising method on the platform of limited power such as front-end image acquisition device.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, apparatus, electronic device and storage medium for blind image denoising. Background Technology

[0002] During image acquisition, noise is inevitably present in the acquired images due to the physical constraints of the acquisition device and the limitations of the external lighting environment, thus affecting image quality. Therefore, existing image acquisition devices employ image denoising techniques to improve their image quality. However, the complexity and diversity of real-world noise, along with the differences in the real-world noise models of each acquisition device, make the removal of real-world noise quite challenging. Current requirements for image denoising include achieving good denoising results when processing input images with varying noise levels, i.e., achieving blind denoising.

[0003] Existing blind denoising methods generally employ a Convolutional Neural Network (CNN)-based approach, which typically involves two steps: First, a CNN sub-network estimates the noise level of the input image to be denoised, obtaining a noise level map. Second, the noise level map is merged with the image to be denoised and then input into another CNN sub-network for processing, yielding the blind denoising result. This combination of noise level estimation and CNN denoising gives the model excellent blind denoising capabilities. However, this method suffers from high overall model complexity, requiring the simultaneous deployment of two CNN networks, which places high demands on the computing power of the hardware platform. This increases the difficulty of deploying the model on computationally limited platforms such as front-end image acquisition devices. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for blind image denoising. By combining image noise calibration and filtering, noise level estimation is achieved, reducing the overall complexity of the blind denoising method.

[0005] In a first aspect, embodiments of the present invention provide a blind image denoising method, comprising:

[0006] Based on the image noise calibration results obtained by performing image noise calibration on the image acquisition device of the image to be denoised, the target noise parameters of the image to be denoised are determined.

[0007] The image to be denoised is subjected to preliminary filtering to obtain a preliminary filtered image of the image to be denoised.

[0008] The noise level estimation result of the image to be denoised is determined based on the target noise parameters and the preliminary filtered image;

[0009] Based on the noise level estimation result, the image to be denoised is subjected to final denoising processing to obtain the final blind denoising result of the image to be denoised.

[0010] Secondly, embodiments of the present invention also provide an image blind denoising apparatus, comprising:

[0011] The image noise parameter determination module is used to determine the target noise parameters of the image to be denoised based on the image noise calibration results obtained by performing image noise calibration on the image acquisition device of the image to be denoised in advance.

[0012] The image preliminary filtering module is used to perform preliminary filtering processing on the image to be denoised to obtain a preliminary filtered image of the image to be denoised.

[0013] A noise level estimation module is used to determine the noise level estimation result of the image to be denoised based on the target noise parameters and the preliminary filtered image;

[0014] The blind denoising module is used to perform final denoising processing on the image to be denoised based on the noise level estimation result, so as to obtain the final blind denoising result of the image to be denoised.

[0015] Thirdly, embodiments of the present invention also provide an electronic device, comprising:

[0016] One or more processors;

[0017] Storage device for storing one or more programs.

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the image blind denoising method as described in any embodiment of the present invention.

[0019] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image blind denoising method as described in any embodiment of the present invention.

[0020] This invention, in its embodiments, determines the target noise parameters of the image to be denoised based on the image noise calibration results obtained from the image acquisition device performing image noise calibration on the image to be denoised. It then performs preliminary filtering on the image to be denoised to obtain a preliminary filtered image. Based on the target noise parameters and the preliminary filtered image, it determines the noise level estimation result of the image to be denoised. Finally, it performs final denoising on the image to be denoised based on the noise level estimation result, obtaining the final blind denoising result. This invention combines image noise calibration with traditional filtering to achieve image noise level estimation, improving the efficiency of image noise level estimation. While maintaining the blind denoising effect, it reduces the overall complexity of the blind denoising method, thereby reducing the deployment difficulty of the blind denoising method on platforms with limited computing power, such as front-end image acquisition devices. Attached Figure Description

[0021] Figure 1 This is a flowchart of the image blind denoising method in Embodiment 1 of the present invention;

[0022] Figure 2 This is a schematic diagram of the image blind denoising device in Embodiment 2 of the present invention;

[0023] Figure 3 This is a schematic diagram of the electronic device in Embodiment 3 of the present invention. Detailed Implementation

[0024] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0025] Example 1

[0026] Figure 1 This is a flowchart of the blind image denoising method in Embodiment 1 of the present invention. This embodiment is applicable to blind denoising of images to be denoised. The method can be executed by an image blind denoising device, which can be implemented in software and / or hardware and can be configured in an electronic device, such as a backend server or other device with communication and computing capabilities. Figure 1 As shown, the method specifically includes:

[0027] Step 101: Determine the target noise parameters of the image to be denoised based on the image noise calibration results obtained by the image acquisition device of the image to be denoised in advance.

[0028] Blind denoising refers to denoising algorithms that can effectively denoise input images with varying noise levels. In this embodiment of the invention, a combination of noise level estimation and denoising algorithms is used to ensure blind denoising capability. When determining the noise level estimation result of the image to be denoised, a combination of image noise calibration and traditional filtering is employed. Image noise calibration determines the target noise parameters of the image to be denoised, whereby the target noise parameters characterize the noise model introduced by the image acquisition device into the acquired image. Since different models of image acquisition devices use different sensors, their noise models also differ. Therefore, in this embodiment of the invention, image noise calibration is performed beforehand for different models of image acquisition devices to obtain their corresponding noise models and characterize the noise in the acquired image. In this embodiment of the invention, only one pre-calibration is needed for each type of image acquisition device. After calibration, the noise model of that type of image acquisition device is determined, eliminating the need for repeated calibration.

[0029] Specifically, at least one type of image acquisition device is pre-determined for image acquisition, and an image noise calibration is performed on it to obtain the corresponding image noise calibration result. The image noise calibration result expresses the degree of influence of the image acquisition device on the noise level in the image during image acquisition. For example, since the image acquisition device has adjustable device parameters during image acquisition, the noise level of the acquired image varies under different device parameters. Therefore, the image noise calibration result obtained by the image acquisition device reflects the influence of different device parameters on the noise of the acquired image. Based on this, the device parameters of the image acquisition device during the acquisition of the image to be denoised are determined, and the target noise parameters of the image to be denoised are determined according to the device parameters and the image noise calibration result.

[0030] For example, the image noise calibration result obtained by performing image noise calibration on the image acquisition device of the image to be denoised includes the mapping relationship between different device parameters and noise parameters. At the same time, the target device parameters of the image to be denoised are determined, and the target noise parameters mapped by the target device parameters are determined according to the mapping relationship.

[0031] In one feasible embodiment, the image noise calibration result includes at least one candidate exposure gain value and an associated candidate noise parameter;

[0032] Accordingly, step 101 includes:

[0033] Determine the target exposure gain value of the image acquisition device when acquiring the image to be denoised;

[0034] Based on the relationship between the target exposure gain value and the candidate exposure gain value, the target noise parameter associated with the target noise parameter is determined according to the associated candidate noise parameter.

[0035] Here, candidate exposure gain values ​​refer to pre-determined acquisition parameter values ​​of the image acquisition device. The specific number and value of candidate exposure gain values ​​can be pre-determined based on the device attributes of the image acquisition device, and are not limited here. For example, candidate exposure gain values ​​are pre-determined as a set of exposure gain values ​​{ISO...} j ;j=1~s}, that is, set s candidate exposure gain values, perform image noise calibration once under each exposure gain value, and obtain candidate noise parameters under each candidate exposure gain value. That is, the image noise calibration result includes the mapping relationship between candidate exposure gain values ​​and candidate noise parameters.

[0036] When determining the target noise parameter of the image to be denoised, the exposure gain value of the image acquisition device when acquiring the image to be denoised is first determined as the target exposure gain value. Since the number of candidate exposure gain values ​​is limited, it is impossible to exhaust all the exposure gain values ​​of the image acquisition device. Therefore, if the target noise parameter is not among the candidate exposure gain values, an interpolation calculation method is used to determine the target noise parameter associated with the target noise parameter based on the relationship between the target exposure gain value and the candidate exposure gain values ​​and the associated candidate noise parameters.

[0037] For example, the image noise calibration result includes a set of candidate exposure gain values ​​{ISO 1000}. j ;j=1~s} and the candidate noise parameter {a} associated with each candidate exposure gain value j,i ;j=1~s,i=0~n}, meaning that each candidate exposure gain value is associated with n+1 candidate noise parameters. Let the target exposure gain value be ISO x If ISO x For {ISO j If any one of {a,j = 1 to s} is selected, then the target noise parameter can be directly obtained from {a}. j,i The determination is made within the range of {j = 1 to s, i = 0 to n}. If ISO x Not {ISO j If any one of j = 1 to s is used, then the current target exposure gain value ISO is accurately solved using linear interpolation. x Associated target noise parameters {a x,i ; i = 0 ~ n}. For example, the target noise parameter {a} is determined according to the following formula. x,i ;i=0~n}:

[0038]

[0039] In one feasible embodiment, the image acquisition device for the image to be denoised is pre-calibrated for image noise, including:

[0040] At least two images of the same scene to be calibrated are acquired using an image acquisition device at each candidate exposure gain value; wherein, the candidate exposure gain value includes at least one.

[0041] Candidate noise parameters for each candidate exposure gain value are determined based on at least two images associated with each candidate exposure gain value. The image noise calibration result of the image acquisition device is determined by at least one candidate exposure gain value and the associated candidate noise parameters.

[0042] When performing image noise calibration in advance, it is necessary to calibrate each candidate exposure gain value, that is, to obtain the candidate noise parameters under each candidate exposure gain value.

[0043] Specifically, a set of candidate exposure gain values ​​{ISO 1000} are pre-determined based on the image acquisition equipment. j For each candidate exposure gain value, multiple images captured by the image acquisition device are used as the image data to be calibrated. Since the multiple images are acquired under the same exposure gain value and the same shooting scene, the noise level under the current exposure gain value can be determined by the differences between these multiple images, and thus the corresponding candidate noise parameters can be determined. To ensure that the obtained calibration image data includes information on various brightness levels, the shooting scene needs to include different brightness information. For example, the shooting scene can be a standard grayscale card.

[0044] For example, in a laboratory setting with controllable ambient brightness, the camera image is aligned with a standard grayscale chart to ensure sharp focus, and the image remains static during calibration. Candidate exposure gain values ​​are set as {100, 400, 1600, 6400, 25600, 102400}. At each candidate exposure gain value, the camera shutter speed or ambient illumination is manually adjusted to achieve moderate image brightness. While maintaining constant image brightness, m frames of image data are continuously acquired as the image data to be calibrated. The noise level of the image is determined based on the m frames of image data at each candidate exposure gain value, for example, by determining the relationship between the differences between each pixel and the pixel brightness, thus determining the candidate noise parameters for each candidate exposure gain value. Finally, the image noise calibration result is constructed based on all candidate exposure gain values ​​and their associated candidate noise parameters.

[0045] For example, when there is only one candidate exposure gain value, that is, the image noise calibration result includes only one candidate exposure gain value and one candidate noise parameter, when determining the target noise parameter of the image to be denoised, the candidate noise parameter can be directly identified as the target noise parameter; or the target noise parameter can be determined according to the relationship between the candidate exposure gain value and the target exposure gain value. For example, the target noise parameter can be obtained by proportionally calculating the candidate noise parameter according to the proportional relationship between the candidate exposure gain value and the target exposure gain value.

[0046] In one feasible embodiment, candidate noise parameters for each candidate exposure gain value are determined based on at least two images of the image to be calibrated associated with that candidate exposure gain value, including:

[0047] The expression for determining the image noise level of the image acquisition device is: Where x represents the target pixel brightness value, V(x) represents the noise variance characterizing the noise level of the target pixel, n represents the polynomial series, and a i The noise parameters to be calibrated;

[0048] The average calibration image is determined based on the pixel average value at each pixel of at least two images to be calibrated associated with the candidate exposure gain value;

[0049] The variance calibration image is determined based on the variance of pixel values ​​at each pixel point of at least two images to be calibrated associated with candidate exposure gain values;

[0050] From the average calibration image and the variance calibration image, at least n+1 pixel pairs with different pixel values ​​are determined. Based on the image noise level expression, the values ​​of the noise parameters to be calibrated are determined according to the pixel pairs, which are candidate noise parameters for candidate exposure gain values.

[0051] The image noise level expression characterizes the functional relationship between image noise variance (a quantized value of the noise level) and pixel brightness. Existing research shows that image noise acquired by image acquisition devices generally follows a Poisson-Gaussian distribution, and the functional relationship between its noise variance and pixel brightness can be represented by the following linear function:

[0052] V(x) = k·x + σ 2 ;

[0053] In this equation, the first term on the right-hand side is a Poisson noise term, whose noise variance is proportional to the pixel brightness; the second term is a Gaussian noise term; and the parameters k and σ... 2 The noise level of an image is determined by the exposure gain of the image acquisition device. However, the above model has certain biases, such as a high-order nonlinear relationship between noise variance and pixel brightness. To more accurately reflect the actual noise model, this embodiment of the invention uses a polynomial function to describe the relationship between noise variance and pixel brightness:

[0054]

[0055] The series n of the polynomial function can be set according to the actual noise characteristics of the image acquisition device and the user's needs. For example, n = 2 is a reasonable value; the parameter {a i;i=0~n} is the noise parameter to be calibrated, which is related to the exposure gain value of the image acquisition device; V(x) represents the noise variance, which is used to quantify the noise level of each pixel in the image.

[0056] Based on the above example, we determine m frames of image data to be calibrated under each candidate exposure gain, calculate the average value of each pixel in the m consecutive frames of image data, and obtain the average calibration image I; simultaneously, we calculate the variance of each pixel in the m consecutive frames of image data, and obtain the variance calibration image V. According to the above expression for image noise level, for any pixel value x in the average calibration image I... p The pixel value V(x) at the corresponding position in the variance-calibrated image V. p It should satisfy the functional relationship in the expression for the image noise level, that is... Since the number of noise parameters to be calibrated needs to be determined in the image noise level expression is n+1, at least n+1 pixel values ​​with different x values ​​need to be determined from the average calibration image and the variance calibration image. p -V(x p Pixel pairs; the specific pixel values ​​selected can be determined based on the actual situation and are not limited here. Finally, x... p -V(x p The specific values ​​of pixel pairs are substituted into the image noise level expression to solve for the (n+1) unknowns, yielding the values ​​of the noise parameters to be calibrated. The candidate noise parameters for the remaining candidate exposure gain values ​​are then calculated sequentially using this method.

[0057] When determining the values ​​of noise parameters to be calibrated based on pixel pairs, the least squares method can be used. For example, suppose t pixels x with different values ​​are selected from images I and V. p -V(x p For any pair of pixels satisfying t≥n+1, the image noise level expression can be solved using the following formula:

[0058] make: Then A = (X) T X) -1 X T V.

[0059] The obtained image noise calibration results include a set of candidate exposure gain values ​​{ISO j ;j=1~s}, and the candidate noise parameter {a} for each candidate exposure gain value. j,i ;j=1~s,i=0~n},where the noise parameter a j,i This represents the i-th order noise parameter under the j-th candidate exposure gain value.

[0060] Step 102: Perform preliminary filtering on the image to be denoised to obtain a preliminary filtered image of the image to be denoised.

[0061] The preliminary filtering process for the image to be denoised aims to smooth the image and eliminate the interference of noise in the noise level estimation calculation. For example, based on the above example, the image noise level expression of the image acquisition device is a function of pixel brightness value and pixel variance. Therefore, to determine the pixel variance of the image to be denoised, it is necessary to determine the pixel brightness value, i.e., eliminate the interference factors of noise in the pixel value. This step does not require high image denoising performance; therefore, simple and efficient conventional image filtering methods are sufficient. The preliminary filtering method here can be Gaussian filtering, mean filtering, median filtering, bilateral filtering, guided filtering, etc., and is not limited here.

[0062] In one feasible embodiment, the preliminary filtering process employs Gaussian filtering, mean filtering, median filtering, bilateral filtering, or guided filtering.

[0063] Specifically, in order to better preserve the detailed features in the filtered image, the initial filtering process can use edge-preserving filters such as bilateral filtering or guided filtering, which can improve the image quality after subsequent denoising.

[0064] Step 103: Determine the noise level estimation result of the image to be denoised based on the target noise parameters and the preliminary filtered image.

[0065] The target noise parameter characterizes the relationship between the noise level and the pixel value in the image. Therefore, the noise level estimation result of the image to be denoised can be determined based on the target noise parameter and the preliminary filtered image after removing the initial noise effect.

[0066] In one feasible embodiment, step 103 includes:

[0067] The expression for determining the image noise level of the image acquisition device is: Where, x p V represents the pixel value of the target pixel in the initially filtered image. p This represents the noise level of the pixel in the image to be denoised that is associated with the target pixel in the initial filtered image, where n represents the series of the polynomial, and a x,i The target noise parameters of the image to be denoised are defined; the initial filtered image and the image to be denoised are of the same size.

[0068] Based on the image noise level expression, the noise level of each pixel is determined according to the target noise parameters and the pixel value of each pixel in the preliminary filtered image, resulting in a noise level map, which is the noise level estimation result of the image to be denoised.

[0069] The noise level map represents the estimated noise variance of each pixel in the input image to be denoised. As mentioned above, a polynomial function is used to describe the relationship between noise variance and pixel brightness, thus determining the image noise level expression of the image acquisition device. Since the target noise parameters are determined, {a} in the image noise level expression represents... x,i Once the initial filtered image is determined, x in the image noise level expression is determined; i = 0 to n}. p The series n of the polynomial is predetermined when determining the candidate noise parameters.

[0070] Since the initial filtered image and the image to be denoised are the same size, any pixel x in the initial filtered image... p The pixel values ​​are input into a defined image noise level expression to obtain the noise level V. p For noise level graphs and x p The pixel values ​​at the same location. By iterating through each pixel value in the initial filtered image, a noise level map is obtained, representing the noise level estimation result of the image to be denoised.

[0071] Step 104: Perform final denoising processing on the image to be denoised based on the noise level estimation results to obtain the final blind denoising result of the image to be denoised.

[0072] Blind denoising is performed on the image based on the noise level estimation result, so as to achieve targeted denoising according to the actual noise situation at each location in the image and improve the denoising effect.

[0073] Specifically, the local denoising intensity of the image to be denoised is determined based on the noise level estimation result, and the final denoising process is performed on the image to be denoised based on the local denoising intensity to obtain the final blind denoising result of the image to be denoised.

[0074] In one feasible embodiment, step 104 includes:

[0075] The noise level map and the image to be denoised are concatenated along the channel dimension to obtain the merged image;

[0076] The merged image is denoised using a pre-trained image noise removal network to obtain the final blind denoising result of the image to be denoised.

[0077] The pre-trained image noise removal network is a CNN network, trained using a pre-acquired dataset. The specific network structure is not limited; for example, CNN denoising network models with architectures such as U-Net and DnCNN can be used. Specific implementation details will not be elaborated further.

[0078] Specifically, the image to be denoised is merged with a noise level map and then input into a preset image noise removal network for denoising. In this way, the image noise removal network utilizes the noise level distribution information of the image to be denoised, achieving better blind denoising results. The final blind denoising result of the image to be denoised has the same image size as the input image.

[0079] To avoid losing image information, the merging of the image to be denoised and the noise level map is achieved by stitching along the channel dimension. For example: let the width of the image to be denoised be w, the height be h, and the number of channels be c. The noise level map has the same dimensions as the image to be denoised, so its width, height, and number of channels are also w, h, and c, respectively. The two images are stitched together along the channel dimension, that is, stacked along the channel dimension, resulting in a merged image with a width of w, a height of h, and 2*c channels.

[0080] This invention combines image noise calibration with traditional filtering to estimate the noise level of an image, thereby improving the efficiency of image noise level estimation. While maintaining the effect of blind denoising, it reduces the overall complexity of the blind denoising method, thus reducing the difficulty of deploying the blind denoising method on platforms with limited computing power, such as front-end image acquisition devices.

[0081] Example 2

[0082] Figure 2 This is a schematic diagram of the image blind denoising device in Embodiment 2 of the present invention. This embodiment is applicable to situations where blind denoising is performed on images to be denoised. Figure 2 As shown, the device includes:

[0083] The image noise parameter determination module 210 is used to determine the target noise parameters of the image to be denoised based on the image noise calibration results obtained by performing image noise calibration on the image acquisition device of the image to be denoised in advance.

[0084] The image preliminary filtering module 220 is used to perform preliminary filtering processing on the image to be denoised to obtain a preliminary filtered image of the image to be denoised.

[0085] The noise level estimation module 230 is used to determine the noise level estimation result of the image to be denoised based on the target noise parameters and the preliminary filtered image;

[0086] The blind denoising module 240 is used to perform final denoising processing on the image to be denoised based on the noise level estimation result, so as to obtain the final blind denoising result of the image to be denoised.

[0087] This invention combines image noise calibration with traditional filtering to estimate the noise level of an image, thereby improving the efficiency of image noise level estimation. While maintaining the effect of blind denoising, it reduces the overall complexity of the blind denoising method, thus reducing the difficulty of deploying the blind denoising method on platforms with limited computing power, such as front-end image acquisition devices.

[0088] Optionally, the device includes an image noise calibration module, comprising:

[0089] The calibration data acquisition unit is used to acquire at least two images of the same shooting scene through the image acquisition device at each candidate exposure gain value; wherein, the candidate exposure gain value includes at least one;

[0090] The noise parameter determination unit is used to determine the candidate noise parameter of the candidate exposure gain value based on the at least two image data to be calibrated associated with each candidate exposure gain value, and to determine the image noise calibration result of the image acquisition device by at least one candidate exposure gain value and the associated candidate noise parameter.

[0091] Optional, noise parameter determination unit, specifically used for:

[0092] The image noise level of the image acquisition device is determined by the following expression: Where x represents the target pixel brightness value, V(x) represents the noise variance characterizing the noise level of the target pixel, n represents the polynomial series, and a i The noise parameters to be calibrated;

[0093] An average calibration image is determined based on the pixel average value at each pixel of the at least two images to be calibrated associated with the candidate exposure gain value;

[0094] Based on the variance of pixel values ​​at each pixel point of the at least two images to be calibrated associated with the candidate exposure gain values, a variance calibration image is determined;

[0095] From the average calibration image and the variance calibration image, at least n+1 pixel pairs with different pixel values ​​are determined. Based on the image noise level expression, the value of the noise parameter to be calibrated is determined according to the pixel pairs, which are the candidate noise parameters for the candidate exposure gain value.

[0096] Optionally, the image noise calibration result includes at least one candidate exposure gain value and an associated candidate noise parameter;

[0097] Correspondingly, the image noise parameter determination module is specifically used for:

[0098] Determine the target exposure gain value of the image acquisition device when acquiring the image to be denoised;

[0099] Based on the relationship between the target exposure gain value and the candidate exposure gain value, the target noise parameter associated with the target noise parameter is determined according to the associated candidate noise parameter.

[0100] Optional, a noise level estimation module, specifically used for:

[0101] The image noise level of the image acquisition device is determined by the following expression: Where, x p V represents the pixel value of the target pixel in the preliminary filtered image. p The noise level of the pixel in the image to be denoised that is associated with the target pixel in the preliminary filtered image is represented by n, where n represents the series of the polynomial, and a x,i The target noise parameters of the image to be denoised are defined as follows; wherein, the preliminary filtered image and the image to be denoised are of the same size.

[0102] Based on the image noise level expression, the noise level of each pixel is determined according to the target noise parameter and the pixel value of each pixel in the preliminary filtered image, resulting in a noise level map, which is the noise level estimation result of the image to be denoised.

[0103] Optional blind denoising module, specifically used for:

[0104] The noise level map and the image to be denoised are concatenated along the channel dimension to obtain a merged image;

[0105] The merged image is denoised using a pre-trained image noise removal network to obtain the final blind denoising result of the image to be denoised.

[0106] Optionally, the preliminary filtering process may employ Gaussian filtering, mean filtering, median filtering, bilateral filtering, or guided filtering.

[0107] The image blind denoising apparatus provided in the embodiments of the present invention can execute the image blind denoising method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the image blind denoising method.

[0108] Example 3

[0109] Figure 3 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Figure 3 A block diagram is shown of an exemplary electronic device 12 suitable for implementing embodiments of the present invention. Figure 3 The electronic device 12 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0110] like Figure 3 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system storage device 28, and bus 18 connecting different system components (including system storage device 28 and processing unit 16).

[0111] Bus 18 represents one or more of several bus architectures, including a memory device bus or memory device controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0112] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0113] System storage device 28 may include computer system readable media in the form of volatile storage devices, such as random access memory (RAM) 30 and / or cache storage device 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 3 Not shown; usually referred to as a "hard drive"). Although Figure 3 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Storage device 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0114] A program / utility 40 having a set (at least one) of program modules 42 may be stored in, for example, storage device 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0115] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with device 12, and / or with any device that enables device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. Figure 3 As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although... Figure 3 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0116] Processing unit 16 executes various functional applications and data processing by running programs stored in system storage device 28, such as implementing the image blind denoising method provided in the embodiments of the present invention, including:

[0117] Based on the image noise calibration results obtained by performing image noise calibration on the image acquisition device of the image to be denoised, the target noise parameters of the image to be denoised are determined.

[0118] The image to be denoised is subjected to preliminary filtering to obtain a preliminary filtered image of the image to be denoised.

[0119] The noise level estimation result of the image to be denoised is determined based on the target noise parameters and the preliminary filtered image;

[0120] Based on the noise level estimation result, the image to be denoised is subjected to final denoising processing to obtain the final blind denoising result of the image to be denoised.

[0121] Example 4

[0122] Embodiment 4 of the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the image blind denoising method provided in the embodiments of the present invention, including:

[0123] Based on the image noise calibration results obtained by performing image noise calibration on the image acquisition device of the image to be denoised, the target noise parameters of the image to be denoised are determined.

[0124] The image to be denoised is subjected to preliminary filtering to obtain a preliminary filtered image of the image to be denoised.

[0125] The noise level estimation result of the image to be denoised is determined based on the target noise parameters and the preliminary filtered image;

[0126] Based on the noise level estimation result, the image to be denoised is subjected to final denoising processing to obtain the final blind denoising result of the image to be denoised.

[0127] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0128] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0129] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0130] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0131] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A blind image denoising method, characterized in that, include: Based on the image noise calibration results obtained by performing image noise calibration on the image acquisition device of the image to be denoised, the target noise parameters of the image to be denoised are determined. The image to be denoised is subjected to preliminary filtering to obtain a preliminary filtered image of the image to be denoised. The noise level estimation result of the image to be denoised is determined based on the target noise parameters and the preliminary filtered image; The image to be denoised is subjected to final denoising processing based on the noise level estimation result to obtain the final blind denoising result of the image to be denoised. The image noise calibration result includes at least one candidate exposure gain value and an associated candidate noise parameter; Accordingly, determining the target noise parameters of the image to be denoised based on the image noise calibration results obtained by pre-calibrating the image acquisition device of the image to be denoised includes: Determine the target exposure gain value of the image acquisition device when acquiring the image to be denoised; Based on the relationship between the target exposure gain value and the candidate exposure gain value, the target noise parameter associated with the target noise parameter is determined according to the associated candidate noise parameter; where the candidate exposure gain value refers to the acquisition parameter value of the image acquisition device in advance, and the specific number and specific value of the candidate exposure gain value are determined in advance according to the device attributes of the image acquisition device; The step of determining the noise level estimation result of the image to be denoised based on the target noise parameters and the preliminary filtered image includes: The image noise level of the image acquisition device is determined by the following expression: ;in, This represents the pixel value of the target pixel in the preliminary filtered image. The noise level of the pixel in the image to be denoised that is associated with the target pixel in the preliminary filtered image is represented by n, where n represents the series of the polynomial. The target noise parameters of the image to be denoised are defined as follows; wherein, the preliminary filtered image and the image to be denoised are of the same size. Based on the image noise level expression, the noise level of each pixel is determined according to the target noise parameter and the pixel value of each pixel in the preliminary filtered image, resulting in a noise level map, which is the noise level estimation result of the image to be denoised.

2. The method according to claim 1, characterized in that, Pre-calibration of image noise in the image acquisition device for the image to be denoised includes: At least two images to be calibrated are acquired by the image acquisition device for the same shooting scene at each candidate exposure gain value; wherein, the candidate exposure gain value includes at least one; Candidate noise parameters for each candidate exposure gain value are determined based on the at least two images to be calibrated associated with each candidate exposure gain value. The image noise calibration result of the image acquisition device is determined by at least one candidate exposure gain value and the associated candidate noise parameters.

3. The method according to claim 2, characterized in that, Determine candidate noise parameters for each candidate exposure gain value based on the at least two images to be calibrated associated with that candidate exposure gain value, including: The image noise level of the image acquisition device is determined by the following expression: ;in, This represents the target pixel brightness value. Let represent the noise variance used to characterize the noise level of the target pixel, and n represent the series of the polynomial. The noise parameters to be calibrated; An average calibration image is determined based on the pixel average value at each pixel of the at least two images to be calibrated associated with the candidate exposure gain value; Based on the variance of pixel values ​​at each pixel point of the at least two images to be calibrated associated with the candidate exposure gain values, a variance calibration image is determined; From the average calibration image and the variance calibration image, at least n+1 pixel pairs with different pixel values ​​are determined. Based on the image noise level expression, the value of the noise parameter to be calibrated is determined according to the pixel pairs, which are the candidate noise parameters for the candidate exposure gain value.

4. The method according to claim 1, characterized in that, Based on the noise level estimation result, the image to be denoised is subjected to final denoising processing to obtain the final blind denoising result of the image to be denoised, including: The noise level map and the image to be denoised are concatenated along the channel dimension to obtain a merged image; The merged image is denoised using a pre-trained image noise removal network to obtain the final blind denoising result of the image to be denoised.

5. The method according to claim 1, characterized in that, The preliminary filtering process employs Gaussian filtering, mean filtering, median filtering, bilateral filtering, or guided filtering.

6. An image blind denoising device, characterized in that, include: The image noise parameter determination module is used to determine the target noise parameters of the image to be denoised based on the image noise calibration results obtained by performing image noise calibration on the image acquisition device of the image to be denoised in advance. The image preliminary filtering module is used to perform preliminary filtering processing on the image to be denoised to obtain a preliminary filtered image of the image to be denoised. A noise level estimation module is used to determine the noise level estimation result of the image to be denoised based on the target noise parameters and the preliminary filtered image; A blind denoising module is used to perform final denoising processing on the image to be denoised based on the noise level estimation result, so as to obtain the final blind denoising result of the image to be denoised. The image noise calibration result includes at least one candidate exposure gain value and an associated candidate noise parameter; Accordingly, the image noise parameter determination module is specifically used for: Determine the target exposure gain value of the image acquisition device when acquiring the image to be denoised; Based on the relationship between the target exposure gain value and the candidate exposure gain value, the target noise parameter associated with the target noise parameter is determined according to the associated candidate noise parameter; where the candidate exposure gain value refers to the acquisition parameter value of the image acquisition device in advance, and the specific number and specific value of the candidate exposure gain value are determined in advance according to the device attributes of the image acquisition device; The noise level estimation module is specifically used for: The image noise level of the image acquisition device is determined by the following expression: ;in, This represents the pixel value of the target pixel in the preliminary filtered image. The noise level of the pixel in the image to be denoised that is associated with the target pixel in the preliminary filtered image is represented by n, where n represents the series of the polynomial. The target noise parameters of the image to be denoised are defined as follows; wherein, the preliminary filtered image and the image to be denoised are of the same size. Based on the image noise level expression, the noise level of each pixel is determined according to the target noise parameter and the pixel value of each pixel in the preliminary filtered image, resulting in a noise level map, which is the noise level estimation result of the image to be denoised.

7. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the image blind denoising method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the image blind denoising method as described in any one of claims 1-5.

Images