Image denoising method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of image processing, and discloses an image denoising method and device, electronic equipment and a storage medium. The electronic equipment comprises a memory and a processor, the memory and the processor are in communication connection with each other, computer instructions are stored in the memory, and the processor executes the image denoising method by executing the computer instructions. Compared with the prior art, the de-noising effect of the wide dynamic range image is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image denoising method, a device, an electronic device and a storage medium. Background Art

[0002] In the related art, a WDR (Wide Dynamic Range) image can generally be generated by frame fusion, for example, by fusing a long frame image and a short frame image into a WDR image. The denoising process of the WDR image is often performed after image fusion. In the image fusion process, the pixels of the image need to be processed, including the reduction or enlargement of the pixel values, and in the process of processing the pixels of the image, it is easy to amplify or reduce the noise of the long frame image or the short frame image. Taking the case where the noise on the long frame image is reduced compared to before the image fusion and the noise on the short frame image is unchanged compared to before the image fusion as an example, due to the limitations of the related image denoising technology, the denoising effect of the WDR image is very different at different pixel points, and the denoising effect of the WDR image is not good. Summary of the invention

[0003] In view of this, the present invention provides an image denoising method, an apparatus, an electronic device and a storage medium to solve the problem of poor WDR image denoising effect existing in the related art.

[0004] In a first aspect, the present invention provides an image denoising method, the image denoising method comprising:

[0005] Acquire a first image and a second image, wherein the first image and the second image are used together to form a wide dynamic range image, and a ratio of an exposure time of the first image to an exposure time of the second image is a set value;

[0006] Performing a first linear transformation process on the first image to obtain a third image, and performing a second linear transformation process on the second image to obtain a fourth image; pixel values ​​of pixels on the third image correspond to a first actual scene brightness range, pixel values ​​of pixels on the fourth image correspond to a second actual scene brightness range, and a maximum value of the first actual scene brightness range is equal to a minimum value of the second actual scene brightness range;

[0007] Performing image fusion processing on the third image and the fourth image to obtain a fifth image;

[0008] The fifth image is subjected to denoising processing to obtain an image with a wide dynamic range.

[0009] By performing the first linear transformation processing and the second linear transformation processing on the first image and the second image used to form the wide dynamic range image, respectively, the present invention can make the maximum value of the first actual scene brightness range corresponding to the pixel value of the pixel point on the generated third image equal to the minimum value of the second actual scene brightness range corresponding to the pixel value of the pixel point on the generated fourth image, and then perform fusion processing on the third image and the fourth image and perform denoising processing on the fifth image obtained after fusion; it can be seen that the present invention can make the pixel values ​​of the pixel points on the multiple frames of images used to form the wide dynamic range image correspond to the actual scene brightness by linear transformation processing before fusion and denoising, so that the image information of the first actual scene brightness range is derived from the first image, and the image information of the second actual scene brightness range is derived from the second image, avoiding the pixel value scaling processing of the first image or the second image due to distinguishing between bright and dark areas during the frame fusion process, thereby avoiding the problem of different effects on the noise on the first image and the noise on the second image during the image fusion process, reducing the difficulty of denoising the wide dynamic range image, making the denoising effect of the wide dynamic range image consistent at different pixel points, and improving the denoising effect of the wide dynamic range image.

[0010] In a second aspect, the present invention provides an image denoising device, the image denoising device comprising:

[0011] An acquisition module, used for acquiring a first image and a second image, wherein the first image and the second image are used together to form an image with a wide dynamic range, and a ratio of an exposure time of the first image to an exposure time of the second image is a set value;

[0012] a transformation module, configured to perform a first linear transformation on the first image to obtain a third image, and further configured to perform a second linear transformation on the second image to obtain a fourth image; pixel values ​​of pixels on the third image correspond to a first actual scene brightness range, pixel values ​​of pixels on the fourth image correspond to a second actual scene brightness range, and a maximum value of the first actual scene brightness range is equal to a minimum value of the second actual scene brightness range;

[0013] A fusion module, used for performing image fusion processing on the third image and the fourth image to obtain a fifth image;

[0014] The denoising module is used to perform denoising on the fifth image to obtain an image with a wide dynamic range.

[0015] In a third aspect, the present invention provides an electronic device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the image denoising method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0016] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the image denoising method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a flowchart of an image denoising method according to an embodiment of the present invention;

[0019] Figure 2 is a schematic flow chart of another image denoising method according to an embodiment of the present invention;

[0020] Figure 3 is a schematic diagram of the corresponding relationship between actual scene brightness and pixel value according to an embodiment of the present invention;

[0021] Figure 4 is a flowchart of another image denoising method according to an embodiment of the present invention;

[0022] Figure 5 is a flowchart of yet another image denoising method according to an embodiment of the present invention;

[0023] Figure 6 is a schematic diagram of a process for obtaining a clean WDR image based on a single-frame denoising network according to an embodiment of the present invention;

[0024] Figure 7 is a schematic diagram of a process of obtaining a clean WDR image based on a frame fusion neural network according to an embodiment of the present invention;

[0025] Figure 8 is a schematic diagram of a training process of a frame fusion neural network according to an embodiment of the present invention;

[0026] Fig. 9 is a structural block diagram of an image denoising device according to an embodiment of the present invention;

[0027] Fig.10 It is a schematic diagram of the hardware structure of the electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0029] A WDR (Wide Dynamic Range) image is an image that maintains rich details under different lighting conditions. A commonly used WDR image generation method is to fuse multiple frame images. Multiple frame images may include long frame images and short frame images. Among them, the long frame image is responsible for improving the signal-to-noise ratio of the dark area, and the short frame image is responsible for collecting the overexposed bright area image of the long frame. Assume that the exposure time ratio of the long frame image to the short frame image is β, that is, β represents the exposure ratio; the frame fusion method can be to set a threshold for the long frame (for example Figure 3 The threshold value on the horizontal axis in is used to distinguish between bright and dark areas. When the pixel value is below the threshold, the pixel value of the long frame image is divided by the exposure ratio. When the pixel value is above the threshold, it is replaced by the pixel value corresponding to the short frame image. In this way, the high signal-to-noise ratio of the long frame image in the dark area is retained, and the image information lost in the overexposed area of ​​the long frame is compensated by the short frame image. Generally, the fused WDR image needs to be denoised by a trained denoising model. The advantage of denoising the fused WDR image is that the number of fused images is smaller, which can save computing power. However, the pixels on the WDR image include pixels from the long frame image and pixels from the short frame image. The noise of the short frame image remains unchanged compared to before the image fusion, but the noise of the long frame image is reduced by β times compared to before the image fusion. Therefore, the noise model (a mathematical model used to describe the noise type and noise statistical characteristics in the image, such as the Poisson-Gaussian noise model, etc.) used to characterize different pixel points in a WDR image is inconsistent, which makes the trained denoising model unable to effectively remove the noise on the WDR image, resulting in poor denoising effect and other problems.

[0030] According to an embodiment of the present invention, an embodiment of an image denoising method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] In this embodiment, an image denoising method is provided, which can be used in electronic devices. Figure 1 is a flow chart of an image denoising method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0032] Step S101 , acquiring a first image and a second image, wherein the first image and the second image are used together to form an image with a wide dynamic range, and a ratio of an exposure time of the first image to an exposure time of the second image is a set value.

[0033] In this embodiment, the first image may be a long frame image, the second image may be a short frame image, and the set value may be, for example, the aforementioned exposure ratio β, but is certainly not limited thereto.

[0034] In some optional implementations, acquiring the first image and the second image includes: reading the first image and the second image from a plurality of frames of images taken by a camera.

[0035] The camera is equipped with an image sensor, and multiple frames of images captured by the camera contain noise. Specifically, several frames of static images with noise are acquired through the sensor, that is, the first image and the second image contain noise.

[0036] For the first image and the second image captured by the camera and used to form a WDR image, this embodiment can apply the image denoising method to an electronic device equipped with a camera to obtain a WDR image denoised by this embodiment.

[0037] In some optional embodiments, the first image and the second image are at least any two of a long frame image, a medium frame image, and a short frame image, the exposure time of the long frame image is greater than the exposure time of the medium frame image, and the exposure time of the medium frame image is greater than the exposure time of the short frame image.

[0038] The images to be fused and denoised in the embodiment of the present invention may include at least two images, for example, three images: a long frame image, a medium frame image, and a short frame image, or any two of the long frame image, the medium frame image, and the short frame image. If three images are taken as an example, the first image in this embodiment includes two frames of images, or the second image includes two frames of images, for example, the first image is a long frame image and a medium frame image and the second image is a short frame image, or the first image is a long frame image and the second image is a medium frame image and a short frame image; of course, based on the above examples of this embodiment, there may be other situations, which are not listed one by one.

[0039] In the following embodiments of the present invention, the first image is a long frame image and the second image is a short frame image as an example for corresponding description, and the implementation of other situations (such as a long frame image and a medium frame image, a medium frame image and a short frame image) is similar.

[0040] The embodiment of the present invention can better process two or three frames of images with different exposure times for forming a wide dynamic range image, and is suitable for better fusion and denoising of long frame images and short frame images, long frame images and medium frame images, medium frame images and short frame images, and long frame images, medium frame images and short frame images. In this embodiment, the fusion and denoising of two frames of images can be extended to the fusion and denoising of three frames of images, and has a wide range of applications.

[0041] Step S102, performing a first linear transformation on the first image to obtain a third image, and performing a second linear transformation on the second image to obtain a fourth image; the pixel values ​​of the pixels on the third image correspond to the first actual scene brightness range, the pixel values ​​of the pixels on the fourth image correspond to the second actual scene brightness range, and the maximum value of the first actual scene brightness range is equal to the minimum value of the second actual scene brightness range.

[0042] The first linear transformation processing and / or the second linear transformation processing in this embodiment include but are not limited to the process of increasing, decreasing and / or scaling the pixel values ​​of the pixels on the image; the pixel values ​​of the pixels on the image in this embodiment may, for example, include but are not limited to the brightness values ​​of the pixels on the image.

[0043] Combination Figure 3 As shown, the process of the first linear transformation of the first image as a long frame image is specifically a process of converting the solid line on the upper side of the figure into the dotted line on the lower side; and the process of the second linear transformation of the second image as a short frame image is specifically a process of converting the solid line on the lower side of the figure into the dotted line on the upper side.

[0044] Step S103: performing image fusion processing on the third image and the fourth image to obtain a fifth image.

[0045] In the image fusion process of this embodiment, multiple frame images (two frame images or three frame images) are fused into one frame image, for example, a long frame image and a short frame image are fused into a fifth image, or a long frame image, a medium frame image and a short frame image are fused into a fifth image.

[0046] This embodiment can use a trained image fusion model (such as a single-frame denoising network) to fuse multiple frames of images into a fifth image, or use a separately developed image fusion algorithm to achieve the fusion function of multiple frames of images, which is not limited in this embodiment.

[0047] Step S104: performing denoising processing on the fifth image to obtain an image with a wide dynamic range.

[0048] Based on the aforementioned linear transformation process of the first image and the second image in this embodiment, the noise of different pixel points on the fifth image can be removed using the same method, thereby reducing the difficulty of the image denoising process.

[0049] In this embodiment, by performing the first linear transformation processing and the second linear transformation processing on the first image and the second image used to form the wide dynamic range image, respectively, the present invention can make the maximum value of the first actual scene brightness range corresponding to the pixel value of the pixel point on the generated third image equal to the minimum value of the second actual scene brightness range corresponding to the pixel value of the pixel point on the generated fourth image, and then perform fusion processing on the third image and the fourth image, and perform denoising processing on the fifth image obtained after fusion; it can be seen that the present invention can make the pixel values ​​of the pixel points on the multiple frames of images used to form the WDR image correspond to the actual scene brightness by linear transformation processing before fusion and denoising, so that the image information of the first actual scene brightness range is derived from the first image, and the image information of the second actual scene brightness range is derived from the second image, avoiding the pixel value scaling processing of the first image or the second image due to distinguishing between bright and dark areas during the frame fusion process, thereby avoiding the problem of different effects on the noise on the first image and the noise on the second image during the image fusion process, reducing the difficulty of denoising the WDR image, making the denoising effect of the WDR image consistent at different pixels, and improving the denoising effect of the WDR image.

[0050] In this embodiment, an image denoising method is provided, which can be used in electronic devices. Figure 2 is a flow chart of an image denoising method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0051] Step S201, obtaining a first image and a second image, the first image and the second image are used together to form a wide dynamic range image, and the ratio of the exposure time of the first image to the exposure time of the second image is a set value. For details, please refer to Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0052] Step S202, performing a first linear transformation on the first image to obtain a third image, and performing a second linear transformation on the second image to obtain a fourth image; the pixel values ​​of the pixels on the third image correspond to the first actual scene brightness range, the pixel values ​​of the pixels on the fourth image correspond to the second actual scene brightness range, and the maximum value of the first actual scene brightness range is equal to the minimum value of the second actual scene brightness range.

[0053] Specifically, performing a first linear transformation process on the first image to obtain a third image includes:

[0054] Step S2021, determining a first difference between a pixel value of a pixel point on the first image and a first preset pixel value.

[0055] Wherein, when the first image is a long frame image, the first preset pixel value is a black level.

[0056] Step S2022, reducing the first difference by a preset multiple and adding the result to the first preset pixel value, and using the result of the addition as the pixel value of the pixel point on the third image; the first preset pixel value is the minimum value of the pixel values ​​of all the pixel points on the first image and the second image.

[0057] The zoom factor is a zoom factor. The same zoom factor is used for the first image and the second image. The zoom factor in this embodiment is represented by a.

[0058] Combination Figure 3 As shown, in this embodiment, taking the first image as a long frame image as an example, the process of performing the first linear transformation processing on the first image is represented in the following manner:

[0059] y L =f L (x L )=(x L -c L ) / a+c L

[0060] Among them, y L represents the pixel value of the long frame image after linear transformation, f L (x) represents the function of linear transformation, x L represents the pixel value of the long frame image before linear transformation, c L Indicates the black level, and a indicates the preset multiple.

[0061] Specifically, performing a second linear transformation process on the second image to obtain a fourth image includes:

[0062] Step S2023, determining a second difference between a pixel value of a pixel point on the second image and a second preset pixel value.

[0063] Wherein, when the second image is a short frame image, the second preset pixel value is the maximum pixel value.

[0064] Step S2024, reducing the second difference by a preset multiple and adding the result to the second preset pixel value, and using the added result as the pixel value of the pixel point on the fourth image; the second preset pixel value is the maximum value of the pixel values ​​of all the pixel points on the first image and the second image.

[0065] In this embodiment, the scaling factor used in the process of performing linear transformation processing on the first image and the second image is both a, so as to ensure that the noise model of the pixel points on the first image is consistent with the noise model of the pixel points on the second image.

[0066] Combination Figure 3 As shown, in this embodiment, taking the second image as a short frame image as an example, the process of performing the second linear transformation processing on the second image is represented in the following manner:

[0067] y S =f S (x S )=(x S -c S ) / a+c S

[0068] Among them, y S Represents the pixel value of the short frame image after linear transformation, f S (x) represents the function of linear transformation, x S represents the pixel value of the short frame image before linear transformation, c S represents the maximum pixel value (the maximum value of the pixel values ​​of all pixels on the long frame image and the short frame image), and a represents the preset multiple.

[0069] Wherein, the number of bits occupied by each pixel is 12, and for 12-bit data, the second preset pixel value can be 2 12 -1=4095, i.e. c S =4095.

[0070] like Figure 3 As shown, the long frame (x L ) The image is linearly transformed to obtain the transformed long frame (y L ) image, short frame (x S ) The image is linearly transformed to obtain the transformed short frame (y S ) image, the range between the actual scene brightness of the long frame image corresponding to the maximum pixel value and the actual scene brightness of the short frame image is the overexposure area. Figure 3 The linear transformation process is shown in FIG. 1 . The preset multiple a in this embodiment is 2-1 / β, where β represents the exposure ratio. Specifically, since the image brightness corresponding to the long frame image is approximately β times that of the short frame, that is, (x L -c L )=β(x S -c L ), so the pixel value of the pixel point on the short frame image corresponding to the threshold used to distinguish between bright and dark areas is x S0 =(x L0 -c L ) / β+c L ; For example, the maximum pixel value of the fused image from the long frame image is f L (x L0 -1), which comes from the fact that the minimum pixel value of the short frame image is f S (x S0), in order to ensure that the order of brightness and darkness before and after fusion remains unchanged, the latter should be greater than the former, so the threshold should satisfy f L (x L0 ) = f S (x S0 ), that is, f L (x L0 ) = f S (x L0 / β-c L / β+c L ). Combining the aforementioned first linear transformation process and the second linear transformation process, we can get the equation (x L0 -c L ) / a+c L =[(x L0 -c L ) / β+c L -c S ] / a+c S , solving this equation yields According to the above results, this embodiment can determine that a is a constant between (1, 2). Generally, x L0 Choose a value slightly smaller than c S If we choose x L0 =c S , then the formula can be simplified to a=2-1 / β.

[0071] The first image of the present embodiment is a long-frame image, and the distance between the pixel value of the pixel point on the first image after the first linear transformation and the first preset pixel value is reduced by a preset multiple, thereby fully utilizing the image information in the lower brightness range on the first image; the second image of the present embodiment is a short-frame image, and the distance between the pixel value of the pixel point on the second image after the second linear transformation and the second preset pixel value is reduced by a preset multiple, thereby fully utilizing the image information in the higher brightness range on the second image, thereby helping to further enhance the image denoising effect of the present invention.

[0072] Step S203: perform image fusion processing on the third image and the fourth image to obtain a fifth image. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0073] Step S204: De-noise the fifth image to obtain a wide dynamic range image. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0074] In this embodiment, an image denoising method is provided, which can be used in electronic devices. Figure 4 is a flow chart of an image denoising method according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0075] Step S401, obtaining a first image and a second image, the first image and the second image are used together to form a wide dynamic range image, and the ratio of the exposure time of the first image to the exposure time of the second image is a set value. For details, please refer to Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0076] Step S402: Perform a first linear transformation on the first image to obtain a third image, and perform a second linear transformation on the second image to obtain a fourth image; the pixel values ​​of the pixels on the third image correspond to the first actual scene brightness range, the pixel values ​​of the pixels on the fourth image correspond to the second actual scene brightness range, and the maximum value of the first actual scene brightness range is equal to the minimum value of the second actual scene brightness range. For details, please refer to Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0077] Step S403: perform image fusion processing on the third image and the fourth image to obtain a fifth image.

[0078] Specifically, the above step S403 includes:

[0079] Step S4031: for each pixel point on the third image, compare the first target pixel value of each pixel point with a preset value.

[0080] Specifically, the pixel points on the third image can be traversed to determine the first target pixel value of each pixel point; the comparison result between the first target pixel value and the preset value may include: the first target pixel value is less than the preset value and the first target pixel value is greater than or equal to the preset value.

[0081] In this embodiment, the preset value is x L0 (For example, you can select the aforementioned c S as the default value).

[0082] Step S4032: If the first target pixel value is less than the preset value, the first target pixel value is used as the fused pixel value.

[0083] In combination with the above-mentioned embodiment, when the pixel value of a certain pixel point on the third image is less than the preset value, that is, x L <x L0 , then the fused pixel value is y L .

[0084] Step S4033, if the first target pixel value is greater than or equal to the preset value, the second target pixel value is used as the fused pixel value; wherein the second target pixel value is the pixel value of the target pixel point on the fourth image, and the relative position of the target pixel point on the fourth image is the same as the relative position of the pixel point corresponding to the first target pixel value on the third image.

[0085] In combination with the above-mentioned embodiment, when the pixel value of a certain pixel point on the third image is greater than or equal to the preset value, that is, x L ≥x L0 , then the fused pixel value is y S .

[0086] Step S4034: Use each fused pixel value as the pixel value of each pixel point on the fifth image.

[0087] Combined with the above-mentioned embodiment, the pixel value of each pixel point on the fifth image is

[0088] Therefore, the embodiment of the present invention can effectively utilize the features of the pixel points on the third image and the features of the pixel points on the fourth image, and select the pixel value of the pixel point on the third image or the pixel value of the pixel point on the fourth image as the pixel value of the pixel point on the fused fifth image through the comparison result of the pixel value of the pixel point on the third image and the preset value. This method can fully utilize the image features of the pixel points on the first image corresponding to the third image and fully utilize the image features of the pixel points on the second image corresponding to the fourth image according to actual conditions; for example, when the first image is a long-frame image and the second image is a short-frame image, the fusion scheme of this embodiment can make the image information of the low-brightness area come from the long-frame image, and the image information of the high-brightness area come from the short-frame image, that is, fully utilize the high signal-to-noise ratio characteristics of the long-frame image in the low-brightness area, and fully utilize the image information of the short-frame image in the high-brightness area, so as to improve the quality of image fusion.

[0089] Step S404: De-noise the fifth image to obtain a wide dynamic range image. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0090] In this embodiment, an image denoising method is provided, which can be used in electronic devices. Figure 5 is a flow chart of an image denoising method according to an embodiment of the present invention. Figure 5 As shown, and can be combined Figure 6 , the process includes the following steps:

[0091] Step S501, obtaining a first image and a second image, the first image and the second image are used together to form a wide dynamic range image, and the ratio of the exposure time of the first image to the exposure time of the second image is a set value. For details, please refer to Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0092] Step S502: Perform a first linear transformation on the first image to obtain a third image, and perform a second linear transformation on the second image to obtain a fourth image; the pixel values ​​of the pixels on the third image correspond to the first actual scene brightness range, the pixel values ​​of the pixels on the fourth image correspond to the second actual scene brightness range, and the maximum value of the first actual scene brightness range is equal to the minimum value of the second actual scene brightness range. For details, please refer to Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0093] like Figure 6 As shown, in some optional implementations, the third image and the fourth image are fused to obtain a fifth image, which may be a noisy WDR image. In this embodiment, denoising the fifth image includes: denoising the fifth image using a trained single-frame denoising network.

[0094] This embodiment can use a clean raw image as a label and a noisy raw image in a training set as an input to train a single-frame denoising network, thereby obtaining a trained single-frame denoising network.

[0095] The input of the single-frame denoising network is the wide dynamic range image to be denoised, and the output of the single-frame denoising network is the denoised wide dynamic range image. Next, the reason why the present invention can use the single-frame denoising network is explained. The following mathematical model is the Poisson-Gaussian noise model used to train the denoising network:

[0096]

[0097] Among them, X represents the random variable of pixel value, g represents the gain, and α represents a coefficient related to the photoelectric conversion efficiency. Indicates the mean Poisson distribution random variable, EX represents the mathematical expectation of X, δ represents the black level, and δ in this embodiment is c L ,N(0,σ1 2 ) represents a normally distributed random variable with mean 0 and standard deviation σ.

[0098] Typically, noise becomes noticeable only at higher gain g, when the gain-independent term The influence of is small and can be ignored. In this embodiment, X' is defined to represent the pixel value of X minus the black level, X'=Xc L , we can get the deformation of the above Poisson-Gaussian noise model:

[0099] Combined with the first linear transformation process of the long frame image, y L '=y L -c L =(x L -c L ) / a=x' / a, a deformation of the joint Poisson-Gaussian noise model, specifically using Y L 'Replace y L ', X' replaces x', the following results can be obtained:

[0100]

[0101] Assuming an ideal single-frame denoising network f g , input X' and get output EX', that is, f g (X') = EX'; Combined with the above-mentioned deformation of the Poisson-Gaussian noise model, set the parameter g to g / a, then f g / a (Y L ')=EY L '.

[0102] Combined with the second linear transformation process of the short frame image, the following can be obtained:

[0103] y S '=y S -c L

[0104] =(x S -c S ) / a+c S -c L

[0105] =(x S -c L ) / a+(c S -c L )(1-1 / a)

[0106] =x' / a+b

[0107] Where b = (c S -c L )(1-1 / a).

[0108] Due to the existence of constant b, the denoising of short frame images is different from the direct denoising of long frame images in preprocessing. In this embodiment, the constant b is replaced by a Poisson distribution random number with a mean of b, and then the following is obtained:

[0109]

[0110] Combined with the deformation of the aforementioned Poisson-Gaussian noise model, specifically using Y S 'Replace y S ', X' replaces x', then the following results can be obtained:

[0111]

[0112] The comparison of the result in this embodiment with the above-mentioned deformation of the Poisson-Gaussian noise model shows that after setting the parameter g to g / a, the short frame image also has f g / a (Y S ')=EY S '. Therefore, it is only necessary to replace the single-frame denoising network f g If the parameter g is reduced by a times, then y L '=y L -c L =(x L -c L ) / a=x' / a preprocessed long frame image and through The preprocessed short frame image is denoised; because the noise models of the long frame image and the short frame image used in the present invention are similar, the single frame denoising network trained on the same noise model can denoise the long frame image and the short frame image at the same time, and thus the single frame denoising network f of this embodiment g It is possible to denoise a WDR image obtained by fusing a long frame image and a short frame image in a manner determined by a preset value.

[0113] Based on the above process, this embodiment denoises the fifth image obtained through the above processing through a single-frame denoising network, which can take into account the denoising efficiency and denoising effect of the image, and can achieve self-learning during continuous use to adapt to more denoising scenarios and application requirements, and further improve the denoising performance.

[0114] like Figure 7 As shown, some embodiments of the present invention further provide another implementation of the image denoising method, which may specifically include but is not limited to the following step S503.

[0115] Step S503, performing image fusion processing on the third image and the fourth image to obtain a fifth image; performing denoising processing on the fifth image to obtain a wide dynamic range image, including: using a trained frame fusion neural network to perform an image fusion processing process and a denoising processing process; wherein the input of the frame fusion neural network is an image processed by a first linear transformation and an image processed by a second linear transformation, and the output of the frame fusion neural network is a denoised wide dynamic range image.

[0116] This embodiment uses the third image and the fourth image as inputs of the frame fusion neural network, and can quickly and accurately infer a wide dynamic range image. It can be seen that the present invention can also simultaneously realize image fusion and image denoising functions.

[0117] like Figure 8 As shown, the present invention also provides a training process of a frame fusion neural network, and the image denoising method of this embodiment also includes:

[0118] Step a1, splitting the clean original image into a first target image and a second target image, wherein the ratio of the exposure time of the first target image to the exposure time of the second target image is a set value.

[0119] The clean original image is a clean raw image. In this embodiment, the clean image can be downloaded through the network, and then the clean image can be subjected to inverse ISP (Image Signal Processor) processing to obtain a clean raw image. The number of clean original images can be several thousand, which are used to form a training set.

[0120] In this embodiment, the first target image is a long frame image, and the second target image is a short frame image, that is, the long frame image and the short frame image are constructed from the clean original image. Specifically, the pixel value of the pixel point on the short frame image is the same as the pixel value of the pixel point on the raw image; the pixel value of the pixel point on the long frame image is the pixel value of the pixel point on the raw image multiplied by the exposure ratio β, and for pixels exceeding the maximum pixel value c S The pixel point whose pixel value is equal to the maximum pixel value c S .

[0121] Step a2: adding first noise to the first target image to obtain a third target image, and adding second noise to the second target image to obtain a fourth target image.

[0122] In the embodiment of the present invention, the manner of adding the first noise may be the same as the manner of adding the second noise. The process of adding noise is described in detail below.

[0123] Combined with the Poisson-Gaussian noise model used to train the denoising network, the variance of the Poisson-Gaussian noise model is obtained: It can be seen that there is a linear relationship between the mean EX and the variance DX, αg is the slope, and g is a set known parameter. In this embodiment, the parameter α (scaling factor) is calibrated in the following way.

[0124] Collect a set of static photos, such as several frames of images, and record (simulate) gain g. In this embodiment, a set of static photos can include 100 frames of images, so as to more accurately calculate the pixel mean and variance. For the scene of photo collection, try to ensure the diversity of brightness. You can shoot a standard color card and ensure the stability of the camera and the brightness of the light source during shooting, that is, ensure that the scene captured is static; the mean and variance of the pixel values ​​of the pixel points on the above 100 frames of images can be calculated respectively, through The slope can be obtained by fitting, and the slope is divided by g to obtain the parameter α (scaling factor).

[0125] The noise added in this embodiment can be, for example, Poisson noise. Specifically, the following operation is performed on each pixel point on a clean raw image: the pixel value of each pixel point is subtracted from the black level, and then divided by αg, and the obtained value is used as the mean of the Poisson distribution. Then, a Poisson distribution random number is generated based on the mean, and the generated Poisson distribution random number is multiplied by αg, and then the black level is added to obtain the pixel value after adding Poisson noise. Among them, the method of generating a Poisson distribution random number can be generated, for example, by the library function random.poisson(lambda) in numpy (an open source scientific computing library). For the above For black frame images, only the last two terms are available. The relationship between variance DX and g is a quadratic function. Under multiple g conditions, several black frame images are collected. The variance DX is calculated for each black frame image corresponding to each g, that is, the variance DX is obtained. Then, the relationship between variance DX and g is fitted, and the two parameters σ1 and σ2 can be obtained. To read out the noise, each pixel of the image with Poisson noise is processed as follows: a normally distributed random number with a standard deviation of σ1 and a mean of 0 is generated and multiplied by the specified gain g. A , and then generate a normally distributed random number with a standard deviation of σ2 and a mean of 0, and add these two data to the original pixel value to become the new pixel value.

[0126] Step a3, performing a first linear transformation on the third target image to obtain a fifth target image, and performing a second linear transformation on the fourth target image to obtain a sixth target image; the pixel values ​​of the pixels on the fifth target image correspond to the third actual scene brightness range, the pixel values ​​of the pixels on the sixth target image correspond to the fourth actual scene brightness range, and the maximum value of the third actual scene brightness range is equal to the minimum value of the fourth actual scene brightness range.

[0127] The process of the first linear transformation and the process of the second linear transformation have been described in detail in the aforementioned embodiments and will not be repeated here.

[0128] Step a4, constructing a training set based on the fifth target image and the sixth target image, using the clean original image as a label, and training the frame fusion neural network using the training set and the label to obtain a trained frame fusion neural network.

[0129] Specifically, the embodiment of the present invention uses clean raw images as labels and noisy raw images in a training set as inputs to train a frame fusion neural network, thereby obtaining a trained frame fusion neural network.

[0130] Based on the above-mentioned processing method of noise addition and linear transformation, the present invention can also construct a training set based on the first target image and the second target image split from the clean original image, so as to perform high-quality training on the frame fusion neural network according to the training set and labels.

[0131] In this embodiment, an image denoising device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0132] This embodiment provides an image denoising device, such as Fig. 9 As shown, including:

[0133] The acquisition module 901 is used to acquire a first image and a second image, where the first image and the second image are used together to form a wide dynamic range image, and the ratio of the exposure time of the first image to the exposure time of the second image is a set value.

[0134] The transformation module 902 is used to perform a first linear transformation on the first image to obtain a third image. The transformation module 902 is also used to perform a second linear transformation on the second image to obtain a fourth image. The pixel values ​​of the pixels on the third image correspond to the first actual scene brightness range, the pixel values ​​of the pixels on the fourth image correspond to the second actual scene brightness range, and the maximum value of the first actual scene brightness range is equal to the minimum value of the second actual scene brightness range.

[0135] The fusion module 903 is used to perform image fusion processing on the third image and the fourth image to obtain a fifth image.

[0136] The denoising module 904 is used to perform denoising on the fifth image to obtain an image with a wide dynamic range.

[0137] In some optional implementations, the transformation module 902 includes:

[0138] The first determining unit is used to determine a first difference between a pixel value of a pixel point on the first image and a first preset pixel value.

[0139] The first calculation unit is used to reduce the first difference by a preset multiple and then add it to the first preset pixel value, and to use the result of the addition as the pixel value of the pixel point on the third image; the first preset pixel value is the minimum value of the pixel values ​​of all the pixel points on the first image and the second image.

[0140] In some optional implementations, the transformation module 902 includes:

[0141] The second determining unit is used to determine a second difference between a pixel value of a pixel point on the second image and a second preset pixel value.

[0142] The second calculation unit is used to reduce the second difference by a preset multiple and add it to the second preset pixel value, and to use the result of the addition as the pixel value of the pixel point on the fourth image; the second preset pixel value is the maximum value of the pixel values ​​of all the pixel points on the first image and the second image.

[0143] In some optional implementations, the fusion module 903 includes:

[0144] The pixel value comparison unit is used to compare the first target pixel value of each pixel point with a preset value for each pixel point on the third image.

[0145] The first judgment unit is configured to use the first target pixel value as a fused pixel value according to the first target pixel value being smaller than a preset value.

[0146] The second judgment unit is used to use the second target pixel value as the fused pixel value according to the first target pixel value being greater than or equal to a preset value; wherein the second target pixel value is the pixel value of the target pixel point on the fourth image, and the relative position of the target pixel point on the fourth image is the same as the relative position of the pixel point corresponding to the first target pixel value on the third image.

[0147] The pixel value determining unit is used to use each fused pixel value as the pixel value of each pixel point on the fifth image.

[0148] In some optional embodiments, the denoising module 904 is a trained single-frame denoising network, and the image denoising device is used to perform denoising on the fifth image using the trained single-frame denoising network; the input of the single-frame denoising network is the wide dynamic range image to be denoised, and the output of the single-frame denoising network is the denoised wide dynamic range image.

[0149] In some optional implementations, the fusion module 903 and the denoising module 904 are integrated into a trained frame fusion neural network. The image denoising device is used to perform an image fusion processing process and a denoising processing process using the trained frame fusion neural network; wherein the input of the frame fusion neural network is an image processed by the first linear transformation and an image processed by the second linear transformation, and the output of the frame fusion neural network is a denoised wide dynamic range image.

[0150] In some optional implementations, the image denoising apparatus further includes:

[0151] The image splitting module is used to split the clean original image into a first target image and a second target image, and the ratio of the exposure time of the first target image to the exposure time of the second target image is a set value.

[0152] The noise adding module is used to add a first noise to the first target image to obtain a third target image. The noise adding module is also used to add a second noise to the second target image to obtain a fourth target image.

[0153] The linear transformation module is used to perform a first linear transformation on the third target image to obtain a fifth target image. The linear transformation module is also used to perform a second linear transformation on the fourth target image to obtain a sixth target image. The pixel values ​​of the pixels on the fifth target image correspond to the third actual scene brightness range, the pixel values ​​of the pixels on the sixth target image correspond to the fourth actual scene brightness range, and the maximum value of the third actual scene brightness range is equal to the minimum value of the fourth actual scene brightness range.

[0154] The network training module is used to construct a training set based on the fifth target image and the sixth target image, and to use the clean original image as a label. The network training module is used to train the frame fusion neural network using the training set and the label to obtain a trained frame fusion neural network.

[0155] In some optional implementations, the acquisition module 901 is specifically configured to read the first image and the second image from multiple frames of images taken by a camera.

[0156] In some optional embodiments, the first image and the second image are at least any two of a long frame image, a medium frame image, and a short frame image, the exposure time of the long frame image is greater than the exposure time of the medium frame image, and the exposure time of the medium frame image is greater than the exposure time of the short frame image.

[0157] The further functional description of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0158] The image denoising device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0159] The embodiment of the present invention also provides an electronic device having the above Fig. 9 The image denoising device shown.

[0160] See also Fig.10 , Fig.10 is a schematic diagram of the structure of an electronic device provided by an optional embodiment of the present invention, such as Fig.10 As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.10 A processor 10 is taken as an example.

[0161] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0162] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0163] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0164] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0165] The electronic device further comprises a communication interface 30 for the electronic device to communicate with other devices or a communication network.

[0166] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0167] The foregoing content can be better understood in accordance with the following terms:

[0168] Clause A1. A method for image denoising, the method comprising:

[0169] Acquire a first image and a second image, wherein the first image and the second image are used together to form a wide dynamic range image, and a ratio of an exposure time of the first image to an exposure time of the second image is a set value;

[0170] Performing a first linear transformation process on the first image to obtain a third image, and performing a second linear transformation process on the second image to obtain a fourth image; the pixel values ​​of the pixels on the third image correspond to a first actual scene brightness range, the pixel values ​​of the pixels on the fourth image correspond to a second actual scene brightness range, and the maximum value of the first actual scene brightness range is equal to the minimum value of the second actual scene brightness range;

[0171] Performing image fusion processing on the third image and the fourth image to obtain a fifth image;

[0172] The fifth image is subjected to denoising processing to obtain the wide dynamic range image.

[0173] Clause A2. The method according to clause A1, wherein the first image is subjected to a first linear transformation to obtain a third image, comprising:

[0174] Determine a first difference between a pixel value of a pixel point on the first image and a first preset pixel value;

[0175] The first difference is reduced by a preset multiple and then added to the first preset pixel value, and the result of the addition is used as the pixel value of the pixel point on the third image; the first preset pixel value is the minimum value of the pixel values ​​of all pixel points on the first image and the second image.

[0176] Clause A3. The method according to clause A2, wherein the second image is subjected to a second linear transformation to obtain a fourth image, comprising:

[0177] Determine a second difference between a pixel value of a pixel point on the second image and a second preset pixel value;

[0178] The second difference is reduced by the preset multiple and then added to the second preset pixel value, and the result of the addition is used as the pixel value of the pixel point on the fourth image; the second preset pixel value is the maximum value of the pixel values ​​of all the pixel points on the first image and the second image.

[0179] Clause A4. According to any one of clauses A1 to A3, the step of fusing the third image with the fourth image to obtain a fifth image comprises:

[0180] For each pixel point on the third image, comparing the first target pixel value of each pixel point with a preset value respectively;

[0181] If the first target pixel value is less than the preset value, taking the first target pixel value as the fused pixel value;

[0182] If the first target pixel value is greater than or equal to the preset value, the second target pixel value is used as the fused pixel value; wherein the second target pixel value is the pixel value of the target pixel point on the fourth image, and the relative position of the target pixel point on the fourth image is the same as the relative position of the pixel point corresponding to the first target pixel value on the third image;

[0183] Each of the fused pixel values ​​is used as the pixel value of each pixel point on the fifth image.

[0184] Clause A5. The method according to any one of clauses A1 to A3, wherein denoising the fifth image comprises:

[0185] The fifth image is denoised using a trained single-frame denoising network; the input of the single-frame denoising network is a wide dynamic range image to be denoised, and the output of the single-frame denoising network is a denoised wide dynamic range image.

[0186] Clause A6. The method according to any one of clauses A1 to A3, wherein the third image and the fourth image are subjected to image fusion processing to obtain a fifth image; and the fifth image is subjected to denoising processing to obtain the wide dynamic range image, comprising:

[0187] Using the trained frame fusion neural network to perform the image fusion processing and the denoising processing;

[0188] Among them, the input of the frame fusion neural network is the image processed by the first linear transformation and the image processed by the second linear transformation, and the output of the frame fusion neural network is a wide dynamic range image after denoising.

[0189] Clause A7. The method according to clause A6, further comprising:

[0190] Splitting the clean original image into a first target image and a second target image, wherein the ratio of the exposure time of the first target image to the exposure time of the second target image is the set value;

[0191] adding a first noise to the first target image to obtain a third target image, and adding a second noise to the second target image to obtain a fourth target image;

[0192] The first linear transformation process is performed on the third target image to obtain a fifth target image, and the second linear transformation process is performed on the fourth target image to obtain a sixth target image; the pixel values ​​of the pixels on the fifth target image correspond to the third actual scene brightness range, the pixel values ​​of the pixels on the sixth target image correspond to the fourth actual scene brightness range, and the maximum value of the third actual scene brightness range is equal to the minimum value of the fourth actual scene brightness range;

[0193] A training set is constructed based on the fifth target image and the sixth target image, the clean original image is used as a label, and the frame fusion neural network is trained using the training set and the label to obtain the trained frame fusion neural network.

[0194] Clause A8. The method according to any one of clauses A1 to A3, wherein acquiring the first image and the second image comprises:

[0195] The first image and the second image are read from a plurality of frames of images taken by a camera.

[0196] Clause A9. A method according to any one of clauses A1 to A3,

[0197] The first image and the second image are at least any two of a long frame image, a medium frame image, and a short frame image. The exposure time of the long frame image is greater than the exposure time of the medium frame image, and the exposure time of the medium frame image is greater than the exposure time of the short frame image.

[0198] Clause A10. An image denoising apparatus, the apparatus comprising:

[0199] An acquisition module, used to acquire a first image and a second image, wherein the first image and the second image are used together to form an image with a wide dynamic range, and a ratio of an exposure time of the first image to an exposure time of the second image is a set value;

[0200] a transformation module, configured to perform a first linear transformation on the first image to obtain a third image, and further configured to perform a second linear transformation on the second image to obtain a fourth image; the pixel values ​​of the pixels on the third image correspond to a first actual scene brightness range, the pixel values ​​of the pixels on the fourth image correspond to a second actual scene brightness range, and the maximum value of the first actual scene brightness range is equal to the minimum value of the second actual scene brightness range;

[0201] a fusion module, used for performing image fusion processing on the third image and the fourth image to obtain a fifth image;

[0202] The denoising module is used to perform denoising on the fifth image to obtain the wide dynamic range image.

[0203] Clause A11. An electronic device comprising:

[0204] A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the image denoising method described in any one of clauses A1 to A9 by executing the computer instructions.

[0205] Item A12. A computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions are used to cause a computer to execute the image denoising method described in any one of Items A1 to A9.

[0206] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An image denoising method, characterized in that: The method comprises: Acquire a first image and a second image, wherein the first image and the second image are used together to form a wide dynamic range image, and a ratio of an exposure time of the first image to an exposure time of the second image is a set value; Performing a first linear transformation process on the first image to obtain a third image, and performing a second linear transformation process on the second image to obtain a fourth image; the pixel values ​​of the pixels on the third image correspond to a first actual scene brightness range, the pixel values ​​of the pixels on the fourth image correspond to a second actual scene brightness range, and the maximum value of the first actual scene brightness range is equal to the minimum value of the second actual scene brightness range; Performing image fusion processing on the third image and the fourth image to obtain a fifth image; The fifth image is subjected to denoising processing to obtain the wide dynamic range image.

2. The method according to claim 1, characterized in that The performing a first linear transformation on the first image to obtain a third image includes: Determine a first difference between a pixel value of a pixel point on the first image and a first preset pixel value; The first difference is reduced by a preset multiple and then added to the first preset pixel value, and the result of the addition is used as the pixel value of the pixel point on the third image; the first preset pixel value is the minimum value of the pixel values ​​of all pixel points on the first image and the second image.

3. The method according to claim 2, characterized in that The performing a second linear transformation on the second image to obtain a fourth image includes: Determine a second difference between a pixel value of a pixel point on the second image and a second preset pixel value; The second difference is reduced by the preset multiple and then added to the second preset pixel value, and the result of the addition is used as the pixel value of the pixel point on the fourth image; the second preset pixel value is the maximum value of the pixel values ​​of all the pixel points on the first image and the second image.

4. The method according to any one of claims 1 to 3, characterized in that The step of performing image fusion processing on the third image and the fourth image to obtain a fifth image includes: For each pixel point on the third image, comparing the first target pixel value of each pixel point with a preset value respectively; If the first target pixel value is less than the preset value, taking the first target pixel value as the fused pixel value; If the first target pixel value is greater than or equal to the preset value, the second target pixel value is used as the fused pixel value; wherein the second target pixel value is the pixel value of the target pixel point on the fourth image, and the relative position of the target pixel point on the fourth image is the same as the relative position of the pixel point corresponding to the first target pixel value on the third image; Each of the fused pixel values ​​is used as the pixel value of each pixel point on the fifth image.

5. The method according to any one of claims 1 to 3, characterized in that The performing denoising processing on the fifth image includes: The fifth image is denoised using a trained single-frame denoising network; the input of the single-frame denoising network is a wide dynamic range image to be denoised, and the output of the single-frame denoising network is a denoised wide dynamic range image.

6. The method according to any one of claims 1 to 3, characterized in that performing image fusion processing on the third image and the fourth image to obtain a fifth image; Performing denoising on the fifth image to obtain the wide dynamic range image includes: Using the trained frame fusion neural network to perform the image fusion processing and the denoising processing; Among them, the input of the frame fusion neural network is the image processed by the first linear transformation and the image processed by the second linear transformation, and the output of the frame fusion neural network is a wide dynamic range image after denoising.

7. The method according to claim 6, characterized in that The method further comprises: Splitting the clean original image into a first target image and a second target image, wherein the ratio of the exposure time of the first target image to the exposure time of the second target image is the set value; adding a first noise to the first target image to obtain a third target image, and adding a second noise to the second target image to obtain a fourth target image; The first linear transformation process is performed on the third target image to obtain a fifth target image, and the second linear transformation process is performed on the fourth target image to obtain a sixth target image; the pixel values ​​of the pixels on the fifth target image correspond to the third actual scene brightness range, the pixel values ​​of the pixels on the sixth target image correspond to the fourth actual scene brightness range, and the maximum value of the third actual scene brightness range is equal to the minimum value of the fourth actual scene brightness range; A training set is constructed based on the fifth target image and the sixth target image, the clean original image is used as a label, and the frame fusion neural network is trained using the training set and the label to obtain the trained frame fusion neural network.

8. The method according to any one of claims 1 to 3, characterized in that The acquiring of the first image and the second image comprises: The first image and the second image are read from a plurality of frames of images taken by a camera.

9. The method according to any one of claims 1 to 3, characterized in that The first image and the second image are at least any two of a long frame image, a medium frame image, and a short frame image. The exposure time of the long frame image is greater than the exposure time of the medium frame image, and the exposure time of the medium frame image is greater than the exposure time of the short frame image.

10. An image denoising device, characterized in that: The device comprises: an acquisition module, used to acquire a first image and a second image, wherein the first image and the second image are used together to form an image with a wide dynamic range, and a ratio of an exposure time of the first image to an exposure time of the second image is a set value; a transformation module, configured to perform a first linear transformation on the first image to obtain a third image, and further configured to perform a second linear transformation on the second image to obtain a fourth image; the pixel values ​​of the pixels on the third image correspond to a first actual scene brightness range, the pixel values ​​of the pixels on the fourth image correspond to a second actual scene brightness range, and the maximum value of the first actual scene brightness range is equal to the minimum value of the second actual scene brightness range; a fusion module, used for performing image fusion processing on the third image and the fourth image to obtain a fifth image; The denoising module is used to perform denoising on the fifth image to obtain the wide dynamic range image.

11. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the image denoising method according to any one of claims 1 to 9 by executing the computer instructions.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the image denoising method according to any one of claims 1 to 9.

Images