Image processing method and device, computer equipment and storage medium

Abstract

The invention relates to an image processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring an original exposure image, wherein the original exposure image comprises an image with at least one exposure time; adjusting the brightness value of the original exposure image based on the brightness alignment parameter to obtain a brightness-aligned first exposure image; adjusting the brightness value of the first exposure image based on a preset noise gain parameter and the noise variance to obtain a target exposure image conforming to preset distribution; and inputting the target exposure image into a pre-trained noise reduction model to obtain a noise reduction image. By adopting the method, the noise image related to the signal can be converted into the image data with independent signals and near Gaussian distribution, the influence of noise difference caused by exposure time and parameters is weakened, noise reduction processing is carried out on the image, noise difference is avoided, the imaging effect is improved, and the quality of the output image is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the continuous development of terminal devices, CMOS Sensors (Complementary Metal Oxide Semiconductor, image sensors) and other image acquisition devices, people are pursuing higher and higher quality visual experience, and need to acquire high-quality images through terminal devices, such as static images and HDR (High Dynamic Range) images. HDR images are obtained by fusing multiple images that are continuously output in the time domain from underexposure to overexposure, and can present both bright and dark details at the same time.

[0003] In the related art, due to the noise and brightness difference in the image collected by the terminal acquisition device, the image quality output by the terminal acquisition device will be poor. Summary of the invention

[0004] Based on this, it is necessary to provide an image processing method, apparatus, computer device, computer-readable storage medium and computer program product that can improve image quality in response to the above technical problems.

[0005] In a first aspect, the present application provides an image processing method. The method comprises:

[0006] Acquire an original exposure image, wherein the original exposure image includes an image of at least one exposure time;

[0007] Adjusting the brightness value of the original exposure image based on the brightness alignment parameter to obtain a brightness aligned first exposure image;

[0008] Adjusting the brightness value of the first exposure image based on a preset noise gain parameter and a noise variance to obtain a target exposure image that conforms to a preset distribution;

[0009] The target exposure image is input into a pre-trained denoising model to obtain a denoised image.

[0010] In one of the embodiments, adjusting the brightness value of the original exposure image based on the brightness alignment parameter to obtain the brightness aligned first exposure image includes:

[0011] For a pixel point included in the original exposure image, a brightness difference between a brightness value of the pixel point and a preset brightness is calculated, and brightness correction is performed on the brightness difference of the pixel point using a brightness alignment parameter to obtain a corrected brightness value corresponding to the pixel point, wherein the preset brightness value is determined according to a brightness level offset parameter of a digital-to-analog converter;

[0012] Based on the corrected brightness values ​​corresponding to the pixel points, a first exposure image with brightness alignment is obtained.

[0013] In one embodiment, the preset noise gain parameter is a Poisson noise gain parameter, and the preset distribution is a Poisson distribution; the brightness value of the first exposure image is adjusted based on the preset noise gain parameter and the noise variance to obtain a target exposure image that conforms to the preset distribution, including:

[0014] Based on the Poisson distribution conversion algorithm, Poisson noise gain parameter and noise variance, a Poisson distribution conversion factor is calculated;

[0015] For a pixel point included in the first exposure image, a product value of a brightness value corresponding to the pixel point and the Poisson noise gain parameter is calculated, and the product value is converted and calculated using the Poisson distribution conversion factor to obtain a converted brightness value corresponding to the pixel point;

[0016] Based on the converted brightness values ​​corresponding to the pixel points, a target exposure image whose brightness values ​​conform to the Poisson distribution is obtained.

[0017] In one embodiment, the original exposure image includes images with multiple exposure times; and the method further includes:

[0018] Acquire each exposure time included in the original exposure image, and a digital-to-analog converter parameter corresponding to each exposure time;

[0019] The ratio of each exposure time included in the original exposure image and the ratio of the digital-to-analog converter parameters corresponding to each exposure time are calculated respectively, and the brightness alignment parameter is determined based on each ratio.

[0020] In one embodiment, the target exposure image includes target images of at least two exposure times, including a target image of a first exposure time and a target image of a second exposure time; and inputting the target exposure image into a pre-trained denoising model to obtain a denoised image includes:

[0021] Inputting the target image at the first exposure time and the target image at the second exposure time into a pre-trained denoising model to obtain an initial denoised image corresponding to the first exposure time and an initial denoised image corresponding to the second exposure time;

[0022] Pixel data of the initial denoised image corresponding to the first exposure time and pixel data of the initial denoised image corresponding to the second exposure time are fused to obtain a denoised image.

[0023] In one embodiment, the method further comprises:

[0024] Acquire initial training data, wherein the initial training data includes a plurality of noise-free images;

[0025] Adjusting the brightness value of each of the noise-free images based on the brightness alignment parameter, the preset noise gain parameter, and the noise variance to obtain target training data;

[0026] The denoising model to be trained is trained unsupervisedly using the target training data to obtain a trained denoising model.

[0027] In one embodiment, the denoising model to be trained includes an image generator and an image discriminator, and the denoising model to be trained is subjected to unsupervised training using the target training data to obtain a trained denoising model, including:

[0028] Inputting the target training data into a generator to obtain disguised image data corresponding to the target training data; and inputting the target training data and the disguised image data into an image discriminator to obtain a discrimination result;

[0029] Calculating a first loss function corresponding to the discriminator based on the target training data and the disguised image data, and calculating a second loss function corresponding to the generator based on the disguised image data;

[0030] When it is determined that both the first loss function and the second loss function satisfy a preset training completion condition, a trained denoising model is generated.

[0031] In a second aspect, the present application also provides an image processing device. The device comprises:

[0032] A first acquisition module, configured to acquire an original exposure image, wherein the original exposure image includes an image of at least one exposure time;

[0033] A brightness alignment module, configured to adjust the brightness value of the original exposure image based on a brightness alignment parameter to obtain a brightness aligned first exposure image;

[0034] A distribution adjustment module, configured to adjust the brightness value of the first exposure image based on a preset noise gain parameter and a noise variance to obtain a target exposure image that conforms to a preset distribution;

[0035] The denoising module is used to input the target exposure image into a pre-trained denoising model to obtain a denoised image.

[0036] In one embodiment, the brightness alignment module is specifically used to:

[0037] For a pixel point included in the original exposure image, a brightness difference between a brightness value of the pixel point and a preset brightness is calculated, and brightness correction is performed on the brightness difference of the pixel point using a brightness alignment parameter to obtain a corrected brightness value corresponding to the pixel point, wherein the preset brightness value is determined according to a brightness level offset parameter of a digital-to-analog converter;

[0038] Based on the corrected brightness values ​​corresponding to the pixel points, a first exposure image with brightness alignment is obtained.

[0039] In one embodiment, the preset noise gain parameter is a Poisson noise gain parameter, and the preset distribution is a Poisson distribution; the distribution adjustment module is specifically used to:

[0040] Based on the Poisson distribution conversion algorithm, Poisson noise gain parameter and noise variance, a Poisson distribution conversion factor is calculated;

[0041] For a pixel point included in the first exposure image, a product value of a brightness value corresponding to the pixel point and the Poisson noise gain parameter is calculated, and the product value is converted and calculated using the Poisson distribution conversion factor to obtain a converted brightness value corresponding to the pixel point;

[0042] Based on the converted brightness values ​​corresponding to the pixel points, a target exposure image whose brightness values ​​conform to the Poisson distribution is obtained.

[0043] In one embodiment, the original exposure image includes images with multiple exposure times; and the device further includes:

[0044] A second acquisition module, used to acquire each exposure time included in the original exposure image, and a digital-to-analog converter parameter corresponding to each exposure time;

[0045] The first calculation module is used to respectively calculate the ratio of each exposure time included in the original exposure image and the ratio of the digital-to-analog converter parameters corresponding to each exposure time, and determine the brightness alignment parameter based on each ratio.

[0046] In one embodiment, the target exposure image includes at least two target images with exposure times, including a target image with a first exposure time and a target image with a second exposure time; and the denoising module is specifically configured to:

[0047] Inputting the target image at the first exposure time and the target image at the second exposure time into a pre-trained denoising model to obtain an initial denoised image corresponding to the first exposure time and an initial denoised image corresponding to the second exposure time;

[0048] Pixel data of the initial denoised image corresponding to the first exposure time and pixel data of the initial denoised image corresponding to the second exposure time are fused to obtain a denoised image.

[0049] In one embodiment, the device further comprises:

[0050] A third acquisition module is used to acquire initial training data, wherein the initial training data includes a plurality of noise-free images;

[0051] A second calculation module is used to adjust the brightness value of each of the noise-free images based on the brightness alignment parameter, the preset noise gain parameter and the noise variance to obtain target training data;

[0052] The training module is used to perform unsupervised training on the denoising model to be trained using the target training data to obtain a trained denoising model.

[0053] In one embodiment, the denoising model to be trained includes an image generator and an image discriminator, and the training module is specifically used to:

[0054] Inputting the target training data into a generator to obtain disguised image data corresponding to the target training data; and inputting the target training data and the disguised image data into an image discriminator to obtain a discrimination result;

[0055] Calculating a first loss function corresponding to the discriminator based on the target training data and the disguised image data, and calculating a second loss function corresponding to the generator based on the disguised image data;

[0056] When it is determined that both the first loss function and the second loss function satisfy a preset training completion condition, a trained denoising model is generated.

[0057] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0058] Acquire an original exposure image, wherein the original exposure image includes an image of at least one exposure time;

[0059] Adjusting the brightness value of the original exposure image based on the brightness alignment parameter to obtain a brightness aligned first exposure image;

[0060] Adjusting the brightness value of the first exposure image based on a preset noise gain parameter and a noise variance to obtain a target exposure image that conforms to a preset distribution;

[0061] The target exposure image is input into a pre-trained denoising model to obtain a denoised image.

[0062] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0063] Acquire an original exposure image, wherein the original exposure image includes an image of at least one exposure time;

[0064] Adjusting the brightness value of the original exposure image based on the brightness alignment parameter to obtain a brightness aligned first exposure image;

[0065] Adjusting the brightness value of the first exposure image based on a preset noise gain parameter and a noise variance to obtain a target exposure image that conforms to a preset distribution;

[0066] The target exposure image is input into a pre-trained denoising model to obtain a denoised image.

[0067] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0068] Acquire an original exposure image, wherein the original exposure image includes an image of at least one exposure time;

[0069] Adjusting the brightness value of the original exposure image based on the brightness alignment parameter to obtain a brightness aligned first exposure image;

[0070] Adjusting the brightness value of the first exposure image based on a preset noise gain parameter and a noise variance to obtain a target exposure image that conforms to a preset distribution;

[0071] The target exposure image is input into a pre-trained denoising model to obtain a denoised image.

[0072] The above-mentioned image processing method, device, computer equipment, storage medium and computer program product, the method comprises: obtaining an original exposure image, the original exposure image includes an image of at least one exposure time; adjusting the brightness value of the original exposure image based on a brightness alignment parameter to obtain a first exposure image with brightness alignment; adjusting the brightness value of the first exposure image based on a preset noise gain parameter and a noise variance to obtain a target exposure image that conforms to a preset distribution; inputting the target exposure image into a pre-trained noise reduction model to obtain a noise reduction image. By adopting this method, a noise image related to a signal can be converted into image data with signal independence and near-Gaussian distribution, weakening the influence of noise differences caused by exposure time and parameters, performing noise reduction processing on the image, avoiding noise differences, improving imaging effects and improving the quality of output images. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 is a schematic flow chart of an image processing method in one embodiment;

[0074] Figure 2 is a schematic flow chart of a brightness alignment step in one embodiment;

[0075] Figure 3 A schematic diagram of a process for obtaining a target exposure image in one embodiment;

[0076] Figure 4 A schematic diagram of a flow chart of a step of determining brightness alignment parameters in one embodiment;

[0077] Figure 5 A schematic diagram of a noise reduction process in one embodiment;

[0078] Figure 6 is a flowchart of a training step in an embodiment;

[0079] Figure 7 A schematic diagram of a process for calculating a loss function in one embodiment;

[0080] Figure 8 is a schematic flow chart of an image processing method in another embodiment;

[0081] Fig. 9 Schematic diagram of a training process of an image processing method in one embodiment;

[0082] Fig.10 A schematic diagram of a generator processing flow in an image processing method in one embodiment;

[0083] Fig.11 is a schematic diagram of a discriminator processing flow of an image processing method in one embodiment;

[0084] Fig.12An image schematic diagram of an image processing method in one embodiment;

[0085] Fig.13 An image schematic diagram of an image processing method in one embodiment;

[0086] Fig.14 is a structural block diagram of an image processing device in one embodiment;

[0087] Fig.15 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0088] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0089] In one embodiment, Figure 1 As shown, an image processing method is provided. This embodiment uses the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The above-mentioned terminal can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server can be implemented with an independent server or a server cluster composed of multiple servers. The terminal includes an image acquisition device. In this embodiment, the image processing method includes the following steps:

[0090] Step 102: Acquire an original exposure image.

[0091] The original exposure image includes an image of at least one exposure time, and the image acquisition device included in the terminal may be a sensor image acquisition device; the original exposure image may be an image output by the sensor image acquisition device, and the exposure time may be an exposure value set when acquiring the time. The original exposure image may include an image of one exposure time, or may include images of multiple exposure times, and the exposure time may include long exposure, medium exposure, and short exposure, and at least one exposure time may also include multiple exposure times corresponding to underexposure to overexposure. That is, in one example, the original exposure image may include a long exposure image, a medium exposure image, and a short exposure image.

[0092] Specifically, the terminal can perform image acquisition through the sensor image acquirer to obtain an original exposure image acquired by the sensor image acquirer and containing at least one exposure time; in one example, the original exposure image acquired by the terminal may include an image corresponding to the first exposure time and an image corresponding to the second exposure time, and may also include a long exposure image, a medium exposure image, a short exposure image, and so on.

[0093] Step 104 : adjusting the brightness value of the original exposure image based on the brightness alignment parameter to obtain a brightness aligned first exposure image.

[0094] The brightness alignment parameter is a parameter used to adjust the brightness value of the image so that the brightness value of the adjusted image is in a brightness alignment state. The specific value of the brightness alignment parameter can be determined based on at least one exposure time corresponding to the original exposure image and a digital-to-analog converter parameter. The digital-to-analog converter parameter can be an attribute parameter of the AD converter, for example, a parameter determined based on the magnification of the AD converter, etc. The brightness aligned first exposure image can be: the brightness values ​​of the images corresponding to the multiple exposure times included in the first exposure image are in the same range, that is, the brightness values ​​of each pixel on the image corresponding to each exposure time are in the same preset range.

[0095] Specifically, after the terminal collects the original exposure image containing at least one exposure time, it can adjust the brightness values ​​of the images of each exposure time contained in the original exposure image based on the determined brightness alignment parameters to obtain images corresponding to each exposure time with brightness values ​​within the same brightness value range.

[0096] In one example, the original exposure image may include a first image with an exposure time of a first exposure time and a second image with an exposure time of a second exposure time. Based on this, the specific process of adjusting the brightness value by the terminal may be that, for the image contained in the original exposure image, the terminal may adjust the brightness value of each pixel point contained in the image based on the determined brightness alignment parameter to obtain the adjusted image. The terminal may obtain the brightness-aligned first exposure image based on the second image and the adjusted first image; the terminal may also obtain the brightness-aligned first exposure image based on the first image and the adjusted second image; the terminal may also obtain the brightness-aligned first exposure image based on the adjusted second image and the adjusted first image.

[0097] In another example, when the original exposure image includes images corresponding to multiple exposure times, the terminal can adjust the brightness value of each image based on the brightness alignment parameter for each image in the original exposure image corresponding to the multiple exposure times to obtain the adjusted image; in this way, the terminal can obtain a brightness-aligned first exposure image based on the images corresponding to the adjusted exposure times; the terminal can also screen the images corresponding to the multiple exposure times included in the original exposure image to obtain images to be adjusted that meet the adjustment conditions, and adjust the brightness value of the images to be adjusted based on the brightness alignment parameter to obtain the adjusted image; in this way, the terminal can obtain a brightness-aligned first exposure image based on the adjusted image and the image that does not meet the adjustment condition.

[0098] Step 106 : adjusting the brightness value of the first exposure image based on a preset noise gain parameter and a noise variance to obtain a target exposure image that complies with a preset distribution.

[0099] The preset noise gain parameter is a parameter used to adjust the brightness value of the image so that the image conforms to the preset distribution; the preset noise gain parameter may be a Poisson noise gain parameter, which may reflect the relationship between the number of photons and the signal strength. The noise variance may be variance data of the target noise, for example, the variance of Gaussian noise, i.e., Gaussian variance; the preset distribution may be a distribution state corresponding to the preset noise gain parameter and corresponding to the noise variance; in one example, when the preset noise gain parameter is a Poisson noise gain parameter and the noise variance is a Gaussian variance, the preset distribution may be a Gaussian distribution.

[0100] Specifically, the terminal can determine the preset noise gain parameters and the noise variance. In this way, the terminal can perform a variance-stabilized transformation on the brightness value of the first exposure image based on the determined preset noise gain parameters and the noise variance through a variance-stabilized transformation algorithm to obtain a target exposure image that conforms to the preset distribution.

[0101] In one example, the terminal may adjust the brightness values ​​of images corresponding to each exposure time contained in the first exposure image based on the determined preset noise gain parameters and noise variance through a variance-stabilized transform algorithm to obtain a target exposure image that conforms to a preset distribution; the terminal may also adjust the brightness values ​​of each pixel point contained in the first exposure image to obtain a target exposure image that conforms to a preset distribution.

[0102] Step 108: input the target exposure image into a pre-trained denoising model to obtain a denoised image.

[0103] The pre-trained denoising model is a model for performing denoising on an image, or a model for performing denoising on an image. The denoising model may be a model obtained by training noise-free image data in an unsupervised manner.

[0104] Specifically, the terminal may input the target exposure image into a pre-trained denoising model, and the denoising model may perform denoising processing on the target exposure image based on various computing layers included in the denoising model to obtain a denoised image.

[0105] In the above image processing method, an original exposure image is obtained, and the original exposure image includes an image of at least one exposure time; the brightness value of the original exposure image is adjusted based on the brightness alignment parameter to obtain a first exposure image with brightness alignment; the brightness value of the first exposure image is adjusted based on the preset noise gain parameter and the noise variance to obtain a target exposure image that conforms to the preset distribution; the target exposure image is input into a pre-trained noise reduction model to obtain a noise reduction image. By adopting this method, a noise image related to a signal can be converted into image data with signal independence and near-Gaussian distribution, weakening the influence of noise differences caused by exposure time and parameters, performing noise reduction processing on the image, avoiding noise differences, improving imaging effects, and improving the quality of output images.

[0106] In one embodiment, Figure 2 As shown, the specific processing process of step 104 "adjusting the brightness value of the original exposure image based on the brightness alignment parameter to obtain the first exposure image with brightness alignment" includes:

[0107] Step 202, for the pixel points included in the original exposure image, calculate the brightness difference between the brightness value of the pixel point and the preset brightness, perform brightness correction on the brightness difference of the pixel point by using the brightness alignment parameter, and obtain the corrected brightness value corresponding to the pixel point.

[0108] Among them, the preset brightness value is determined according to the brightness level offset parameter of the digital-to-analog converter, and the brightness level offset parameter of the digital-to-analog converter can be the fixed offset added to the image by the digital-to-analog converter included in the image acquisition device before outputting the image; in one example, the preset brightness value (OB) can be the black level data corresponding to the image acquisition device.

[0109] Specifically, for the images corresponding to each exposure time contained in the original exposure image, the terminal can calculate the brightness difference between each pixel on the image and the preset brightness value, and then perform brightness correction processing on the brightness difference corresponding to the pixel through the brightness alignment parameter to obtain the corrected brightness value.

[0110] Step 204 : obtaining a brightness-aligned first exposure image based on the corrected brightness value corresponding to each pixel point.

[0111] Specifically, the terminal can obtain the image corresponding to each exposure time after brightness alignment based on the corrected brightness values ​​corresponding to each pixel point contained in the image corresponding to each exposure time; in this way, the terminal can obtain the first exposure image with brightness alignment based on the image corresponding to each exposure time after brightness alignment.

[0112] In this embodiment, by performing black level processing on the exposure image through a preset brightness value, it can be ensured that the real brightness information of the exposure image is restored. By processing the exposure image through the brightness alignment parameter, it can be ensured that the brightness of the images corresponding to each exposure time is at the same brightness level, thereby improving image processing efficiency and providing a stable image data foundation.

[0113] In one embodiment, the preset noise gain parameter is a Poisson noise gain parameter, and the preset distribution is a Poisson distribution. Specifically, the terminal can determine the specific type of the preset noise gain parameter based on the counting process of photons reaching the photosensitive device. Since the counting process of photons reaching the photosensitive device is consistent with the Poisson process, the terminal can determine that the preset noise gain parameter is a Poisson noise gain parameter; the terminal can also determine that the noise variance is a Gaussian variance based on the strong correlation between the variance and the signal; the corresponding preset distribution is a Poisson distribution.

[0114] Accordingly, if Figure 3 As shown, the specific processing process of step 106 "adjusting the brightness value of the first exposure image based on the preset noise gain parameter and the noise variance to obtain a target exposure image that meets the preset distribution" includes:

[0115] Step 302: Calculate a Poisson distribution conversion factor based on a Poisson distribution conversion algorithm, a Poisson noise gain parameter, and a noise variance.

[0116] The Poisson distribution conversion algorithm may be a variance stabilization transformation algorithm for converting an image into a image that conforms to the distribution; and the Poisson distribution conversion factor is an influencing factor for performing conversion processing on the first exposure image.

[0117] Specifically, the terminal may perform processing based on the determined Poisson noise gain parameter and noise variance and the Poisson distribution conversion algorithm to calculate the Poisson distribution conversion factor.

[0118] Step 304 , for each pixel included in the first exposure image, calculate the product of the brightness value corresponding to the pixel and the Poisson noise gain parameter, and convert the product using the Poisson distribution conversion factor to obtain a converted brightness value corresponding to the pixel.

[0119] Specifically, for the pixel points contained in the first exposure image, the terminal can calculate the product of the brightness value of the pixel point and the Poisson noise gain parameter. After determining the product of the brightness value of the pixel point and the Poisson noise gain parameter, the terminal can convert the product value corresponding to the pixel point through the Poisson distribution conversion factor to calculate the converted brightness value corresponding to the pixel point.

[0120] In one example, the terminal can calculate the target sum value based on the product of the brightness value of the pixel point and the Poisson noise gain parameter, the sum of the first target value and the square value of the Poisson noise gain parameter, and the square value of the Gaussian variance, and perform operations on the target sum value through a preset operation algorithm to obtain an operation result; based on this, the terminal can process the operation result through the ratio of the second target value and the Poisson noise gain parameter to obtain the converted brightness value corresponding to the pixel point; wherein the first target value can be three-eighths, the preset operation algorithm can be a square root processing algorithm, and the second target value can be 2.

[0121] Step 306 : Based on the converted brightness values ​​corresponding to each pixel point, a target exposure image whose brightness values ​​conform to the Poisson distribution is obtained.

[0122] Specifically, the terminal can obtain a target exposure image whose brightness value conforms to the Poisson distribution through the converted brightness values ​​corresponding to each pixel point. In one example, when the first exposure image includes multiple images corresponding to exposure times, the terminal can obtain a target exposure image whose brightness value conforms to the Poisson distribution based on the converted brightness values ​​corresponding to the pixels contained in each image.

[0123] In this embodiment, by performing variance stabilization transformation on the exposure image, an exposure image in which the brightness values ​​of the pixels meet the preset distribution can be obtained, thereby reducing the difficulty of fusing multiple exposure images, improving the quality of the output image, and ensuring the user experience of the end user.

[0124] In one embodiment, the original exposure image includes images with multiple exposure times. The multiple exposure times correspond to multiple images, wherein each image is obtained by capturing the same environment or the same object, and the exposure time in the image capture parameters of each image is different.

[0125] Accordingly, if Figure 4 As shown, the image processing method also includes:

[0126] Step 402: Acquire each exposure time included in the original exposure image and a digital-to-analog converter parameter corresponding to each exposure time.

[0127] Specifically, the DAC parameter may be a parameter determined based on the gain in the DAC, or the gain in the DAC may be determined as a DAC parameter, such as a gain value. The terminal may determine each exposure time and the DAC parameter corresponding to each exposure time.

[0128] Step 404 , respectively calculating the ratio of each exposure time included in the original exposure image and the ratio of the digital-to-analog converter parameters corresponding to each exposure time, and determining the brightness alignment parameter based on each ratio.

[0129] Specifically, for the ratio between multiple exposure times included in the original exposure image, the first ratio is determined, and the terminal can also calculate the ratio between the digital-to-analog converter parameters corresponding to each exposure time, and determine it as the second ratio. Based on this, the terminal can calculate the product value between the first ratio and the second ratio, and determine the product value as the brightness alignment parameter.

[0130] In one example, the original exposure image may include an image of a first exposure time and an image of a second exposure time, the D / A converter parameter corresponding to the first exposure time may be a first D / A converter parameter, and the D / A converter parameter corresponding to the second exposure time may be a second D / A converter parameter. Based on this, the terminal may calculate a first ratio between the first exposure time and the second exposure time, calculate a second ratio between the first D / A converter parameter and the second D / A converter parameter, and calculate a product value between the first ratio and the second ratio. In this way, the terminal may determine the product value between the first ratio and the second ratio as a brightness alignment parameter.

[0131] In this embodiment, by calculating the brightness alignment parameters through each exposure time and the digital-to-analog converter parameters corresponding to each exposure time, the effectiveness of brightness alignment processing of the brightness value of the image based on the brightness alignment parameters can be ensured, thereby reducing the difficulty and complexity of subsequent image processing.

[0132] In one embodiment, the target exposure image includes target images of at least two exposure times, including a target image of a first exposure time and a target image of a second exposure time.

[0133] Accordingly, if Figure 5 As shown, the specific processing process of step 108 "inputting the target exposure image into the pre-trained denoising model to obtain the denoised image" includes:

[0134] Step 502: Input the target image at the first exposure time and the target image at the second exposure time into a pre-trained denoising model to obtain an initial denoised image corresponding to the first exposure time and an initial denoised image corresponding to the second exposure time.

[0135] Specifically, in the case where the original exposure image includes the original image of the first exposure time and the original image of the second exposure time, based on the steps described in the above embodiment, the target image corresponding to the original image of the first exposure time and the target image corresponding to the original image of the second exposure time can be obtained, that is, the target image that conforms to the preset distribution is obtained. In this way, the terminal can input the target image of the first exposure time and the target image of the second exposure time into a pre-trained denoising model, and perform denoising on the target image of the first exposure time and the target image of the second exposure time through the denoising model, so as to obtain an initial denoised image corresponding to the first exposure time and an initial denoised image corresponding to the second exposure time.

[0136] In one example, the terminal can stitch a target image of a first exposure time and a target image of a second exposure time according to channels to obtain a stitching result, and input the stitching result into a pre-trained denoising model to obtain an initial denoised image corresponding to the first exposure time and an initial denoised image corresponding to the second exposure time.

[0137] Step 504 , the pixel data of the initial denoised image corresponding to the first exposure time and the pixel data of the initial denoised image corresponding to the second exposure time are merged to obtain a denoised image.

[0138] Specifically, the terminal can fuse the pixel data of each pixel point contained in the initial denoised image corresponding to the first exposure time with the pixel data of each pixel point contained in the initial denoised image corresponding to the corresponding second exposure time to obtain the fused pixel data corresponding to each pixel point. In this way, the terminal can generate a fused image, i.e., a denoised image, based on the fused pixel data corresponding to each pixel point.

[0139] In this embodiment, by performing denoising processing on the image through a pre-trained denoising model, it is possible to ensure that the output image is noise-free, improve the imaging effect, and ensure the image quality of the output image.

[0140] In one embodiment, Figure 6 As shown, the image processing method also includes:

[0141] Step 602: Obtain initial training data.

[0142] The initial training data contains multiple noise-free images.

[0143] Specifically, the terminal can obtain multiple noise-free images as initial training data. In one example, the terminal can obtain noise-free images with clarity greater than or equal to a preset clarity threshold as initial training data. The preset clarity threshold can be determined based on the actual application scenario. The present disclosure does not specifically limit the specific value of the preset clarity threshold.

[0144] Step 604 , adjusting the brightness value of each noise-free image based on the brightness alignment parameter, the preset noise gain parameter, and the noise variance to obtain target training data.

[0145] Specifically, after acquiring the initial training data, the terminal can adjust the brightness value of the image included in the initial training data based on the brightness alignment parameter and the preset brightness value to obtain a brightness-aligned image; and adjust the brightness value of the brightness-aligned image based on the preset noise gain parameter and the noise variance to obtain an image that conforms to the preset distribution. Based on this, the terminal can determine the image that conforms to the preset distribution as the target training data.

[0146] Step 606: Perform unsupervised training on the denoising model to be trained using the target training data to obtain a trained denoising model.

[0147] Among them, the denoising model for unsupervised training may include an image generator and an image discriminator. The image generator and the image discriminator may be subjected to adversarial training to obtain a trained denoising model.

[0148] Specifically, the terminal can input the target training data containing multiple noise-free images into the denoising model to be trained. The denoising model can be trained through an unsupervised training mode. The image generator and image discriminator in the denoising model to be trained can be trained adversarially. The image discriminator is trained through the first output result of the image generator, and the image generator is trained through the second output result of the image discriminator. When the image generator meets the preset training completion conditions and the image discriminator meets the preset training completion conditions, the terminal can determine that a trained denoising model has been generated.

[0149] In this embodiment, by performing unsupervised training based on noise-free images to obtain a denoising model, the difficulty of obtaining paired high-definition images and noisy images can be reduced, and a large number of noise-free images can be easily obtained. The cost of training can be reduced while ensuring the training accuracy, thereby achieving stable conversion of original image data into noise-free high-quality image data.

[0150] In one embodiment, the denoising model to be trained includes an image generator (G) and an image discriminator (D).

[0151] like Figure 7 As shown, the specific processing process of step 606 "performing unsupervised training on the denoising model to be trained by using the target training data to obtain a trained denoising model" includes:

[0152] Step 702: input the target training data into the generator to obtain disguised image data corresponding to the target training data, and input the target training data and the disguised image data into the image discriminator to obtain a discrimination result.

[0153] Step 704: Calculate a first loss function corresponding to the image discriminator based on the target training data and the disguised image data, and calculate a second loss function corresponding to the generator based on the disguised image data.

[0154] Step 706: When it is determined that both the first loss function and the second loss function satisfy the preset training completion condition, a trained denoising model is generated.

[0155] Specifically, the denoising model to be trained may be an adversarial model including a generator and a discriminator, for example, the image generator may be a UNet model. In the case where the target training data includes images corresponding to multiple exposure times, the terminal may splice the images corresponding to the multiple exposure times according to the channels to obtain a spliced ​​image. In this way, the terminal may input the spliced ​​image into the image generator in the denoising model to be trained, downsample the spliced ​​image through the image generator, obtain the multi-dimensional features of the spliced ​​image, and splice and convolve the multi-dimensional features of the spliced ​​image according to the channels based on the resolution features of the encoder and the resolution features of the decoder to obtain camouflaged image data.

[0156] Specifically, the image discriminator is a classifier. The image discriminator takes high-definition noise-free data and / or disguised image data (fake image) generated by the generator as input, classifies the high-definition noise-free data and / or disguised image data (fake image) generated by the generator through judgment, and provides a feedback signal of the input data to the image generator.

[0157] In one example, the terminal can calculate the first loss function corresponding to the image discriminator based on the target training data and the disguised image data, and calculate the second loss function corresponding to the generator based on the disguised image data. When it is determined that both the first loss function and the second loss function meet the preset training completion conditions, a trained denoising model is generated; specifically, the terminal can calculate the first loss function L of the image discriminator by the following formula: D :

[0158] L D =-log(D(x))-log(1-D(G(z)));

[0159] Where x represents noise-free high-definition image data, i.e., target training data, G(z) represents the output of the generator, i.e., disguised image data, and z represents image data with noise. D(x) is the discriminator’s judgment result on noise-free high-definition image data, and D(G(z)) is the discriminator’s judgment result on disguised image data.

[0160] The terminal can calculate the second loss function L of the image generator by the following formula G :

[0161] L G = -log(D(G(z))).

[0162] Among them, the terminal can first train the discriminator, fix the model parameters of the generator, input the high-definition data set (target training data) and the samples generated by the generator (camouflaged image data) into the discriminator for judgment, obtain the judgment result, and calculate the loss function corresponding to the discriminator; when the discriminator meets the preset training completion conditions, fix the model parameters of the discriminator, and calculate the second loss function of the image generator to optimize the model parameters of the generator; in this way, the terminal can fix the generator parameters and train the discriminator model when it is determined that the generator meets the preset training completion conditions, and obtain the final trained denoising model through reciprocating iterations.

[0163] In this embodiment, by performing unsupervised training on the model, the difficulty of obtaining paired high-definition images and noisy images can be reduced, and a large number of noise-free images can be easily obtained. The cost of training can be reduced while ensuring the training accuracy, and the stable conversion of original image data into noise-free high-quality image data can be achieved.

[0164] The specific implementation process of the above image processing method is described in detail below in conjunction with a specific embodiment:

[0165] With the continuous development of modern technology, the widespread popularity of image acquisition such as mobile phones, CCD and CMOS cameras, people are pursuing a higher quality visual experience. For example, terminal devices support the shooting of high dynamic range HDR images, which can simultaneously present image details of bright parts and dark parts in one photo, ensuring the output of real, vivid and impactful images.

[0166] In the related technology, images from underexposure to overexposure are continuously output in the time domain, and then HDR images are obtained through image fusion. However, the image interval of this scheme is the exposure time of a single frame, the local motion is large, and ghosting is prone to occur during fusion. Digital Overlap (DOL) is a sensor multi-frame technology of Sony that supports "quasi-simultaneous" output of multiple frames of images with different exposure times. Its line interleaving feature causes overlap between two frames of exposure images, reduces the image interval, and is more friendly to ghosting. Although DOLHDR technology can improve image effects, the noise and brightness differences in the long and short exposure images themselves will increase the difficulty of fusion. In addition, the fused image will also produce SNRdrop, so it is crucial to improve the quality of long and short exposure images fused by DOLHDR.

[0167] The application scenario of this solution is two-exposure or three-exposure images collected by DOL technology, and it processes multiple frames of long and short exposure images. Based on the principle of optical imaging, this solution first performs black level correction on the raw image, and then uses it for processing. At this time, the x in the transformation formula represents the real data obtained by the digital-to-analog conversion of the real number of photons. In addition, the VST transformation formula of this solution does not have the parameter M. In addition, this solution processes pairs of long and short exposure images, so brightness alignment operations are performed before transforming the short frames. In terms of denoising models, retrieval result 1 uses convolutional neural network processing. Considering that there are actually no paired high-definition training sets for model training, this solution uses unsupervised training, that is, a generative adversarial model is used to generate high-quality images.

[0168] The disadvantage is that the traditional time domain multi-frame method has a large frame interval, complex local image motion, and is prone to ghosting. DOLHDR technology can effectively eliminate fusion ghosting, but the noise difference and brightness difference of the long and short exposure images themselves will cause problems such as SNR drop in the fusion result, resulting in reduced image quality.

[0169] The image processing method provided in this embodiment is a DOLHDR denoising algorithm based on VST variance stabilization transform, which combines traditional knowledge with deep learning, uses image transformation to convert signal-related noise images into near-Gaussian distribution data, weakens the noise morphology difference between long-exposure and short-exposure images to a certain extent, and generates high-quality images for subsequent algorithms by generating adversarial model denoising. Figure 8 As shown, this is a DOLHDR denoising algorithm process based on VST transformation:

[0170] First, the terminal needs to preprocess the long and short exposure images, including black level correction, brightness alignment and VST transformation, to obtain the target exposure image; based on this, the terminal can generate adversarial model noise reduction processing and output long and short exposure high-quality results, that is, denoised images.

[0171] In one example, the long and short exposure image preprocessing may include black level correction, brightness alignment, and VST conversion.

[0172] Specifically, when the image acquisition device (camera) in the terminal forms an image, the information collected by the CMOS sensor will be converted to generate raw RAW format data. The accuracy of the AD chip (digital-to-analog conversion chip) cannot convert a small part of the voltage value. Therefore, the sensor will add a fixed offset before the AD output to keep the output pixel value within a certain range (for example, 8-bit data is kept between 5-255) to fully retain the details of the dark part.

[0173] The terminal can achieve black level correction through the following formula:

[0174] I'=I–OB,

[0175] Among them, I is the original image obtained, I' is the black level correction result, OB is the black level data, 10-bit data is generally 64, black level correction can prevent the image from deviating from the brightness of the real black, and the long and short exposure images obtained by DOL technology can restore the real brightness information of the image.

[0176] The terminal can achieve brightness alignment through the following formula:

[0177] S_L=(S–OB)×R,

[0178] Among them, S is the short exposure image collected by DOL technology, S-OB is the black level correction, OB of 10-bit data is usually 64, and S_L is the result after brightness alignment. Through brightness alignment, the image is pulled to the same brightness level.

[0179] The terminal can calculate the brightness alignment parameter using the following formula:

[0180]

[0181] Among them, exp and gain are the exposure time and gain value of the image respectively; R is the brightness alignment parameter, which can be obtained according to the exposure time of the long and short exposure images and the set gain parameters and other information.

[0182] The terminal can implement VST conversion through the following VST formula:

[0183]

[0184] Among them, a is the gain parameter of Poisson noise, which reflects the relationship between the number of photons and signal intensity, b is the Gaussian variance, and t is the transformed data, which obeys an approximate Gaussian distribution.

[0185] Optionally, in CCD / CMOS imaging systems, the image signal noise basically conforms to the Poisson-Gaussian joint distribution, where Gaussian noise corresponds to the noise of the photosensitive device itself, and the Poisson process corresponds to the counting process of photons reaching the photosensitive device, and its variance is strongly correlated with the signal. For long and short exposure images captured by DOL technology, the image brightness is inconsistent, the exposure time and analog amplification gain value are different, and the noise morphology is quite different, which increases the difficulty of fusion. Therefore, it is crucial to convert the noisy image into signal-independent data.

[0186] In one example, the terminal can generate an adversarial model to achieve image denoising, where deep learning requires a large amount of training data, but in fact there are no pairs of high-definition noisy images. The image processing method provided in this embodiment can achieve model training through an unsupervised deep learning model, namely, an adversarial model, without requiring a one-to-one correspondence between target images and original noisy data. Only high-definition noise-free images are needed, and a generator can be constructed through training optimization to convert raw data into high-quality images.

[0187] like Fig. 9 As shown in the figure, it is the GAN model architecture. The high-definition data set is used to generate a Gaussian-Poisson noise image, which is input into the generation model to obtain a fake image. The high-definition data set (real image) and the fake image are then input into the discriminant model to obtain the discrimination results, including fake / real.

[0188] like Fig.10As shown, this is the processing flow of the generator model (UNet model) in the GAN model: Taking the input of three exposures of long, medium and short as an example, the terminal can splice the input image by channel, and then send it to the model for processing. The model uses 5 downsampling to obtain multi-granularity features, and then splices and convolves the features of the corresponding resolution in the encoder and the features of the corresponding resolution in the decoder by channel to prevent information loss. For example, it is possible to encode features of 3×96×96 dimensions to obtain features of 32×48×48 dimensions, and input the features of 32×48×48 dimensions to the decoder for processing to obtain features of 3×96×96 dimensions; it is also possible to encode features of 32×48×48 dimensions to obtain features of 64×24×24 dimensions, and input the features of 64×24×24 dimensions to the decoder for processing to obtain features of 32×48×48 dimensions; it is also possible to encode features of 64×24×24 dimensions to obtain features of 128×12×12 dimensions, The features of 128×12×12 dimensions are input into the decoder for processing to obtain features of 64×24×24 dimensions. The features of 128×12×12 dimensions can be encoded to obtain features of 256×6×6 dimensions, and the features of 256×6×6 dimensions are input into the decoder for processing to obtain features of 128×12×12 dimensions. The features of 256×6×6 dimensions can be encoded to obtain features of 256×3×3 dimensions, and the features of 256×3×3 dimensions are input into the decoder for processing to obtain features of 256×6×6 dimensions.

[0189] like Fig.11 As shown in the figure, it is the processing flow of the GAN discriminator: decode the 3×96×96 dimensional features to obtain the 32×48×48 dimensional features, decode the 32×48×48 dimensional features to obtain the 64×23×23 dimensional features, decode the 64×23×23 dimensional features to obtain the 128×10×10 dimensional features, decode the 128×10×10 dimensional features to obtain the 256×7×7 dimensional features, and decode the 256×7×7 dimensional features to obtain the 1×6×6 dimensional features.

[0190] The GAN discriminator is a classifier. It accepts samples (high-definition datasets or fake images generated by the generator) as input, classifies the samples through judgment, and provides feedback signals for the generated samples to the generator.

[0191] Specifically, the image discriminator is a classifier. The image discriminator takes high-definition noise-free data and / or disguised image data (fake image) generated by the generator as input, classifies the high-definition noise-free data and / or disguised image data (fake image) generated by the generator through judgment, and provides a feedback signal of the input data to the image generator.

[0192] In one example, the terminal can calculate the first loss function corresponding to the image discriminator based on the target training data and the disguised image data, and calculate the second loss function corresponding to the generator based on the disguised image data. When it is determined that both the first loss function and the second loss function meet the preset training completion conditions, a trained denoising model is generated; specifically, the terminal can calculate the first loss function L of the image discriminator by the following formula: D :

[0193] L D =-log(D(x))-log(1-D(G(z)));

[0194] Where x represents noise-free high-definition image data, i.e., target training data, G(z) represents the output of the generator, i.e., disguised image data, and z represents image data with noise. D(x) is the discriminator’s judgment result on noise-free high-definition image data, and D(G(z)) is the discriminator’s judgment result on disguised image data.

[0195] The terminal can calculate the second loss function L of the image generator by the following formula G :

[0196] L G = -log(D(G(z))).

[0197] Among them, the terminal can first train the discriminator, fix the model parameters of the generator, input the high-definition data set (target training data) and the samples generated by the generator (camouflaged image data) into the discriminator for judgment, obtain the judgment result, and calculate the loss function corresponding to the discriminator; when the discriminator meets the preset training completion conditions, fix the model parameters of the discriminator, and calculate the second loss function of the image generator to optimize the model parameters of the generator; in this way, the terminal can fix the generator parameters and train the discriminator model when it is determined that the generator meets the preset training completion conditions, and obtain the final trained denoising model through reciprocating iterations.

[0198] The denoising model is trained based on a preset software package, for example, it can be trained using the Keras package on an NVIDIA GTX 1080Ti GPU. The network is optimized using stochastic gradient descent (SGD) with a batch size of 64. The parameters of momentum and weight decay are 0.9 and 0.0001, respectively. In addition, the learning rate is initialized to 0.01 and decreases by one tenth every 30 rounds. Training for 200 rounds takes 30 hours.

[0199] The image processing method provided in this embodiment is based on the DOLHDR denoising algorithm of VST transformation. The core idea is to use the VST transformation formula to convert the signal-related noise image into image data with signal independence and near-Gaussian distribution, thereby weakening the influence of noise differences caused by exposure time and parameters. Fig.12 as well as Fig.13 The denoising results of this scheme are shown. On the left is the HDR result synthesized from the original long and short exposure images, in the middle is the result of adjusting the parameters of the original data, and on the far right is the HDR image output by this scheme. It can be seen that there are noise differences in the long and short exposure images themselves, which will cause misjudgments during image fusion, and ghost images will be produced in the fusion results. Although short frames are selected in the motion area after parameter adjustment, which greatly avoids ghost images, there are noise differences (SNR drop) at the junction of long and short exposures, which affects the subjective effect. This scheme performs denoising on the long and short exposure images. There is no noise in the processed images, which will reduce misjudgments during fusion, and there will be no noise differences in the fusion results, and the imaging effect is the best. Therefore, it has been verified that the present invention is an effective scheme.

[0200] The image processing method provided in this embodiment combines traditional prior knowledge and deep learning methods, and proposes a method that can convert signal-related multi-exposure images into images with near-Gaussian distribution noise, and use a generative adversarial model for processing. Through the VST transformation, the correlation between noise and brightness can be hidden, reducing the difficulty of model denoising. Experimental results show that incorporating prior knowledge such as VST transformation into the network can effectively improve the model effect. The DOLHDR image processed using this solution has a closer noise morphology, better noise reduction effect, and is more beneficial to subsequent HDR fusion.

[0201] The image processing method provided in this embodiment uses VST transformation to convert the signal-related noise image into noise-independent and approximately Gaussian distributed data, which can be applied to the demosaic field. In the image preprocessing stage, VST transformation is used. In addition, this part can be integrated into the model, and the convolution operation can be used to adaptively perform VST transformation to reduce the preprocessing time. At this time, a supervision loss can also be added to the generator to facilitate model optimization. It is applicable to image data obtained by any multi-exposure method, including but not limited to DOL HDR, DCG HDR, etc.

[0202] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0203] Based on the same inventive concept, the embodiment of the present application also provides an image processing device for implementing the above-mentioned image processing method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image processing device embodiments provided below can refer to the limitations on the image processing method above, and will not be repeated here.

[0204] In one embodiment, Fig.14 As shown, an image processing device 1400 is provided, comprising:

[0205] A first acquisition module 1402 is used to acquire an original exposure image, where the original exposure image includes an image of at least one exposure time;

[0206] A brightness alignment module 1404 is used to adjust the brightness value of the original exposure image based on the brightness alignment parameter to obtain a brightness aligned first exposure image;

[0207] A distribution adjustment module 1406 is used to adjust the brightness value of the first exposure image based on a preset noise gain parameter and a noise variance to obtain a target exposure image that meets a preset distribution;

[0208] The denoising module 1408 is used to input the target exposure image into a pre-trained denoising model to obtain a denoised image.

[0209] In one embodiment, the brightness alignment module is specifically used to:

[0210] For a pixel point included in the original exposure image, a brightness difference between a brightness value of the pixel point and a preset brightness is calculated, and brightness correction is performed on the brightness difference of the pixel point using a brightness alignment parameter to obtain a corrected brightness value corresponding to the pixel point, wherein the preset brightness value is determined according to a brightness level offset parameter of a digital-to-analog converter;

[0211] Based on the corrected brightness values ​​corresponding to the pixel points, a first exposure image with brightness alignment is obtained.

[0212] In one embodiment, the preset noise gain parameter is a Poisson noise gain parameter, and the preset distribution is a Poisson distribution; the distribution adjustment module is specifically used to:

[0213] Based on the Poisson distribution conversion algorithm, Poisson noise gain parameter and noise variance, a Poisson distribution conversion factor is calculated;

[0214] For a pixel point included in the first exposure image, a product value of a brightness value corresponding to the pixel point and the Poisson noise gain parameter is calculated, and the product value is converted and calculated using the Poisson distribution conversion factor to obtain a converted brightness value corresponding to the pixel point;

[0215] Based on the converted brightness values ​​corresponding to the pixel points, a target exposure image whose brightness values ​​conform to the Poisson distribution is obtained.

[0216] In one embodiment, the original exposure image includes images with multiple exposure times; and the device further includes:

[0217] A second acquisition module, used to acquire each exposure time included in the original exposure image, and a digital-to-analog converter parameter corresponding to each exposure time;

[0218] The first calculation module is used to respectively calculate the ratio of each exposure time included in the original exposure image and the ratio of the digital-to-analog converter parameters corresponding to each exposure time, and determine the brightness alignment parameter based on each ratio.

[0219] In one embodiment, the target exposure image includes at least two target images with exposure times, including a target image with a first exposure time and a target image with a second exposure time; and the denoising module is specifically configured to:

[0220] Inputting the target image at the first exposure time and the target image at the second exposure time into a pre-trained denoising model to obtain an initial denoised image corresponding to the first exposure time and an initial denoised image corresponding to the second exposure time;

[0221] Pixel data of the initial denoised image corresponding to the first exposure time and pixel data of the initial denoised image corresponding to the second exposure time are fused to obtain a denoised image.

[0222] In one embodiment, the device further comprises:

[0223] A third acquisition module is used to acquire initial training data, wherein the initial training data includes a plurality of noise-free images;

[0224] A second calculation module is used to adjust the brightness value of each of the noise-free images based on the brightness alignment parameter, the preset noise gain parameter and the noise variance to obtain target training data;

[0225] The training module is used to perform unsupervised training on the denoising model to be trained using the target training data to obtain a trained denoising model.

[0226] In one embodiment, the denoising model to be trained includes an image generator and an image discriminator, and the training module is specifically used for:

[0227] Inputting the target training data into a generator to obtain disguised image data corresponding to the target training data; and inputting the target training data and the disguised image data into an image discriminator to obtain a discrimination result;

[0228] Calculating a first loss function corresponding to the discriminator based on the target training data and the disguised image data, and calculating a second loss function corresponding to the generator based on the disguised image data;

[0229] When it is determined that both the first loss function and the second loss function satisfy a preset training completion condition, a trained denoising model is generated.

[0230] Each module in the above image processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0231] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.15 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store image data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image processing method is implemented.

[0232] Those skilled in the art will understand that Fig.15The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0233] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0234] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0235] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0236] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0237] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0238] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0239] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an original exposure image, wherein the original exposure image includes an image of at least one exposure time; Adjusting the brightness value of the original exposure image based on the brightness alignment parameter to obtain a brightness aligned first exposure image; Adjusting the brightness value of the first exposure image based on a preset noise gain parameter and a noise variance to obtain a target exposure image that conforms to a preset distribution; The target exposure image is input into a pre-trained denoising model to obtain a denoised image.

2. The method according to claim 1, characterized in that The step of adjusting the brightness value of the original exposure image based on the brightness alignment parameter to obtain a brightness aligned first exposure image includes: For a pixel point included in the original exposure image, a brightness difference between a brightness value of the pixel point and a preset brightness is calculated, and brightness correction is performed on the brightness difference of the pixel point using a brightness alignment parameter to obtain a corrected brightness value corresponding to the pixel point, wherein the preset brightness value is determined according to a brightness level offset parameter of a digital-to-analog converter; Based on the corrected brightness values ​​corresponding to the pixel points, a first exposure image with brightness alignment is obtained.

3. The method according to claim 1, characterized in that The preset noise gain parameter is a Poisson noise gain parameter, and the preset distribution is a Poisson distribution; the brightness value of the first exposure image is adjusted based on the preset noise gain parameter and the noise variance to obtain a target exposure image that conforms to the preset distribution, including: Based on the Poisson distribution conversion algorithm, Poisson noise gain parameter and noise variance, a Poisson distribution conversion factor is calculated; For a pixel point included in the first exposure image, a product value of a brightness value corresponding to the pixel point and the Poisson noise gain parameter is calculated, and the product value is converted and calculated using the Poisson distribution conversion factor to obtain a converted brightness value corresponding to the pixel point; Based on the converted brightness values ​​corresponding to the pixel points, a target exposure image whose brightness values ​​conform to the Poisson distribution is obtained.

4. The method according to claim 1, characterized in that: The original exposure image includes images with various exposure times; the method further includes: Acquire each exposure time included in the original exposure image, and a digital-to-analog converter parameter corresponding to each exposure time; The ratio of each exposure time included in the original exposure image and the ratio of the digital-to-analog converter parameters corresponding to each exposure time are calculated respectively, and the brightness alignment parameter is determined based on each ratio.

5. The method according to claim 1, characterized in that The target exposure image includes at least two target images with exposure times, including a target image with a first exposure time and a target image with a second exposure time; and inputting the target exposure image into a pre-trained denoising model to obtain a denoised image includes: Inputting the target image at the first exposure time and the target image at the second exposure time into a pre-trained denoising model to obtain an initial denoised image corresponding to the first exposure time and an initial denoised image corresponding to the second exposure time; Pixel data of the initial denoised image corresponding to the first exposure time and pixel data of the initial denoised image corresponding to the second exposure time are fused to obtain a denoised image.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Acquire initial training data, wherein the initial training data includes a plurality of noise-free images; Adjusting the brightness value of each of the noise-free images based on the brightness alignment parameter, the preset noise gain parameter, and the noise variance to obtain target training data; The denoising model to be trained is trained unsupervisedly using the target training data to obtain a trained denoising model.

7. The method according to claim 6, characterized in that The denoising model to be trained includes an image generator and an image discriminator. The denoising model to be trained is subjected to unsupervised training using the target training data to obtain a trained denoising model, including: Inputting the target training data into a generator to obtain disguised image data corresponding to the target training data; and inputting the target training data and the disguised image data into an image discriminator to obtain a discrimination result; Calculating a first loss function corresponding to the discriminator based on the target training data and the disguised image data, and calculating a second loss function corresponding to the generator based on the disguised image data; When it is determined that both the first loss function and the second loss function satisfy a preset training completion condition, a trained denoising model is generated.

8. An image processing device, characterized in that: The device comprises: A first acquisition module, configured to acquire an original exposure image, wherein the original exposure image includes an image of at least one exposure time; A brightness alignment module, configured to adjust the brightness value of the original exposure image based on a brightness alignment parameter to obtain a brightness aligned first exposure image; A distribution adjustment module, configured to adjust the brightness value of the first exposure image based on a preset noise gain parameter and a noise variance to obtain a target exposure image that conforms to a preset distribution; The denoising module is used to input the target exposure image into a pre-trained denoising model to obtain a denoised image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images