Stacking noise reduction method, device, electronic device and storage medium
By stacking noise reduction and grayscale transformation processing on multi-frame night scene images, combined with weighted fusion and single-frame image noise reduction, the problem of high noise is solved, and a clearer super night scene image is generated, improving the user experience.
Patent Information
- Application Number
- CN202011065546.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-01-07
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2040-01-07
AI Technical Summary
The prior art is noisy when generating super night scene images, resulting in poor enhancement effects, especially in night scene photos where the bright or dark parts of the image are not clear enough.
By acquiring continuous multi-frame original images, including one frame of underexposed images and multiple frames of normal exposure images, stacked noise reduction and grayscale transformation processing are performed, and super night scene images are generated by fusion, and noise reduction is reduced by weighted fusion and single-frame image noise reduction. , and use detail enhancement algorithm to improve image clarity.
It effectively reduces the noise of night scene images, improves the clarity and user experience of the image, especially in night scenes, which significantly improves the brightness and contrast of the image.
Smart Images

Figure CN112288646B_ABST
Abstract
Description
[0001] This application is a divisional application of the invention patent application filed by the applicant Yingshi Innovation Technology Co., Ltd. on January 7, 2020 to the State Intellectual Property Office of China with application number 202010014410.7, application date January 7, 2020, and name “Method, device, electronic device and storage medium for generating super night scene images”. Technical Field
[0002] The present invention belongs to the field of image processing technology, and in particular relates to a method, device, electronic device and storage medium for generating a super night scene image. Background Art
[0003] Traditional night scene enhancement processing typically involves enhancing the captured night scene image using a brightness or contrast enhancement algorithm after the image is captured. This improves the brightness and contrast of dark areas. However, since night scene images are often noisy and this noise can be enhanced, the overall enhancement effect is poor. Furthermore, schemes that determine the enhancement coefficient based on local variance are prone to causing halos in areas with extreme brightness and darkness.
[0004] There is a solution in the existing technology that uses multiple frames of images to generate high dynamic range images, but this solution works well in scenes shot during the day but has a lot of noise in night photos. There is also a processing solution in the existing technology that directly fuses and removes noise from multiple frames of night scene images, but the night scene images obtained by this solution are still not clear enough in the bright or dark parts of the image. Summary of the Invention
[0005] The object of the present invention is to provide a method, device, electronic device and storage medium for generating a super night scene image, aiming to solve the problem of high noise in the super night scene images generated in the prior art.
[0006] In one aspect, the present invention provides a method for generating a super night scene image, the method comprising the following steps:
[0007] Acquire a plurality of consecutive frames of original images, wherein the plurality of frames of original images include one frame of underexposed image and a plurality of frames of normally exposed images;
[0008] Performing stacking noise reduction processing on the multiple frames of normal exposure images to obtain a frame of normal noise reduction image;
[0009] Performing grayscale transformation on the normal noise reduction image to obtain an overexposed image;
[0010] The underexposed image, the normal noise reduction image and the overexposed image are fused to obtain a super night scene image.
[0011] Preferably, the step of performing stacking noise reduction processing on the multiple frames of normal exposure images to obtain a frame of normal noise reduction image includes:
[0012] Performing weighted fusion noise reduction processing on the multiple frames of normal exposure images to obtain a frame of stacked noise reduction image;
[0013] The stacked denoised image is subjected to denoising processing using a single-frame image denoising method to obtain the normal denoised image.
[0014] Preferably, the step of performing weighted fusion noise reduction processing on the multiple frames of normal exposure images to obtain a frame of stacked noise-reduced image includes:
[0015] Selecting one frame of image from the multiple frames of normally exposed images as a reference image;
[0016] performing alignment processing on the images other than the reference image in the multiple frames of normally exposed images with the reference image;
[0017] Performing weighted fusion noise reduction processing on the aligned multiple frames of normal exposure images according to the first formula to obtain the stacked noise reduction image;
[0018] Among them, the first formula is: represents the stacked denoised image, I i (x, y) represents the i-th image in the multiple frames of normal exposure images after alignment processing, M-1 represents the number of frames of the normal exposure image, w i (x, y) represents the weight of weighted fusion, the weight w i (x,y) is calculated using the following formula:
[0019] d(x,y)=|I i (x, y)-I0(x, y)|, where I0(x, y) represents the reference image and n represents the intensity value of the preset image noise.
[0020] Preferably, the step of performing weighted fusion noise reduction processing on the aligned multiple frames of normally exposed images according to the first formula includes:
[0021] In the YUV space, weighted fusion noise reduction processing is performed on the aligned multiple frames of normally exposed images according to the first formula.
[0022] Preferably, the step of performing weighted fusion noise reduction processing on the aligned multiple frames of normally exposed images according to the first formula in the YUV space includes:
[0023] Acquire the Y component data of the multiple frames of normally exposed images after alignment processing, and the U component data and V component data of the reference image;
[0024] performing weighted fusion noise reduction processing on the Y component data according to the first formula, and performing edge-preserving filtering processing on the U component data and the V component data;
[0025] The Y component data after the weighted fusion noise reduction process, and the U component data and the V component data after the edge-preserving filtering process are synthesized to obtain the stacked noise reduction image.
[0026] Preferably, the step of performing noise reduction processing on the stacked noise-reduced image using a single-frame image noise reduction method comprises:
[0027] Randomly select several reference pixels in the neighborhood window of the target pixel;
[0028] Acquire pixel blocks with the target pixel point and each of the reference pixel points as centers respectively;
[0029] Performing DCT transformation on each of the pixel blocks, and updating the DCT coefficients corresponding to each of the pixel blocks according to a preset threshold;
[0030] Performing an inverse DCT transform on the updated DCT coefficients to reconstruct each of the pixel blocks;
[0031] The pixel values of the pixel points corresponding to the position of the target pixel point in each reconstructed pixel block are weighted averaged, and the pixel value after the weighted average is used as the pixel value of the target pixel point.
[0032] Preferably, the step of updating the DCT coefficient corresponding to each pixel block according to a preset threshold comprises:
[0033] The coefficients of the DCT coefficients that are smaller than the preset threshold are set to zero.
[0034] Preferably, after the step of fusing the underexposed image, the normal noise reduction image and the overexposed image to obtain a super night scene image, the following steps are included:
[0035] Performing detail enhancement processing on the super night scene image using a detail enhancement algorithm to obtain a detail-enhanced super night scene image;
[0036] Wherein, the detail enhancement algorithm is a detail enhancement algorithm based on edge-preserving filtering, and the super night scene image after detail enhancement is: I′(x,y)=k1I(x,y)+(1-k1)S1(x,y), wherein I′(x,y) represents the super night scene image after detail enhancement, I(x,y) represents the super night scene image, S1(x,y) represents the image obtained after edge-preserving filtering of the super night scene image, k1 represents the coefficient of image detail enhancement, and k1>1; or
[0037] The detail enhancement algorithm is a detail enhancement algorithm based on ordinary filtering, and the super night scene image after detail enhancement is: I′(x, y)=max(min(T(x, y),I d (x,y)),I e (x, y)), where T(x, y) = k2I(x, y) + (1-k2)S2(x, y), I′(x, y) represents the detail-enhanced super night scene image, I(x, y) represents the super night scene image, S2(x, y) represents the image obtained by performing ordinary filtering on the super night scene image, and I d (x, y) represents the image obtained after the super night scene image is expanded, I e (x, y) represents the image obtained after the super night scene image is corroded, k2 represents the coefficient of image detail enhancement, and k2>1.
[0038] Preferably, the step of performing grayscale transformation processing on the normal denoised image includes:
[0039] Performing gamma transformation processing on the normal noise reduction image.
[0040] Preferably, the gamma coefficient when performing gamma transformation processing on the normal noise reduction image is 0.625.
[0041] In another aspect, the present invention provides a device for generating a super night scene image, the device comprising:
[0042] An image acquisition unit, configured to acquire a plurality of consecutive original image frames, wherein the plurality of original image frames include one underexposed image frame and a plurality of normally exposed image frames;
[0043] a stacking noise reduction unit, configured to perform stacking noise reduction processing on the plurality of frames of normal exposure images to obtain a frame of normal noise reduction image;
[0044] a grayscale conversion unit, configured to perform grayscale conversion processing on the normal noise reduction image to obtain an overexposed image; and
[0045] The image synthesis unit is used to fuse the underexposed image, the normal noise reduction image and the overexposed image to obtain a super night scene image.
[0046] On the other hand, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.
[0047] On the other hand, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.
[0048] The present invention obtains multiple frames of continuous original images, which include one frame of underexposed image and multiple frames of normally exposed images, performs stacking noise reduction processing on the multiple frames of normally exposed images to obtain a frame of normal noise reduction image, performs grayscale transformation processing on the normal noise reduction image to obtain a frame of overexposed image, and fuses the underexposed image, the normal noise reduction image and the overexposed image to obtain a frame of super night scene image, thereby reducing the noise of the night scene image and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 This is a flowchart of the method for generating a super night scene image provided in the first embodiment of the present invention;
[0050] Figure 2 is a flowchart of a method for generating a super night scene image provided by the second embodiment of the present invention;
[0051] Figure 3 is a structural diagram of a device for generating a super night scene image provided by Embodiment 3 of the present invention; and
[0052] Figure 4 It is a structural diagram of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0054] The following describes the specific implementation of the present invention in detail with reference to specific embodiments:
[0055] Example 1:
[0056] Figure 1The implementation process of the method for generating a super night scene image provided by the first embodiment of the present invention is shown. For ease of explanation, only the part related to the embodiment of the present invention is shown, which is detailed as follows:
[0057] In step S101 , a plurality of consecutive frames of original images are acquired, where the plurality of frames of original images include one frame of underexposed image and a plurality of frames of normally exposed images.
[0058] The embodiment of the present invention is applicable to electronic devices, which may include mobile phones, watches, tablets, computers, cameras and other devices with camera functions. In the embodiment of the present invention, multiple frames of original images continuously shot by the electronic device in a dark place are obtained. The dark place may be at night or other scenes with low light. For ease of explanation, the number of frames of the original image in this example is represented by M, M≥3, and the multiple frames of original images should include one frame of underexposed image and multiple frames of normally exposed images. In practice, for ease of operation, the underexposed image is usually the first frame of image obtained. For example, the continuously shot original images are 9 frames, the first frame is an underexposed image, and the next 8 frames are normally exposed images.
[0059] In step S102 , stacking noise reduction processing is performed on the multiple frames of normal exposure images to obtain a frame of normal noise reduction image.
[0060] In the embodiment of the present invention, when performing stacking noise reduction processing on the multiple frames of normal exposure images to obtain a frame of normal noise reduction image, the following steps may be specifically included:
[0061] S1021: Perform weighted fusion noise reduction processing on the multiple frames of normal exposure images to obtain a frame of stacked noise reduction image.
[0062] Specifically, a frame of image is selected from the multiple frames of normal exposure images as a reference image, and the images other than the reference image in the multiple frames of normal exposure images are aligned with the reference image, and then the multiple frames of normal exposure images after alignment are subjected to weighted fusion noise reduction processing to obtain the stacked noise reduction image, so as to reduce the adverse effects caused by the image not being present. Among them, the first frame of image in the normal exposure image can be selected as the reference image, the second frame of image in the normal exposure image can be selected as the reference image, and the last frame of image in the normal exposure image can also be selected as the reference image, which is not limited here. When the images other than the reference image in the multiple frames of normal exposure images are aligned with the reference image, since the alignment of a certain frame of image with other frames of image belongs to the existing technology, such as alignment can be performed through sparse optical flow, it will not be described in detail here.
[0063] Preferably, weighted fusion noise reduction processing is performed on the multiple frames of normal exposure images after alignment processing according to the first formula to obtain the stacked noise reduction image;
[0064] Among them, the first formula is: Represents the stacked denoised image, I i (x, y) represents the i-th image in the multiple frames of normal exposure images after alignment processing, M-1 represents the number of frames of the normal exposure image, and w i (x, y) represents the weight of weighted fusion. The weight of weighted fusion is determined by the difference between the current image and the reference image. The greater the difference, the smaller the weight. The weight w i (x,y) is calculated using the following formula:
[0065] d(x,y)=|I i (x,y)-I0(x,y)|, where I0(x,y) represents the reference image and d(x,y) represents the multi-frame normal exposure image I after M-1 alignment processing. i The brightness difference between the i-th image in (x, y) and the reference image I0(x, y), n represents the intensity value of the preset image noise. The noise intensity n is determined by the ISO of the electronic device, the exposure time and the properties of the sensor itself. For the same sensor, ISO represents the photosensitivity speed of the CCD or CMOS photosensitive element. The larger the ISO, the larger n is, and the shorter the exposure time, the larger n is.
[0066] It should be noted that weighted fusion denoising can be performed on the aligned multiple frames of normal exposure images according to the first formula in RGB space. Preferably, weighted fusion denoising can be performed on the aligned multiple frames of normal exposure images according to the first formula in YUV space. Further preferably, the Y component data of the aligned multiple frames of normal exposure images and the U component data and V component data of the reference image are obtained, weighted fusion denoising is performed on the Y component data according to the first formula, and edge-preserving filtering is performed on the U component data and the V component data. The Y component data after weighted fusion denoising and the U component data and the V component data after edge-preserving filtering are synthesized to obtain the stacked denoised image, thereby improving the computational speed of the weighted fusion denoising process. The edge-preserving filtering can be bilateral filtering or guided filtering, which is not limited here; the stacked denoised image can be an image stored in YUV space or an image converted to RGB space or other color space, which is also not limited here.
[0067] S1022: Generate a normal denoised image by using a single-frame image denoising method on the stacked denoised images.
[0068] A single-frame image denoising method is applied to the stacked denoised image to generate a normal denoised image, further eliminating the effects of noise. The single-frame image denoising method is performed in discrete cosine space. When using discrete cosine transform (DCT) for denoising, one pixel is slid at a time. For ease of explanation, in this embodiment, the pixel corresponding to each slide is referred to as a target pixel. Preferably, several reference pixels are randomly selected within a neighbourhood window of the target pixel. Pixel blocks are acquired with the target pixel and each of the reference pixels as centres. A discrete cosine transform (DCT) is performed on each pixel block. The DCT coefficients corresponding to each pixel block are updated according to a preset threshold. The updated DCT coefficients are then inversely transformed to reconstruct each pixel block. The pixel values of the pixels corresponding to the target pixel in each reconstructed pixel block are weighted averaged, and the weighted averaged pixel value is used as the pixel value of the target pixel, thereby effectively improving the denoising effect of the image. The size of the neighborhood window is typically 8×8, and accordingly, the size of each pixel block is also typically 8×8. The preset threshold is typically set manually, and the threshold can be set based on the image noise level. The noise level can be determined by the ISO, camera exposure time, and the camera sensor itself. A higher ISO value indicates a stronger sensitivity of the photosensitive component. When updating the DCT coefficients corresponding to each pixel block according to the preset threshold, the DCT coefficients that are less than the preset threshold are optionally set to zero.
[0069] In step S103, grayscale transformation is performed on the normal noise reduction image to obtain an overexposed image.
[0070] In the embodiment of the present invention, when grayscale transformation is performed on the normal denoised image, the normal denoised image may be subjected to inverse transformation, logarithmic transformation, or piecewise linear transformation. Preferably, the normal denoised image is subjected to gamma transformation to obtain an overexposed image with enhanced contrast and discernible details. The formula for the gamma transformation is: s = cr γ , where s represents the overexposed image, r represents the grayscale value of the normal denoised image, and its value range is [0,1]. c represents the grayscale scaling coefficient, which is used to stretch the grayscale of the image as a whole and is usually 1. γ represents the gamma coefficient, preferably, γ = 0.625, to improve the image enhancement effect.
[0071] In step S104, the underexposed image, the normal noise reduction image and the overexposed image are fused to obtain a super night scene image.
[0072] In an embodiment of the present invention, the underexposed image can provide rich details for the highlights of the image, the normal denoised image can provide rich details for the mid-brightness of the image, and the overexposed image can provide details for the dark areas of the image. By fusing the underexposed image, the normal denoised image, and the overexposed image, a fused super night scene image is obtained, which is an HDR (High Dynamic Range) image. When fusing the underexposed image, the normal denoised image, and the overexposed image, preferably, using the normal exposure image as a reference, motion pixels are removed from the underexposed image and the overexposed image to obtain a third image and a fourth image, respectively. The normal exposure image, the third image, and the fourth image are downsampled, a first weight map is calculated, and the three downsampled images are converted into grayscale images. Multi-resolution fusion is performed on the grayscale images, and a second weight map is calculated. The second weight map is upsampled to the same size as the original image. The normal exposure image, the underexposed image, and the overexposed image are weightedly fused, thereby improving the visual effect of the high dynamic range image. Of course, other fusion processing methods in the prior art may also be used to perform fusion processing on the underexposed image, the normal noise reduction image, and the overexposed image, which will not be described in detail here.
[0073] In an embodiment of the present invention, multiple consecutive frames of original images are acquired, each of which includes one underexposed image and multiple normally exposed images. Stacking noise reduction processing is performed on the multiple normally exposed images to obtain a normally denoised image. Grayscale conversion is performed on the normally denoised image to obtain an overexposed image. The underexposed image, the normally denoised image, and the overexposed image are fused to obtain a super night scene image. This reduces noise in the night scene image and improves the user experience. When stacking noise reduction on the multiple normally exposed images, weighted fusion noise reduction is performed on the aligned normally exposed images in YUV space, thereby increasing the processing speed of the night scene image and further improving the user experience.
[0074] Example 2:
[0075] Figure 2 The following illustrates the implementation process of the method for generating a super night scene image provided by the second embodiment of the present invention. For ease of explanation, only the portion related to the embodiment of the present invention is shown, which is described in detail as follows:
[0076] In step S201 , a plurality of consecutive frames of original images are acquired, where the plurality of frames of original images include one frame of underexposed image and a plurality of frames of normally exposed images.
[0077] In step S202 , stacking noise reduction processing is performed on the multiple frames of normal exposure images to obtain a frame of normal noise reduction image.
[0078] In step S203, grayscale transformation is performed on the normal noise reduction image to obtain an overexposed image.
[0079] In step S204, the underexposed image, the normal noise reduction image and the overexposed image are fused to obtain a super night scene image.
[0080] In the embodiment of the present invention, the implementation of steps S201 - S204 may refer to the description of steps S101 - S104 in the aforementioned first embodiment, and will not be repeated here.
[0081] In step S205 , a detail enhancement algorithm is used to perform detail enhancement processing on the super night scene image to obtain a detail-enhanced super night scene image.
[0082] In the embodiment of the present invention, a detail enhancement algorithm is used to perform detail enhancement processing on the super night scene image to obtain a detail-enhanced super night scene image, so as to further improve the image details and further improve the image clarity.
[0083] Preferably, the detail enhancement algorithm is a detail enhancement algorithm based on edge-preserving filtering. The super night scene image after detail enhancement is: I′(x,y)=k1I(x,y)+(1-k1)S1(x,y), where I′(x,y) represents the super night scene image after detail enhancement, I(x,y) represents the super night scene image, S1(x,y) represents the image obtained after edge-preserving filtering of the super night scene image, k1 represents the coefficient of image detail enhancement, and k1>1. Specifically, edge-preserving filtering is performed on the super night scene image I(x,y) to obtain the edge-preserving filtered image S1(x,y). The super night scene image after detail enhancement can be obtained by the formula I′(x,y)=k1I(x,y)+(1-k1)S1(x,y).
[0084] In another embodiment, the detail enhancement algorithm is a detail enhancement algorithm based on ordinary filtering, and the super night scene image after detail enhancement is: I′(x, y)=max(min(T(x, y),I d (x,y)),I e (x, y)), where T(x, y) = k2I(x, y) + (1-k2)S2(x, y), I′(x, y) represents the super night scene image after detail enhancement, I(x, y) represents the super night scene image, S2(x, y) represents the image obtained after ordinary filtering of the super night scene image, and I d (x, y) represents the image obtained after the super night scene image is expanded, I e(x, y) represents the image obtained after the super night scene image is eroded, k2 represents the coefficient of image detail enhancement, and k2>1. Specifically, the super night scene image I(x, y) is subjected to ordinary filtering, dilation, and erosion respectively to obtain the image S2(x, y) after ordinary filtering, the image I2(x, y) after dilation, and the image S2(x, y) after dilation. d (x,y) and the image after corrosion processing I e (x,y), then according to the formula: I′(x,y)=max(min(T(x,y),I d (x,y)),I e (x,y)), T(x,y)=k2I(x,y)+(1-k2)S2(x,y), and the super night scene image with enhanced details can be obtained.
[0085] In an embodiment of the present invention, multiple frames of continuous original images are obtained, which include one frame of underexposed image and multiple frames of normally exposed images. Stacking noise reduction processing is performed on the multiple frames of normally exposed images to obtain a frame of normal noise reduction image. Grayscale transformation processing is performed on the normal noise reduction image to obtain a frame of overexposed image. The underexposed image, the normal noise reduction image and the overexposed image are fused to obtain a frame of super night scene image. A detail enhancement algorithm is used to perform detail enhancement processing on the super night scene image, thereby further improving the details of the night scene image, thereby further improving the user experience.
[0086] Example 3:
[0087] Figure 3 The structure of the apparatus for generating a super night scene image provided by the third embodiment of the present invention is shown. For ease of explanation, only the parts related to the embodiment of the present invention are shown, including:
[0088] The image acquisition unit 31 is used to acquire multiple frames of original images, where the multiple frames of original images include one frame of underexposed image and multiple frames of normally exposed images;
[0089] The stacking noise reduction unit 32 is used to perform stacking noise reduction processing on the multiple frames of normal exposure images to obtain a frame of normal noise reduction image;
[0090] A grayscale conversion unit 33 is configured to perform grayscale conversion on the normal noise reduction image to obtain an overexposed image; and
[0091] The image synthesis unit 34 is configured to fuse the underexposed image, the normal noise reduction image, and the overexposed image to obtain a super night scene image.
[0092] Preferably, the stacked noise reduction unit comprises:
[0093] The first denoising subunit is configured to perform weighted fusion denoising processing on the multiple frames of normally exposed images to obtain a frame of stacked denoised image;
[0094] The second denoising subunit is configured to perform denoising processing on the stacked denoised image by adopting a single-frame image denoising method to obtain the normal denoised image.
[0095] Preferably, the first noise reduction subunit includes:
[0096] A reference image selection unit, configured to select one frame of image from the multiple frames of normally exposed images as a reference image;
[0097] an image alignment processing unit, configured to align images other than the reference image in the plurality of frames of normally exposed images with the reference image;
[0098] A third denoising subunit is configured to perform weighted fusion denoising on the aligned multiple frames of normal exposure images according to the first formula to obtain the stacked denoised image;
[0099] Among them, the first formula is: Represents the stacked denoised image, I i (x, y) represents the i-th image in the multiple frames of normal exposure images after alignment processing, M-1 represents the number of frames of the normal exposure image, and w i (x,y) represents the weight of weighted fusion, the weight w i (x,y) is calculated using the following formula:
[0100] d(x,y)=|I i (x, y)-I0(x, y)|, where I0(x, y) represents the reference image and n represents the intensity value of the preset image noise.
[0101] Preferably, the second noise reduction subunit includes:
[0102] A pixel selection unit is used to randomly select a number of reference pixels in a neighborhood window of the target pixel;
[0103] a pixel block acquisition unit, configured to acquire pixel blocks centered on the target pixel point and each of the reference pixels;
[0104] A DCT transform unit is used to perform DCT transform on each pixel block and update the DCT coefficient corresponding to each pixel block according to a preset threshold;
[0105] a DCT inverse transform unit, configured to perform DCT inverse transform on the updated DCT coefficients to reconstruct each pixel block;
[0106] The pixel value calculation unit is used to perform weighted averaging on the pixel values of the pixels corresponding to the position of the target pixel in each of the reconstructed pixel blocks, and use the pixel value after the weighted averaging as the pixel value of the target pixel.
[0107] Preferably, the DCT transform unit includes:
[0108] The DCT transformation subunit is used to set the coefficients of the DCT coefficients that are smaller than the preset threshold to zero.
[0109] Preferably, the third noise reduction subunit includes:
[0110] The fourth denoising subunit is configured to perform weighted fusion denoising on the aligned multiple frames of normally exposed images according to the first formula in the YUV space.
[0111] Preferably, the fourth noise reduction subunit includes:
[0112] a component data acquisition unit, configured to acquire the Y component data of the aligned multiple frames of normally exposed images, and the U component data and the V component data of the reference image; or
[0113] a component data acquisition unit, configured to perform weighted fusion noise reduction processing on the Y component data according to the first formula, and perform edge-preserving filtering processing on the U component data and the V component data;
[0114] The component data synthesis unit is used to synthesize the Y component data after the weighted fusion noise reduction processing, and the U component data and the V component data after the edge-preserving filtering processing to obtain the stacked noise reduction image.
[0115] Preferably, the grayscale conversion unit includes:
[0116] The gamma transform unit is used to perform gamma transform processing on the normal noise reduction image.
[0117] Preferably, the gamma coefficient when performing gamma transformation processing on the normal noise reduction image is 0.625.
[0118] Preferably, the device further comprises:
[0119] a detail enhancement processing unit, configured to perform detail enhancement processing on the super night scene image using a detail enhancement algorithm to obtain a detail-enhanced super night scene image;
[0120] The detail enhancement algorithm is a detail enhancement algorithm based on edge-preserving filtering, and the super night scene image after detail enhancement is: I′(x,y)=k1I(x,y)+(1-k1)S1(x,y), wherein I′(x,y) represents the super night scene image after detail enhancement, I(x,y) represents the super night scene image, S1(x,y) represents the image obtained after edge-preserving filtering of the super night scene image, k1 represents the coefficient of image detail enhancement, and k1>1; or
[0121] The detail enhancement algorithm is a detail enhancement algorithm based on ordinary filtering. The super night scene image after detail enhancement is: I′(x,y)=max(min(T(x,y),I d (x,y)),I e (x, y)), where T(x, y) = k2I(x, y) + (1-k2)S2(x, y), I′(x, y) represents the super night scene image after detail enhancement, I(x, y) represents the super night scene image, S2(x, y) represents the image obtained after ordinary filtering of the super night scene image, and I d (x, y) represents the image obtained after the super night scene image is expanded, I e (x, y) represents the image obtained after the super night scene image is corroded, k2 represents the coefficient of image detail enhancement, and k2>1.
[0122] In the embodiments of the present invention, each unit of the apparatus for generating a super night scene image may be implemented by corresponding hardware or software units. Each unit may be an independent hardware or software unit, or may be integrated into a single hardware or software unit, without limiting the present invention. The specific implementation of each unit of the apparatus for generating a super night scene image can be found in the description of the aforementioned method embodiment and will not be repeated here.
[0123] Example 4:
[0124] Figure 4 The structure of an electronic device provided by a fourth embodiment of the present invention is shown. For ease of description, only the parts related to the embodiment of the present invention are shown.
[0125] The electronic device 4 of the embodiment of the present invention includes a processor 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40. When the processor 40 executes the computer program 42, the steps in the above-mentioned method embodiments are implemented, for example Figure 1 Alternatively, when the processor 40 executes the computer program 42, the functions of each unit in the above-mentioned device embodiments are realized, for example Figure 3 The functions of the units 31 to 34 are shown.
[0126] In an embodiment of the present invention, multiple frames of continuous original images are obtained, which include one frame of underexposed image and multiple frames of normally exposed images. Stacking noise reduction processing is performed on the multiple frames of normally exposed images to obtain a frame of normal noise reduction image. Grayscale transformation processing is performed on the normal noise reduction image to obtain a frame of overexposed image. The underexposed image, the normal noise reduction image and the overexposed image are fused to obtain a frame of super night scene image, thereby reducing the noise of the night scene image and improving the user experience.
[0127] Embodiment 5:
[0128] In an embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps in the above method embodiment are implemented. For example, Figure 1 Alternatively, when the computer program is executed by a processor, the functions of each unit in the above-mentioned device embodiments are realized, for example Figure 3 The functions of the units 31 to 34 are shown.
[0129] In an embodiment of the present invention, multiple frames of continuous original images are obtained, which include one frame of underexposed image and multiple frames of normally exposed images. Stacking noise reduction processing is performed on the multiple frames of normally exposed images to obtain a frame of normal noise reduction image. Grayscale transformation processing is performed on the normal noise reduction image to obtain a frame of overexposed image. The underexposed image, the normal noise reduction image and the overexposed image are fused to obtain a frame of super night scene image, thereby reducing the noise of the night scene image and improving the user experience.
[0130] The computer-readable storage medium of the embodiment of the present invention may include any entity, device, or recording medium capable of carrying computer program code, for example, ROM / RAM, magnetic disk, optical disk, flash memory, or other memory.
[0131] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A stacking noise reduction method, characterized in that: The method comprises: Acquire multiple frames of continuous normal exposure images; The weighted fusion noise reduction process is performed on the multiple frames of normally exposed images, including: selecting a reference image, determining a fusion weight according to a brightness difference between other images and the reference image, and performing a weighted fusion operation based on the fusion weight to obtain a stacked noise reduction image; The stacked denoised image is denoised using a single-frame image denoising method to obtain a frame of normal denoised image.
2. The method according to claim 1, characterized in that The steps of selecting a reference image, determining a fusion weight according to a brightness difference between other images and the reference image, and performing a weighted fusion operation based on the fusion weight include: Selecting one frame of image from the multiple frames of normally exposed images as a reference image; Aligning the images other than the reference image in the multiple frames of normal exposure images with the reference image; The weighted fusion noise reduction processing is performed on the multiple frames of normal exposure images after the alignment processing, including: determining a fusion weight according to the brightness difference between the other images after the alignment processing and the reference image, and performing a weighted fusion operation based on the fusion weight to obtain a frame of stacked noise reduction image.
3. The method according to claim 2, characterized in that The step of determining a fusion weight according to a brightness difference between the other image after the alignment process and the reference image, and performing a weighted fusion operation based on the fusion weight comprises: Performing weighted fusion denoising on the aligned multiple frames of normal exposure images according to the first formula to obtain the stacked denoised image; Among them, the first formula is: represents the stacked denoised image, I i (x, y) represents the i-th image in the multiple frames of normal exposure images after alignment processing, M-1 represents the number of frames of the normal exposure images, and w i (x, y) represents the weight of weighted fusion, the weight w i (x,y) is calculated using the following formula: d(x, y) = |I i (x, y) - I0(x, y)|, where, I0(x, y) represents the reference image, and n represents the intensity value of the preset image noise.
4. The method according to claim 3, characterized in that The step of performing weighted fusion noise reduction processing on the aligned multiple frames of normal exposure images according to the first formula comprises: In the YUV space, weighted fusion noise reduction processing is performed on the aligned multiple frames of normal exposure images according to the first formula.
5. The method according to claim 4, characterized in that The step of performing weighted fusion noise reduction processing on the aligned multiple frames of normal exposure images according to the first formula in the YUV space includes: Acquire the Y component data of the multiple frames of normal exposure images after the alignment processing, and the U component data and V component data of the reference image; Performing weighted fusion noise reduction processing on the Y component data according to the first formula, and performing edge-preserving filtering processing on the U component data and the V component data; The Y component data after the weighted fusion denoising process, and the U component data and the V component data after the edge-preserving filtering process are synthesized to obtain the stacked denoised image.
6. The method according to claim 5, characterized in that The step of performing noise reduction processing on the stacked noise reduction image using the single frame image noise reduction method comprises: Randomly select a number of reference pixels in the neighborhood window of the target pixel, where the target pixel is the corresponding pixel after each sliding; Acquire pixel blocks with the target pixel point and each of the reference pixel points as centers respectively; Performing DCT transformation on each of the pixel blocks, and updating the DCT coefficients corresponding to each of the pixel blocks according to a preset threshold; Performing an inverse DCT transform on the updated DCT coefficients to reconstruct each of the pixel blocks; The pixel values of the pixel points corresponding to the position of the target pixel point in each reconstructed pixel block are weighted averaged, and the pixel value after the weighted average is used as the pixel value of the target pixel point.
7. The method according to claim 6, characterized in that The step of updating the DCT coefficients corresponding to each pixel block according to a preset threshold comprises: The coefficients of the DCT coefficients that are smaller than the preset threshold are set to zero.
8. A stacked noise reduction device, characterized in that: The device comprises: An image acquisition unit, used for acquiring a plurality of continuous frames of normally exposed images; A stacking noise reduction unit is used to perform weighted fusion noise reduction processing on the multiple frames of normal exposure images, including: selecting a reference image, determining a fusion weight according to the brightness difference between other images and the reference image, and performing a weighted fusion operation based on the fusion weight to obtain a frame of stacked noise reduction image; The normal noise reduction unit is used to perform noise reduction processing on the stacked noise reduction image by adopting a single-frame image noise reduction method to obtain a frame of normal noise reduction image.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
