Image noise reduction method and device, equipment and storage medium
Through the combination of time domain processing of multi-frame images and AI network models, the process of stationary and moving areas is separately solved, and the problem of poor image noise reduction effect in traditional technology under extremely low illumination is achieved, and higher image quality and clear details are achieved Spend.
Patent Information
- Application Number
- CN202411007216.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-15
AI Technical Summary
Traditional airspace noise reduction technology is difficult to preserve image details and remove noise at the same time in extremely low illumination scenarios. Especially when video content is moving, single frame image information is not enough to effectively restore the details covered by noise.
Multi-frame images are used for noise estimation and averaging, combining time domain processing and AI network model, signals and noise are separated, filtered for stationary and moving areas, and fused images through weighted overlay to improve image quality and detail clarity .
Significantly reduce noise under low illumination conditions, improve image quality and detail clarity, avoid motion artifacts and details loss, and enhance adaptability to complex noise and blur.
Smart Images

Figure CN118967498A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing, and in particular to an image noise reduction method, device, equipment and storage medium. Background Art
[0002] Image denoising is an important task in digital image processing, which aims to remove noise from an image while preserving the details and edge features of the image as much as possible. Noise can come from a variety of sources, such as sensor noise, compression noise, transmission noise, etc. Different noise reduction techniques can be selected according to the characteristics of the noise and the image processing requirements.
[0003] In extremely low-light scenes, traditional spatial noise reduction technology does have certain limitations. At present, when processing video images, the noise reduction process is performed on each frame separately. However, when the content in the video moves, due to the randomness of the noise, the information content of the single frame image is not enough to effectively restore the details covered by the noise. In other words, a single image cannot provide enough statistical information to effectively distinguish between noise and real signals. Especially in low-light scenes, traditional noise reduction methods are difficult to retain details and remove noise at the same time. Summary of the invention
[0004] The purpose of the present invention is to provide an image denoising method, device, equipment and storage medium that can more accurately separate signals and noise by using multiple frame images for noise estimation and averaging, thereby significantly reducing the noise in images under low illumination conditions, improving image quality and detail clarity, and utilizing the randomness of image time domain noise to achieve effective image quality improvement for static areas of the image through the superposition and fusion of multiple frame images.
[0005] In a first aspect, an image denoising method comprises: acquiring an input image and a first reference image, wherein the input image is an original image acquired by an image sensor; performing time domain processing on the acquired input image and the first reference image to obtain a second reference image, comprising: performing motion detection on the input image and the reference image to obtain a static pixel area and a motion pixel area; performing filtering on the static pixel area and the motion pixel area; fusing the static pixel area and the motion pixel area after filtering to obtain a second reference image; performing denoising on the fused second reference image through an AI network model; and exporting the image after denoising by the AI network model.
[0006] In some embodiments, the motion area and intensity of each pixel are determined according to at least one of frame difference method, background subtraction method, optical flow method, connected region analysis, spectral analysis, and multimodal fusion.
[0007] Furthermore, motion detection is performed on the input image and the reference image to obtain a static pixel area and a motion pixel area, including: calculating a pixel difference between each pixel of the input image and the reference image; setting a threshold and a pixel difference for comparison, when the pixel difference of any pixel is greater than a preset threshold, the area is a motion pixel area, and when the pixel difference of any pixel is less than or equal to the preset threshold, the area is a static pixel area.
[0008] Furthermore, the threshold is obtained by presetting or by constructing a Laplacian pyramid.
[0009] Further, by constructing a Laplacian pyramid, the input image and the reference image are decomposed into Laplacian pyramids respectively to obtain L ni and L nr Among them, L ni Represents the Laplacian image of the input image at the nth level of the Laplacian pyramid, L nr represents the Laplacian image of the reference image at the nth layer of the Laplacian pyramid, where n represents the number of layers; L ni and L nr After subtracting, take the absolute value and get |L ni –L nr |; Right|L ni –L nr | Perform a 2×2 mean filter to obtain multiple thresholds; the resolution of the Nth layer of the Laplacian pyramid image is the original image Each threshold generated is applied to the 2 N ×2 N pixels.
[0010] Further, calculating the pixel difference between each pixel of the input image and the reference image includes: calculating the pixel difference between each pixel of the input image and the reference image by using at least one of a SAD operator, an SSD operator or a census operator for an image block centered on the target pixel.
[0011] Furthermore, filtering the motion pixel region includes: filtering the motion pixel region using spatial domain or frequency domain filtering, including filtering the motion pixel region using any one of frequency domain filtering, median filtering, non-local mean filtering, and bilateral filtering.
[0012] Furthermore, filtering the static pixel region includes: filtering the static pixel region using time domain filtering, or filtering the static pixel region and the moving pixel region using any one of IIR filtering, Kalman filtering, and Wiener filtering.
[0013] Furthermore, the method also includes calculating the noise level of all pixels of the input image.
[0014] Furthermore, the method further includes: performing a dilation algorithm or a mask smoothing process on the static pixel area or the dynamic pixel area in the area connected with the static pixel area to obtain a transition pixel area.
[0015] Furthermore, the excessive pixel region is processed by the filtering processing method performed on the moving pixel region and the filtering processing method performed on the moving pixel region respectively, and through weighted superposition, a noise-reduced excessive pixel region is obtained.
[0016] Furthermore, weighted superposition includes: adjusting the ratio between the pixel area obtained by filtering the motion pixel area with the excessive pixel area and the pixel area obtained by filtering the static pixel area with the static pixel area by means of linear gradient weights and noise level weights.
[0017] Further, fusing the static pixel region and the motion pixel region after filtering to obtain a second reference image includes: fusing the transition pixel region after noise reduction, the static pixel region and the motion pixel region after filtering to obtain a second reference image.
[0018] Furthermore, the input image is a current frame image, the first reference image is a previous frame obtained by the method described in the above embodiment, and the second reference image is a current frame image obtained by the method described in the above embodiment.
[0019] Furthermore, the noise level of all pixels of the input image is calculated using the following formula:
[0020] D N ′=|l in -l smth |
[0021] Among them, l in is the input image, for the input image l in Perform low-pass filtering to obtain a low-pass filtered image l smth , calculate the absolute value D of the difference between the input image and the low-pass filtered image N ′,D N ′ represents the preliminary noise level of each pixel in the input image;
[0022] D N =S N ×D N '
[0023] Among them, D N is the noise level of all pixels in the input image, S N It is the scale factor obtained by calibration of the sensor used to acquire the input image.
[0024] In a third aspect, an acquisition unit is used to acquire an input image and a first reference image, wherein the input image is an original image acquired by an image sensor;
[0025] The time domain processing unit is used to perform time domain processing on the input image and the first reference image to obtain the second reference image, including: performing motion detection on the input image and the reference image to obtain a static pixel area and a moving pixel area; performing filtering processing on the static pixel area and the moving pixel area; and fusing the static pixel area and the moving pixel area after filtering processing to obtain the second reference image;
[0026] An AI processing unit, used for performing noise reduction processing on the fused second reference image through an AI network model;
[0027] The export unit is used to export the image after the noise reduction process is performed by the AI network model.
[0028] In a third aspect, the present application further proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the at least one processor can execute the method described in the above embodiment.
[0029] In a fourth aspect, the present application further proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.
[0030] The beneficial effects of the present invention are as follows: the traditional spatial noise reduction method is usually based on the statistical characteristics of a single frame image, and has limited noise suppression effect under low illumination conditions. In contrast, the present application uses multiple frames of images for noise estimation and averaging, which can more accurately separate signals and noise, thereby significantly reducing the noise in images under low illumination conditions and improving image quality and detail clarity;
[0031] In dynamic scenes, traditional spatial domain methods have difficulty processing moving objects or background changes, which can easily lead to motion artifacts or loss of details. This application uses motion detection and region classification, combined with processing strategies for different regions, to optimize processing for static and moving regions, avoiding motion blur and other common problems in dynamic scenes;
[0032] Appropriately process the transition area and determine the contribution of the final processing result according to the noise level and dynamic changes by weight superposition, which can effectively balance the processing effects of different areas and avoid obvious processing boundaries or visual incoherence;
[0033] The processed image is output to the subsequent AI noise reduction module for further processing, which further optimizes the image quality and the noise reduction effect under low illumination conditions, and enhances the adaptability to complex noise and blur;
[0034] The static area and the moving area are distinguished by two frames of images, and multiple frames of images are used for superposition processing to improve the signal-to-noise ratio by accumulating more image information. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is a structural block diagram of the method of the present invention;
[0036] Figure 2 It is a flowchart of the present invention;
[0037] Figure 3 is a schematic structural diagram of a receiving device provided by an embodiment of the present invention;
[0038] Figure 4 This is a structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.
[0040] Combined with Figure 1 and Figure 2 , the present application proposes an image denoising method, comprising:
[0041] S101, acquiring an input image and a first reference image, wherein the input image is an original image acquired by an image sensor;
[0042] S102, performing time domain processing on the input image and the first reference image to obtain a second reference image includes:
[0043] Performing motion detection on the input image and the reference image to obtain a stationary pixel area and a moving pixel area;
[0044] Performing filtering on the static pixel area and the moving pixel area;
[0045] Fusing the static pixel region and the moving pixel region after filtering to obtain a second reference image;
[0046] S103, performing noise reduction processing on the fused second reference image through an AI network model;
[0047] S104, exporting the image after noise reduction processing by the AI network model.
[0048] Specifically, by differentially processing the static and moving pixel areas and generating a second reference image through fusion, the dynamic and static parts of the image can be effectively processed, thereby improving the visual experience and analysis effect of the observer. The processed image is output to the subsequent AI denoising module for further processing, further optimizing the image quality and the denoising effect under low illumination conditions, and enhancing the adaptability to complex noise and blur. The AI network model can use mainstream or improved models on the market for denoising;
[0049] Since the pixel values in the static area change less in time and are mainly affected by random noise, the noise can be effectively reduced by averaging in the time domain. Since the expected value of noise is zero, the noise will cancel each other out after multiple frames are superimposed.
[0050] Since the pixel values in the moving area vary greatly over time, in addition to noise, they also contain information about the object's motion. Time domain averaging may cause motion blur. Therefore, motion compensation technology is needed to align moving objects in different frames to effectively reduce noise.
[0051] In some embodiments, the motion area and intensity of each pixel are determined according to at least one of frame difference method, background subtraction method, optical flow method, connected region analysis, spectral analysis, and multimodal fusion.
[0052] In some embodiments, motion detection is performed on an input image and a reference image to obtain a static pixel region and a motion pixel region, including: calculating a pixel difference between each pixel of the input image and the reference image; setting a threshold and a pixel difference for comparison; when the pixel difference of any pixel is greater than a preset threshold, the region is a motion pixel region; and when the pixel difference of any pixel is less than or equal to the preset threshold, the region is a static pixel region.
[0053] In some embodiments, the threshold is obtained by being preset or by constructing a Laplacian pyramid.
[0054] In some embodiments, the Laplacian pyramid is constructed to obtain: decomposing the input image and the reference image into Laplacian pyramids respectively to obtain L ni and L nr Among them, Lni Represents the Laplacian image of the input image at the nth level of the Laplacian pyramid, L nr represents the Laplacian image of the reference image at the nth layer of the Laplacian pyramid, where n represents the number of layers; L ni and L nr After subtracting, take the absolute value and get |L ni –L nr |; Right|L ni –L nr | Perform a 2×2 mean filter to obtain multiple thresholds; the resolution of the Nth layer of the Laplacian pyramid image is the original image Each threshold generated is applied to the 2 N ×2 N pixels.
[0055] Specifically, taking N = 2 as an example, the result is |L 2i –L 2r |, and then perform a 2x2 mean filter on the result to get the threshold. Because the Laplacian graph of N=2 is 1 / 4 of the original image (the length and width are 1 / 2 of the original image), each threshold obtained here is applied to 4 pixels of the original image.
[0056] In some embodiments, calculating the pixel difference between each pixel of the input image and the reference image includes: calculating the pixel difference between each pixel of the input image and the reference image by using at least one of a SAD operator, an SSD operator or a census operator for an image block centered on a target pixel.
[0057] By calculating the input image l in and the reference image l ref The difference between the pixels in the moving area and the still area is used to identify which pixels belong to the moving area. Use SAD, SSD or census operators for motion estimation and calculate the transition area.
[0058] Specifically, the difference value is calculated using the SAD, SSD or census operator. The difference value calculated for a single method is as follows:
[0059] SAD (Sum of Absolute Differences): SAD is the sum of the absolute differences between two image blocks; the formula is as follows:
[0060] D ISAD (x,y)=|l in (x,y)-l ref (x,y)|
[0061] l in (x,y) and l ref(x,y) are the pixel values of the input image and the reference image at position (x,y), respectively.
[0062] For each pixel (x,y), calculate the absolute difference between the current image and the reference image.
[0063] SSD (Sum of Squared Differences):
[0064] SSD is the sum of the squares of the differences between two image patches. The formula is as follows:
[0065] D ISSD (x,y)=(l in (x,y)-l ref (x,y) 2
[0066] For each pixel (x, y) calculate the sum of the squares of the difference between the current image and the reference image.
[0067] Census operator: Census transform converts the intensity value of each pixel into a binary string, which represents the comparison result between the pixel and its neighboring pixels. The difference value is calculated by Hamming distance. The formula is as follows:
[0068] D Icensus (x,y)=H(Census(l in (x,y)),Census(l ref (x,y)
[0069] Where H represents the Hanming distance and Census represents the Census transformation operation.
[0070] The difference values calculated for two of the methods are as follows:
[0071] The formula for the combination of SAD and SSD is as follows:
[0072] D ISAD+SSD (x,y)=ω1·SAD(x,y)+ω2·SSD(x,y)
[0073] The formula for combining SAD and Census is as follows:
[0074] D ISAD+Census (x,y)=ω1·SAD(x,y)+ω2·Census(x,y)
[0075] The formula for combining SSD and Census is as follows:
[0076] D ISSD+Census(x,y)=ω1·SSD(x,y)+ω2·Census(x,y)
[0077] In specific implementation, the weight coefficients ω1 and ω2 are parameters used to adjust the influence of the two methods in the combined difference value. These two parameters are determined according to requirements through experiments or prior knowledge to achieve the best effect.
[0078] When judging the moving area and the stationary area, a threshold T is set to distinguish the moving pixels from the stationary pixels.
[0079] For each pixel (x, y), if its difference value D I If it is greater than the threshold T, the pixel is considered to belong to the motion area; otherwise, it belongs to the static area.
[0080] The judgment formula is as follows:
[0081] If D I (x,y)>T, then the pixel (x,y) belongs to the motion area.
[0082] If D I (x,y)≤T, then the pixel (x,y) belongs to the static area.
[0083] In some embodiments, spatial domain filtering of the motion pixel area includes: using spatial domain or frequency domain filtering on the motion pixel area, including filtering the motion pixel area in the frequency domain, median filtering, non-local mean filtering, or bilateral filtering. In specific implementation, the filtering method used is not limited to this.
[0084] Specifically, the principle of frequency domain filtering: Frequency domain filtering processes the image by converting it to the frequency domain (Fourier transform) and then converting it back to the spatial domain (inverse Fourier transform). In the frequency domain, it is easier to separate components of different frequencies, thereby performing denoising or enhancement. Steps: Perform Fourier transform on the input image to convert the image from the spatial domain to the frequency domain; apply the filter in the frequency domain; perform inverse Fourier transform on the filtered frequency domain image to convert it back to the spatial domain.
[0085] Median filtering principle: Median filtering effectively removes salt and pepper noise by replacing the central pixel value with the median value of the pixels in the neighborhood while maintaining the integrity of the edge. Steps: For each moving pixel in the image, select a neighborhood window containing the pixel; calculate the median value of all pixels in the window; replace the value of the central pixel with the calculated median value.
[0086] Non-local mean filtering (NLM) principle: Non-local mean filtering replaces the current pixel value with the weighted average of similar blocks in the entire image. This method takes advantage of the self-similarity of the image and can effectively remove noise while retaining details. Steps: For each moving pixel in the image, define a search window; within the search window, find all blocks similar to the current pixel block; calculate the weighted average of these similar blocks, with the weight determined by the similarity; replace the current pixel value with the weighted average.
[0087] Bilateral filtering principle: Bilateral filtering is an edge-preserving filtering method that considers both spatial distance and pixel value differences. This method can well preserve the edges of the image while removing noise. Steps: For each moving pixel in the image, define a neighborhood window; calculate the spatial distance weight and color similarity weight of each pixel in the window with the center pixel; calculate the weighted average, where the weight is determined by the spatial distance and color similarity; replace the value of the current pixel with the weighted average.
[0088] In some embodiments, time domain filtering of the static pixel region includes filtering the static pixel region using any one of IIR filtering, Kalman filtering, and Wiener filtering. In specific implementations, the filtering method used is not limited thereto.
[0089] Specifically, the principle of IIR filtering (infinite impulse response filtering): IIR filter is a recursive filter whose output depends not only on the current input, but also on the past input and output. Compared with FIR (finite impulse response) filter, IIR filter requires a lower filter order to achieve the same effect. IIR filter is suitable for occasions that require smooth output, such as audio signal processing and image denoising.
[0090] Kalman filtering is a recursive filtering algorithm used to estimate the state of a dynamic system, especially in the presence of noise. The pixels in the stationary area can be regarded as the observations of a dynamic system, and Kalman filtering is used for smoothing.
[0091] Status update formula:
[0092] x k =x k-1
[0093] P k =P k-1 +Q
[0094] Observation update formula:
[0095]
[0096] x k =x k+K k (z k -x k )
[0097] P k =(1-K k ) k
[0098] in:
[0099] x k is the current state estimate.
[0100] P k is the current estimated covariance.
[0101] Q is the process noise covariance.
[0102] R is the observation noise covariance (here the noise level D n express).
[0103] K k is the Kalman gain.
[0104] Z k is the current observation value (current pixel value).
[0105] Wiener filtering is a linear minimum mean square error filter used to restore signals in the presence of noise. It optimizes the filtering effect by minimizing the mean square error between the output signal and the expected signal.
[0106] In some embodiments, the method further includes calculating the noise level D of all pixels of the input image. n .
[0107] In some embodiments, the method further includes: performing a dilation algorithm or a mask smoothing process on the static pixel region or the dynamic pixel region in the area where the static pixel region and the dynamic pixel region are connected to obtain a transition pixel region.
[0108] Specifically, the dilation algorithm is a morphological operation used to expand a specific area in an image. Through dilation, a transition area can be generated between a stationary pixel area and a moving pixel area.
[0109] Steps: Define the structural element: Select a structural element (such as a 3x3 or 5x5 rectangle) for the dilation operation. Dilate the static pixel area: Align the center of the structural element to each pixel in the static pixel area. If there are any static pixels in the area covered by the structural element, set the center pixel to the static pixel. Dilate the moving pixel area: Similarly, you can also choose to dilate the moving pixel area.
[0110] Mask smoothing generates transition regions by smoothing the masks of stationary and moving regions.
[0111] Steps: Generate Mask: Create binary masks of static and moving areas; Apply Gaussian filter: Smooth the mask; Define transition region: The values of the smoothed mask between 0 and 1 represent the transition region.
[0112] Through the above steps, a transition area is formed on the boundary of the static pixel area or the moving pixel area.
[0113] In some embodiments, the excessive pixel region is processed by the filtering processing method performed on the moving pixel region and the filtering processing method performed on the moving pixel region respectively, and through weighted superposition, a noise-reduced excessive pixel region is obtained.
[0114] Specifically, the determined excessive pixel area is filtered by the noise reduction method used for the moving pixel area, that is, any one of IIR filtering, Kalman filtering, and Wiener filtering, and the noise reduction method used for the static pixel area is filtered, including: filtering the moving pixel area by any one of IIR filtering, Kalman filtering, and Wiener filtering, and the results of the two filtering are weighted added.
[0115] In some embodiments, weighted superposition includes: adjusting the ratio between the pixel area obtained by filtering the motion pixel area with the excessive pixel area and the pixel area obtained by filtering the static pixel area with the static pixel area by means of linear gradient weights and noise level weights.
[0116] Specifically, the proportion is calculated by the linear gradient weight calculation method, and the formula is as follows:
[0117]
[0118] ω static (x,y)=1-ω motion (x,y)
[0119] Among them, ω motion is the weight ratio of the moving pixel area, ω static is the weight ratio of the static pixel area,
[0120] d(x,y) is the distance from the current pixel to the boundary of the static area, and D is the width of the transition area.
[0121] The proportion is calculated by the noise level weight calculation method, the formula is as follows:
[0122]
[0123] ω static(x,y)=1-ω motion (x,y)
[0124] The weights are based on the noise level D n Determined by the noise level D nmax is the maximum value of the noise level.
[0125] The calculation formula for pixels in the transition area is as follows:
[0126] I fused (x,y)=ω motion (x,y)·I motion (x,y)+ω static (x,y)·I static (x,y)
[0127] I fused (x, y) is the result after denoising the transition area. motion (x, y) is the result after moving area denoising, I static (x,y) is the result after the static area moves.
[0128] In some embodiments, fusing the static pixel region and the moving pixel region after filtering processing to obtain a second reference image includes: fusing the transition pixel region after noise reduction, the static pixel region and the moving pixel region after filtering processing to obtain a second reference image.
[0129] Specifically, the formula after fusion is:
[0130] I blend (x,y)=I fused (x,y)+I motion (x,y)+I static (x,y)
[0131] Among them I blend (x, y) is the structure after fusion, that is, the second reference image.
[0132] In some embodiments, the input image is a current frame image, the first reference image is a previous frame obtained by the method described in the above embodiment, and the second reference image is a current frame image obtained by the method described in the above embodiment.
[0133] In some embodiments, calculating the noise level of all pixels of the input image includes the following formula:
[0134] D N ′=|l in -l smth |
[0135] Among them, lin is the input image, for the input image l in Perform low-pass filtering to obtain a low-pass filtered image l smth , calculate the absolute value D of the difference between the input image and the low-pass filtered image N ′,D N ′ represents the preliminary noise level of each pixel in the input image;
[0136] D N =S N ×D N '
[0137] Among them, D N is the noise level of all pixels in the input image, S N It is the scale factor obtained by calibration of the sensor used to acquire the input image.
[0138] Based on the image data of the input frame, the noise level of each pixel is effectively estimated to determine the weight of the transition area when performing Kalman filtering on the pixels in the static area and fusing them.
[0139] Combination Figure 3 , the present application also proposes an image noise reduction device, comprising:
[0140] An acquisition unit 201 is used to acquire an input image and a first reference image, wherein the input image is an original image acquired by an image sensor;
[0141] The time domain processing unit 202 is used to perform time domain processing on the input image and the first reference image to obtain the second reference image, including: performing motion detection on the input image and the reference image to obtain a static pixel area and a moving pixel area; performing filtering processing on the static pixel area and the moving pixel area; and fusing the static pixel area and the moving pixel area after filtering processing to obtain the second reference image.
[0142] An AI processing unit 203 is used to perform noise reduction processing on the fused second reference image through an AI network model;
[0143] The export unit 204 is used to export the image after the noise reduction process is performed by the AI network model.
[0144] All relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here.
[0145] In some other embodiments of the present invention, an electronic device 300 is disclosed. Figure 4As shown, the electronic device 300 may include: one or more processors 301; a memory 302; a display 303; one or more applications (not shown); and one or more computer programs 304. The above components may be connected via one or more communication buses 305. The one or more computer programs 304 are stored in the memory 302 and configured to be executed by the one or more processors 301. The one or more computer programs 304 include instructions, which may be used to execute the following: Figure 1-Figure 4 And each step in the corresponding embodiment.
[0146] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0147] Each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0148] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: flash memory, mobile hard disk, read-only memory, random access memory, disk or optical disk and other media that can store program codes.
[0149] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.
Claims
1. An image denoising method, characterized in that: include: Acquire an input image and a first reference image, wherein the input image is an original image acquired by an image sensor; The input image and the first reference image are subjected to time domain processing to obtain a second reference image, comprising: Performing motion detection on the input image and the reference image to obtain a stationary pixel area and a moving pixel area; Performing filtering on the static pixel area and the moving pixel area; Fusing the static pixel region and the moving pixel region after filtering to obtain a second reference image; The fused second reference image is subjected to noise reduction processing through the AI network model; Export the image after noise reduction through the AI network model.
2. The image denoising method according to claim 1, characterized in that: Performing motion detection on the input image and the reference image to obtain the static pixel area and the motion pixel area includes: Calculate the pixel difference between each pixel of the input image and the reference image; The threshold and pixel difference are set for comparison. When the pixel difference of any pixel is greater than the preset threshold, the area is a moving pixel area. When the pixel difference of any pixel is less than or equal to the preset threshold, the area is a static pixel area.
3. The image denoising method according to claim 2, characterized in that: The threshold is obtained by being preset or by constructing a Laplacian pyramid.
4. The image denoising method according to claim 3, characterized in that: By constructing a Laplace pyramid, we get: Decompose the input image and the reference image into Laplacian pyramids respectively, and get L ni and L nr ; Among them, L ni Represents the Laplacian image of the input image at the nth level of the Laplacian pyramid, L nr Represents the Laplacian image of the reference image at the nth layer of the Laplacian pyramid, where n represents the number of layers; Put L ni and L nr After subtraction, take the absolute value and get |L nu –L nr |; Yes | ni –L nr |Perform a 2×2 mean filter to obtain multiple thresholds; The resolution of the Nth layer of the Laplacian pyramid image is the original image Each threshold generated is applied to the 2 N ×2 N pixels.
5. The image denoising method according to claim 2, characterized in that: Calculating the pixel difference between each pixel of the input image and the reference image includes: The image block centered on the target pixel calculates the pixel difference between each pixel of the input image and the reference image by at least one of a SAD operator, an SSD operator or a census operator.
6. The image denoising method according to claim 1, characterized in that: The filtering process for the moving pixel area includes: Using spatial domain or frequency domain filtering on the motion pixel area includes filtering the motion pixel area in the frequency domain, median filtering, non-local mean filtering, or bilateral filtering.
7. The image denoising method according to claim 1, characterized in that: The filtering process for the stationary pixel area includes: The time domain filtering of the static pixel region includes filtering the static pixel region by any one of IIR filtering, Kalman filtering, and Wiener filtering.
8. The image denoising method according to claim 1, characterized in that: It also includes calculating the noise level of all pixels in the input image.
9. The image denoising method according to claim 8, characterized in that: Also includes: In the area where the static pixel area and the moving pixel area are connected, a dilation algorithm or a mask smoothing process is performed on the static pixel area or the moving pixel area to obtain a transition pixel area.
10. The image denoising method according to claim 9, characterized in that: The excessive pixel region is processed by the filtering processing method performed on the moving pixel region and the filtering processing method performed on the moving pixel region respectively, and a noise-reduced excessive pixel region is obtained through weighted superposition.
11. The image denoising method according to claim 10, characterized in that: Included via weighted overlay: The ratio between the pixel area obtained by filtering the moving pixel area with the excessive pixel area and the pixel area obtained by filtering the still pixel area with the still pixel area is adjusted by means of linear gradient weight and noise level weight.
12. The image denoising method according to claim 11, characterized in that: Fusing the static pixel region and the moving pixel region after filtering to obtain a second reference image includes: The excess pixel region after noise reduction, the static pixel region after filtering, and the motion pixel region are fused to obtain a second reference image.
13. The image denoising method according to claim 1, characterized in that: The input image is a current frame image, the first reference image is a previous frame image obtained by the method as described in any one of claims 1-10, and the second reference image is a current frame image obtained by the method as described in any one of claims 1-10.
14. The image denoising method according to claim 8, characterized in that: Calculating the noise level of all pixels in the input image includes the following formula: D N ′=|l in -l smth | Among them, l in is the input image, for the input image l in Perform low-pass filtering to obtain a low-pass filtered image l smth , D N ′ represents the preliminary noise level of each pixel in the input image; D N =S N ×D N ’ Among them, D N is the noise level of all pixels in the input image, S N It is the scale factor obtained by calibration of the sensor used to acquire the input image.
15. An image noise reduction device, characterized in that: include: An acquisition unit, configured to acquire an input image and a first reference image, wherein the input image is an original image acquired by an image sensor; The time domain processing unit is used to perform time domain processing on the input image and the first reference image to obtain the second reference image, including: performing motion detection on the input image and the reference image to obtain a static pixel area and a moving pixel area; performing filtering processing on the static pixel area and the moving pixel area; and fusing the static pixel area and the moving pixel area after filtering processing to obtain the second reference image; An AI processing unit, used for performing noise reduction processing on the fused second reference image through an AI network model; The export unit is used to export the image after the noise reduction process is performed by the AI network model.
16. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and when the computer program is executed by the processor, the at least one processor is able to execute the method according to any one of claims 1 to 14.
17. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 14 are implemented.
Citation Information
Cited By
Medical data processing system and method
CN119379838A
Video image denoising method based on gradient distribution
CN122347522A
