Multi-frame image time domain noise reduction method, electronic equipment and storage medium

Abstract

The invention relates to a multi-frame image time domain noise reduction method, electronic equipment and a storage medium, and the method sequentially comprises the following steps: 1, motion estimation: employing a block matching or simplified optical flow method to calculate a motion vector between adjacent frames, and calculating a fusion weighting coefficient according to a registration result; step 2, motion compensation: compensating the reference frame according to the motion vector so as to align motion areas in the multiple frames of images; and step 3, adaptive weighted fusion: in combination with a motion detection registration result, carrying out weighted fusion on the compensated reference frame and the current frame in a registration region, and using pixel information of the current frame in an active region. Under the condition that N-frame motion compensation and time domain average noise reduction are most ideal, the noise level is 1 / N of that of an original image, and the noise reduction effect is remarkable.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital image processing, in particular to a multi-frame image time domain noise reduction method, electronic equipment and storage medium, and specifically relates to a multi-frame image time domain noise reduction method based on motion detection and compensation, electronic equipment and storage medium. BACKGROUND

[0002] In digital image processing, noise suppression is an important technology, especially in video surveillance, medical imaging, low-light photography and other scenarios, high-quality images are crucial for subsequent analysis and judgment.

[0003] The traditional multi-frame average noise reduction method cannot meet the demand of capturing clear details of moving objects when facing moving objects, because the position of the object changes in different frames, and the average processing will cause the moving object to be blurred or produce residual images.

[0004] After searching, it is found that Chinese patent No. CN1901620A discloses a video image noise reduction method based on motion detection and adaptive filtering, which estimates the noise variance of the input video image and calculates the threshold, uses macroblock-based motion detection to segment moving and stationary regions, uses temporal Kalman filter for stationary regions, and uses spatial adaptive filter for moving regions. This scheme uses different filtering methods after simply segmenting the region by motion detection, but it does not fully consider the alignment of the position of moving objects between multiple frames, resulting in poor noise reduction effect and easy loss of motion details.

[0005] After searching, it is found that Chinese patent No. CN116258648A discloses an image noise reduction method, device and electronic equipment, which inputs adjacent image groups (including a to-be-reduced image and its previous frame image) into a motion detection architecture composed of a multi-layer motion detection network, performs multi-scale motion detection to obtain a motion mask, and fuses and reduces noise according to the mask. Although this scheme has certain improvement in motion detection accuracy, the training of the neural network model requires a large amount of data and computing resources, has high hardware requirements, and has deficiencies in real-time performance, which is difficult to meet the scene requirements of real-time processing such as video surveillance.

[0006] In view of the above-mentioned defects, the present design person actively researches and innovates to create a multi-frame image time domain noise reduction method, electronic equipment and storage medium, so as to make it more useful in industry. SUMMARY

[0007] To solve the above technical problems, the purpose of the present application is to provide a multi-frame image time domain noise reduction method, electronic equipment and storage medium, To achieve the above purpose, the present application adopts the following technical solutions: One of the purposes of the present application is: A multi-frame image time domain noise reduction method, sequentially comprising the following steps: Step 1, motion estimation: using block matching or simplified optical flow method, calculating the motion vector between adjacent frames, and calculating the fusion weighting coefficient according to the registration result; Step 2, motion compensation: compensating the reference frame according to the motion vector to align the motion area in the multi-frame image; Step 3, adaptive weighted fusion: combining the motion detection registration result, using the compensated reference frame and the current frame for weighted fusion in the registration area, and using the pixel information of the current frame in the active area.

[0008] As a further improvement of the application, in step 1, the current frame is divided into multiple macroblocks, and the matching block with the smallest residual error is searched near the original coordinates of the corresponding macroblock of the reference frame to determine the motion vector.

[0009] As a further improvement of the application, the size of the macroblock can be adjusted according to the image resolution, and the macroblock size of 8x8 pixels is used in the case of 1K image, and the macroblock size of 16x16 pixels is used in the case of 2K image.

[0010] As a further improvement of the application, the way to search for the minimum residual error is the minimum residual error or the minimum residual square or the maximum normalized cross-correlation.

[0011] As a further improvement of the application, the motion estimation in step 1 uses full search algorithm or fast algorithm, and the fast algorithm is gradient descent method.

[0012] As a further improvement of the application, in step 3, when processing multi-frame images, first, the motion compensation and time domain average noise reduction between the current frame and each reference frame are performed to obtain multiple noise reduction images, then the weight of each noise reduction image is set and weighted summation is performed to obtain the final noise reduction image.

[0013] As a further improvement of the application, the sum of the weights of the multiple noise reduction images is 1.0.

[0014] As a further improvement of the application in step 3, when processing multi-frame images, the fusion result of each reference frame and the current frame is directly calculated.

[0015] The second purpose of the application is: An electronic device, comprising a memory and a processor, the memory storing a computer program, and the computer program being executed by the processor to make the processor execute the steps of the multi-frame image time domain noise reduction method as described above.

[0016] The third purpose of the application is: A computer readable storage medium, having stored thereon a computer program, the computer program being executed by a processor to implement the steps of the multi-frame image time domain noise reduction method as described above.

[0017] By the above scheme, the present application has at least the following advantages: 1. Noise reduction performance: In the most ideal case of N-frame motion compensation and time domain average noise reduction, the noise level is 1 / N of the original image, and the noise reduction effect is significant.

[0018] 2. Motion preservation: Because motion detection and compensation and adaptive weighted fusion are used, motion smearing effect of moving objects caused by time domain averaging is avoided, and motion details are well preserved.

[0019] 3. Real-time: The algorithm complexity is reduced to 30% of the traditional optical flow method, and through the acceleration of the processor (GPU), 1K@60fps and 2K@30fps real-time processing of DSA images is supported.

[0020] 4. Motion compensation and noise reduction fusion: The motion compensation technology in MPEG compression is transferred to the noise reduction scene, solving the motion blur problem of traditional multi-frame averaging method.

[0021] 5. Adaptive weight mechanism: The fusion weight is dynamically adjusted according to the motion vector and the registration level, such as reducing the weight of historical frames in low registration area to improve the noise reduction effect and motion detail preservation ability.

[0022] The above description is only a summary of the technical scheme of the present application, in order to more clearly understand the technical means of the present application, and can be implemented according to the content of the specification, the following is the preferred embodiment of the present application and the detailed description of the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical scheme of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments, it should be understood that the following drawings only show some embodiments of the present application, therefore should not be regarded as a limitation on the scope, for those skilled in the art, without creative labor, other related drawings can also be obtained from these drawings.

[0024] Figure 1 is a flowchart of a multi-frame image time domain noise reduction method based on motion detection and compensation of the present application; Figure 2 is a schematic diagram of the present application when there is no noise reduction in the first embodiment; Figure 3 is a schematic diagram of the present application when multi-frame average time domain noise reduction is performed in the first embodiment; Figure 4This is a schematic diagram of multi-frame motion compensation temporal noise reduction in the first embodiment of the present invention. Detailed Implementation

[0025] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0027] First embodiment of the present invention: like Figure 1 As shown in this embodiment, a multi-frame image temporal denoising method based on motion detection and compensation is proposed. By estimating and compensating the motion of multiple frames, the pixel information of the current frame and the reference frame is adaptively fused according to the registration result, so as to preserve motion details while suppressing noise.

[0028] I. Core Steps: 1. Motion estimation: Based on block matching or simplified optical flow, calculate the motion vectors between adjacent frames. Specifically, the current frame is divided into multiple macroblocks (such as 8×8 pixels or 16×16 pixels). For each macroblock, search for matching blocks with the minimum residual (such as SAD, SSD, NCC, etc.) near the original coordinates of the macroblock in the reference frame to form a motion vector table.

[0029] 2. Motion compensation: The reference frame is compensated based on the motion vector to align the motion regions in multiple frames, so that the position of the moving object remains consistent in different frames.

[0030] 3. Adaptive Weighted Fusion: Combining motion detection registration results, a compensated weighted fusion is applied to the registered region, while the current frame result is used for the active region. For multi-frame (N-frame) processing, each reference frame undergoes motion compensation and temporal average noise reduction with the current frame, resulting in N-1 denoised images. Weights are assigned to these N-1 denoised images (the sum of the weights is 1.0), and the multi-frame motion compensation and temporal average noise reduction result is finally obtained by weighted summation; alternatively, each reference frame can be directly calculated and fused.

[0031] As Figures 2-4 In the motion compensation temporal noise reduction, the contrast (X-Ray dynamic image with walking clock) before and after the noise reduction shows the effect of noise reduction and motion blur elimination. Among them, Figure 2 It is a schematic diagram when there is no noise reduction at present; Figure 3 It is a schematic diagram when multi-frame average temporal noise reduction is performed; Figure 4 It is a schematic diagram when multi-frame motion compensation temporal noise reduction is performed.

[0032] Second, the core steps (motion estimation-motion compensation-adaptive weighted fusion) of the multi-frame image temporal noise reduction method based on motion detection and compensation of the embodiment are not isolated, but form an indivisible whole through time sequence association, functional complementation and logical closed loop. Missing any step or changing the order of steps cannot achieve the core goal of "noise reduction while avoiding motion blur", which is analyzed as follows: 1. Mutual combination and improvement logic between steps: Motion estimation is the premise: to provide "motion reference" for subsequent steps Motion estimation accurately locates the position association of the motion area in multi-frame images by calculating the motion vector (such as the displacement direction and distance of the object) of adjacent frames. The core role of this step is to quantify motion information, provide "alignment coordinates" for motion compensation, and provide "weight judgment basis" for adaptive fusion (such as the residual size of the motion vector reflecting the registration accuracy).

[0033] If motion estimation is missing: motion compensation will lose direction and distance reference and cannot align the motion area in multiple frames; adaptive fusion cannot distinguish between "motion area" and "still area" and can only degenerate into traditional multi-frame averaging, which will inevitably produce motion blur.

[0034] Motion compensation is a bridge: to solve the core contradiction of "multi-frame alignment" Motion compensation adjusts the coordinates of the reference frame according to the motion vector obtained by motion estimation, so that the same moving object in different frames is aligned in spatial position. The role of this step is to eliminate the frame-to-frame misalignment caused by motion, so that the "effective signal" (such as the stable features of the moving object) of multi-frame images can be fused, while isolating "noise interference".

[0035] If motion compensation is missing: even if the object motion trajectory is known through motion estimation, the motion area in multi-frame images is still in a misaligned state, and fusion at this time will forcibly superimpose pixels at different positions, resulting in blurred edges of the moving object (the essence of blur is the average result of misaligned pixels), which has no essential difference from traditional methods.

[0036] Adaptive weighted fusion is the result: to achieve the balance of "noise reduction and motion preservation" Adaptive weighted fusion dynamically adjusts the fusion weight of the current frame and the reference frame based on the "registration accuracy" (such as residual size) of motion estimation and the "alignment result" of motion compensation: for well-aligned static areas, noise reduction is achieved through multi-frame weighted fusion; for unaligned motion areas, the current frame information is retained to avoid smearing. This step is the key to converting the motion information from the previous steps into a noise reduction strategy.

[0037] If adaptive weighted fusion is missing: even if motion estimation and compensation are completed, if fixed weight fusion (such as equal weighting) is used, for areas that are not completely aligned by motion compensation (such as the edges of fast-moving objects), blurring will still occur due to the superposition of multiple frames; and using only single-frame information will lose the advantage of multi-frame noise reduction, and will not be able to reduce the noise level.

[0038] 2. The order of the steps cannot be changed: reversing the order will cause logical breaks The steps of this embodiment must strictly follow the order of motion estimation - motion compensation - adaptive weighted fusion, and reversing the order will directly cause functional failure: If "motion compensation - motion estimation" is performed first: Motion compensation requires motion vectors as a basis, and after reversing, motion estimation is performed later, making compensation "blind" (not knowing which direction to adjust the reference frame), and ultimately only random alignment can be achieved, leaving the motion areas of multiple images in a chaotic state, and subsequent fusion will inevitably produce smearing.

[0039] If "adaptive weighted fusion - motion estimation / compensation" is performed first: Without motion estimation and compensation before fusion, the motion areas of multiple images are not aligned, and at this time the essence of fusion is "the superposition of misaligned pixels", which directly produces motion smearing; subsequent motion estimation and compensation are meaningless (the fused image has lost the original motion details and cannot restore the true motion trajectory).

[0040] If the intermediate step (such as "motion estimation - adaptive weighted fusion") is skipped: Without motion compensation, the vector information from motion estimation cannot be converted into actual frame alignment operations, and during fusion, the system still relies on misaligned pixels from the original frame, and the final result is consistent with the traditional multi-frame averaging method, which cannot solve the smearing problem.

[0041] 3. The overall steps cannot be divided: the target fails to lack any necessary step The core goal of this embodiment is to "significantly reduce noise while avoiding motion smearing", and this goal relies on the "motion quantization - alignment correction - intelligent fusion" closed loop formed by the three steps: If motion estimation is missing: the system cannot identify "which areas are in motion", and motion compensation and adaptive fusion both lose the judgment standard, degenerating into traditional multi-frame averaging (with smearing and limited noise reduction).

[0042] If motion compensation is missing: the motion area of multi-frame images is always misaligned, and the pixels of the moving object are forced to be superimposed during fusion, which will inevitably produce trailing (contrary to the goal of "avoiding trailing").

[0043] If adaptive weighted fusion is missing: even if motion estimation and compensation are completed, fixed weight fusion will cause blurring for areas that are not completely aligned after compensation (such as fast-moving edges), and cannot balance "noise reduction" and "motion preservation" (contrary to the dual goals of "significant noise reduction + no trailing").

[0044] In summary, the three core steps of the present embodiment are the whole of "causal correlation, time sequence enforcement, and functional closed loop": motion estimation provides "motion information", motion compensation achieves "spatial alignment", and adaptive weighted fusion completes "intelligent noise reduction", which together solve the contradiction between "noise reduction and trailing" in traditional methods. Missing any step or changing the order will cause the overall logic to break down, and the technical effect of "noise reduction while preserving motion details" cannot be achieved. Therefore, all steps of the scheme must be considered as an indivisible whole in order to achieve the core invention purpose of the present embodiment.

[0045] Second embodiment of the present application: A multi-frame image time domain noise reduction method based on motion detection and compensation in the present embodiment, motion compensation between two frames and time domain average noise reduction: Definition of frame: The two frames involved in the calculation are defined as the current frame and the reference frame.

[0046] Current frame: image to be denoised.

[0047] Reference frame: frame adjacent in time to the current frame, using the information in the frame to participate in the noise reduction processing of the current frame.

[0048] Step 1: Motion estimation Block division: divide the current frame into multiple macroblocks (such as 8x8 pixels or 16x16 pixels).

[0049] Motion vector calculation: for each macroblock, search for a matching block with the smallest residual error (SAD, Sum of Absolute Differences) in the vicinity of the original coordinates of the macroblock in the reference frame, forming a motion vector table.

[0050] Step 2: Matching determination Matching degree determination: for each macroblock of the image, calculate the average weighting coefficient according to the value of the minimum residual error to form a weighting coefficient table The larger the residual error value (indicating the poorer the matching degree), the smaller the weighting coefficient.

[0051] ‌Step 3: Adaptive temporal fusion ‌Motion compensation: Align the macroblocks of the reference frame to the current frame according to the motion vector table to obtain the compensated reference frame.

[0052] Weighted fusion: Calculate the image after the fusion of the current frame and the reference frame according to the coefficients in the weighted coefficient table.

[0053] Formula example: , wherein ， is the image pixel coordinate, wherein: is the average weighted coefficient.

[0054] Calculate the motion compensation and temporal average denoising between multiple frames The multiple frames (N frames) involved in the calculation are defined as the current frame (1 frame) and multiple reference frames (N-1 frames). Each reference frame is repeated "calculate the motion compensation and temporal average denoising between two frames" with the current frame to obtain N-1 motion compensation and temporal average denoising images , set the weight for the N-1 denoising images ( ) , the calculation of the motion compensation and temporal average denoising of multiple frames is: Third embodiment of the present application: In this embodiment, when processing multiple reference frames, in order to achieve simple computer programming, the problem is divided into multiple "motion compensation and temporal average denoising between two frames" and then the final ordinary multiple frame average processing is performed. It can also be directly calculated for each reference frame and fused, and the formula is as follows: Here, is the average weighted coefficient of different compensation frames, and in The actual numerical value is not equal.

[0055] In this embodiment, when performing macroblock division, the size of the macroblock can be defined and is not limited to, for example, 8x8 pixels or 16x16 pixels. In the actual verification of this method, 8x8 pixel macroblock size was used in the case of 1K image, and 16x16 pixel macroblock size was used in the case of 2K image. In the motion vector calculation search of this embodiment, in order to ensure accuracy, an exhaustive search algorithm is used. Other fast algorithms can also be used, such as "gradient descent method" and "dynamic resolution adjustment".

[0056] The embodiment searches for minimum residual in motion vector calculation using minimum residual (SAD, Sum of Absolute Differences), but is not limited to the minimum residual method, and can also use minimum residual square (SSD, Sum of Squared Differences), or search for maximum normalized cross-correlation (NCC, Normalized Cross-Correlation), etc.

[0057] In the motion vector calculation, the search range of the macro block in the reference frame directly determines the size of the calculation amount, and a suitable search range can be determined according to an actual scene (such as the motion condition of a target object, a frame rate, etc.). A search range of 7x7 is used in actual verification of the method, and the real-time requirement of practical application is met.

[0058] A fourth embodiment of the application is as follows: 1. Motion estimation implementation Scene judgment and method selection: Before motion estimation, the image scene is preliminarily analyzed. The motion speed and texture complexity of the object are judged by calculating the gradient distribution of the image, the complexity of the object edge, etc. If the gradient change of most regions in the image is small, the object edge is relatively smooth, and the object motion speed is slow according to prior knowledge or scene characteristics, the block matching method is selected. If the image gradient changes sharply, the texture is complex, and the object can exist fast motion, the simplified optical flow method is selected.

[0059] Specific operation of the block matching method: when the block matching method is used, the current frame is divided into multiple macro blocks. For each macro block, search is performed near the original coordinates of the macro block in the reference frame. The search range is dynamically determined according to the motion speed of the object and the acquisition frame rate. For example, in the case of a high acquisition frame rate and a slow object motion speed, the search range can be set to a small area (such as a range of ±3 pixels with the center of the macro block as the origin). In the case of a low acquisition frame rate or a fast object motion speed, the search range is appropriately expanded (such as a range of ±7 pixels). The minimum residual (such as SAD) algorithm is used to calculate the matching block, and a motion vector table is formed. At the same time, the macro block size is dynamically adjusted according to the texture complexity of the region where the macro block is located. In the texture complex region, the macro block size is set to 8x8 pixels; in the texture flat region, the macro block size is set to 16x16 pixels.

[0060] Simplified optical flow method specific operation (optional scheme): if the simplified optical flow method is adopted, first, the image is preprocessed, such as Gaussian filtering to reduce noise interference. Then, according to the algorithm flow of the simplified optical flow method, the motion vector of the pixel points between adjacent frames is calculated. In the calculation process, by setting a reasonable error threshold (such as 0.5 pixels), the multiple iteration optimization steps of the motion vector in the traditional optical flow method are omitted, and the motion vector meeting the accuracy requirement is quickly obtained. At the same time, according to the resolution of the image and the approximate motion range of the object, the parameters of the simplified optical flow method are adjusted to adapt to different scene requirements.

[0061] 2、Motion compensation implementation Complex motion transformation compensation (optional scheme): according to the motion vector obtained by motion estimation, if it is detected that the object in the image exists rotation or scaling motion, the rotation angle and scaling ratio of the object are calculated. For example, by extracting the feature points (such as SIFT feature points) in the image, comparing the position and distribution changes of the feature points in adjacent frames, the rotation angle and scaling ratio of the object are calculated. Then, affine transformation is performed on the reference frame to make the posture and size of the object consistent in different frames. After the affine transformation, the similarity between the transformed reference frame and the corresponding region of the current frame is calculated (using the SSIM index). If the similarity is lower than the set threshold (such as 0.8), according to the difference of the similarity, the parameters of the affine transformation are fine-tuned, and the transformation and similarity calculation are performed again until the similarity reaches the threshold requirement, ensuring the accurate alignment of the motion area.

[0062] 3、Adaptive weighted fusion implementation Multi-factor weight calculation: a weight calculation model for multi-factor fusion is constructed, which inputs include motion vector, registration accuracy, local noise level, object motion direction and speed, etc. For the local noise level, the gray value variance of the local region of the image is calculated. For the object motion direction and speed, the motion vector obtained by motion estimation is calculated. For example, the direction of the motion vector is the motion direction of the object, and the ratio of the length of the motion vector to the time interval is the motion speed of the object. By normalizing these factors and according to their influence on the fusion effect, different weight coefficients are given to calculate the weight of each reference frame and the current frame when fused.

[0063] Improved multi-frame fusion: In multi-frame processing, first, according to the time interval and motion correlation between frames, the multi-frame images are divided into multiple subgroups. For example, frames with similar time intervals and high motion correlation are divided into a group. Within the subgroup, the motion estimation, compensation and adaptive weighted fusion method proposed in this embodiment is used for inter-frame fusion to obtain the subgroup fusion result. Then, the subgroup fusion results are subjected to secondary fusion. In the secondary fusion process, the weights of the subgroup fusion results are dynamically adjusted according to the quality (such as clarity, noise level, etc.) of the subgroup fusion results, and finally a high-quality denoised image is obtained.

[0064] The difference from the prior art is remarkable: 1. This embodiment not only simply filters the divided regions based on macroblock motion detection, but also uses a dynamically selected hybrid motion estimation strategy, considers compensation for complex motion, and adaptive weighted fusion for multi-factor fusion, which is innovative in comprehensiveness and complexity of technical means. The weight determination of this embodiment is not simply based on filtering and simple motion detection, but integrates multiple factors and has a unique phased fusion strategy in multi-frame processing. Compared with the existing motion detection architecture relying on neural networks, the innovative improvement of this embodiment based on traditional algorithms has obvious advantages in hardware requirements, real-time performance and algorithm interpretability. Unlike existing methods that only obtain motion information from channel images and adjust the filtering strength based on position, this embodiment constructs a comprehensive system from motion estimation to compensation and fusion. Unlike existing methods that introduce human eye adaptive weights, the weight adjustment of this embodiment is based on motion-related multiple factors and runs through the entire processing flow.

[0065] 2. The prior art has long had the problems of blurring and smearing of moving objects during denoising, and difficulty in balancing denoising effect and moving object preservation. This embodiment solves these problems through innovative technical solutions, achieving high-precision motion estimation and compensation, and precise adaptive weighted fusion, significantly improving the quality of denoised images and the ability to preserve details of moving objects.

[0066] 3. In actual tests, for images in complex motion scenes, the signal-to-noise ratio of the denoised images is improved by more than 30% compared to traditional methods, the edge clarity of moving objects is improved by more than 50%, and the motion smearing phenomenon is basically eliminated. In low-light photography scenes, it can effectively reduce noise while preserving more image details, greatly improving the visual effect of the images.

[0067] Fifth embodiment of the present application: An electronic device includes a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the multi-frame image temporal denoising method as described above.

[0068] A sixth embodiment of the present application: A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the multi-frame image time domain noise reduction method as described above.

[0069] In the description of the present application, it is to be understood that the orientations or positional relationships indicated by the terms 'center', 'longitudinal', 'transverse', 'upper', 'lower', 'front', 'back', 'left', 'right','vertical', 'horizontal', 'top', 'bottom', 'inner', 'outer', etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms 'first','second', etc. are only for the purpose of description and cannot be understood as indicating or implying relative importance or implying the number of technical features indicated. Therefore, the features defined with 'first','second', etc. can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, the meaning of 'a plurality of' is two or more.

[0070] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms'mounting', 'connecting', 'connection' should be understood broadly, for example, it can be fixed connection, or detachable connection, or integrally connected, it can be mechanical connection, or electrical connection, it can be directly connected, or indirectly connected through an intermediate medium, it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood through specific circumstances.

[0071] The above description is only the preferred embodiments of the present application and is not intended to limit the present application. It should be noted that for those skilled in the art, without departing from the technical principles of the present application, a number of improvements and modifications can be made, and these improvements and modifications should be considered as the protection scope of the present application.

Claims

1. A method for temporal noise reduction of a multi-frame image, characterized in that: The steps include the following in sequence: Step 1, motion estimation: using block matching or simplified optical flow method, the motion vector between adjacent frames is calculated, and the fusion weighting coefficient is calculated according to the registration result; Step 2, motion compensation: according to the motion vector, the reference frame is compensated to align the motion area in the multi-frame image; Step 3, adaptive weighted fusion: combined with the motion detection registration result, the compensated reference frame and the current frame are weighted and fused for the registration area, and the pixel information of the current frame is used for the active area.

2. The method of claim 1, wherein the step of applying a time domain filter to the plurality of frames of image data comprises the step of: In the step 1, the current frame is divided into multiple macroblocks, and the matching block with the minimum residual error is searched near the original coordinates of the corresponding macroblock of the reference frame for each macroblock to determine the motion vector. ​ 3. The method of claim 2, wherein, The size of the macroblock can be adjusted according to the image resolution, and the macroblock size of 8x8 pixels is used in the case of 1K image, and the macroblock size of 16x16 pixels is used in the case of 2K image.

4. The method of claim 2, wherein the step of applying a time domain filter to the plurality of frames of image data comprises the step of: The search method of the minimum residual error is the minimum residual error or the minimum residual square or the maximum normalized cross-correlation. ​ 5. The method of claim 1, wherein, The motion estimation in the step 1 uses full search algorithm or fast algorithm, and the fast algorithm is gradient descent method.

6. The method of claim 1, wherein, In the step 3, when processing multi-frame images, the motion compensation and time domain average noise reduction between the current frame and each reference frame are first performed to obtain multiple denoising images, and then the weight of each denoising image is set and weighted summation is performed to obtain the final denoising image.

7. A method of temporal noise reduction of a multi-frame image as claimed in claim 6, characterized in that, The sum of the weights of the multiple denoising images is 1.

0.

8. The method of claim 1, wherein, In the step 3, when processing multi-frame images, the fusion result of each reference frame and the current frame is directly calculated.

9. An electronic device comprising a memory and a processor, said memory having stored therein a computer program, characterized in that, The computer program is executed by the processor, so that the processor executes the steps of the multi-frame image time domain noise reduction method according to any one of claims 1-8.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor, so that the processor executes the steps of the multi-frame image time domain noise reduction method according to any one of claims 1-8.

Images