Dynamic weight-based adaptive noise-aware frame averaging image generation method

Abstract

This invention discloses a method for generating averaged images based on dynamically weighted adaptive noise-aware frames, comprising: S10, input image sequence; S20, noise level estimation; S30, spatiotemporal consistency detection; S40, adaptive weight calculation; S50, Bayesian state update; and S60, output generation. This invention maintains a dynamic, multi-dimensional state estimate for each pixel location and, based on a Bayesian update rule, calculates an optimal fusion weight each time a new image frame is received, based on the new observation value of the pixel and its local context information, thereby updating the average estimate and confidence level of that pixel. This method abandons fixed historical frame windows or global motion models, instead employing a "soft decision" recursive update approach.

Description

Technical Field

[0001] This invention belongs to the field of machine vision technology, and specifically relates to a method for generating averaged images based on dynamic weighted adaptive noise-sensing frames. Background Technology

[0002] In numerous fields such as machine vision, video analytics, medical imaging, astronomical observation, and industrial inspection, extracting clear and stable static scene information from noisy dynamic image sequences is a fundamental and crucial task. Noise in image sequences originates from a wide range of sources, including sensor thermal noise, photon shot noise, quantization noise, and random fluctuations in ambient lighting. This noise is random and difficult to completely eliminate in a single frame. However, by performing specific processing on multiple consecutive frames of the same scene, the statistical properties of the noise can be utilized to effectively suppress its impact, resulting in output images with higher signal-to-noise ratios and richer details. Among these methods, frame averaging is one of the most classic and widely used multi-frame image enhancement techniques. Its core idea is to assume that the static content of the scene remains unchanged over a short period, while noise is independently and identically distributed across each frame. By arithmetically averaging the pixel values ​​at each spatial coordinate position in the aligned multi-frame images, the static signal is enhanced, while random noise is suppressed due to its zero mean. Ideally, averaging N frames of images can improve the signal-to-noise ratio (SNR) of the image by N times.

[0003] Although the basic principle of frame averaging is clear, generating frame-averaged images efficiently, robustly, and with high quality remains a challenge in practical engineering applications, especially in complex and non-ideal environments. These challenges include, but are not limited to: local or global motion in the scene (such as camera shake or target movement), gradual or abrupt changes in lighting conditions, transient occlusions or anomalous frames in the image sequence (such as flashes or lens smudges), and the inherent contradiction between noise suppression and motion blur.

[0004] To address these challenges, existing technologies have evolved from simple static averaging to a series of more complex adaptive or weighted averaging methods. The most similar existing technologies can be broadly categorized as follows: 1. Simple Time Series Averaging Method: This is the most basic implementation. Given an image sequence of length L... ,in Represents the pixel coordinates of frame t. The intensity value at that location (for color images, each color channel can be processed independently). The final frame-averaged image. Calculated by direct arithmetic mean: ; This method is computationally simple and works well under ideal conditions where the scene is absolutely still and the lighting is constant. However, it is extremely sensitive to any motion in the scene. Even tiny uncompensated motions (at the sub-pixel level) can cause blurring and loss of detail in the averaged image. Furthermore, it treats all frames in the sequence equally; if the sequence contains low-quality frames due to momentary overexposure, underexposure, or occlusion, the bad pixels in these frames will directly contaminate the final averaging result.

[0005] 2. Frame averaging method based on motion compensation: To overcome the blurring caused by motion, such methods perform motion estimation and alignment (registration) on each frame in the sequence before averaging. Typically, a specific frame (such as the middle frame or the first frame) is selected as the reference frame. Then, using techniques such as optical flow, feature point matching (e.g., SIFT, ORB), or phase correlation, the remaining frames are estimated. Global or local motion transformation model relative to a reference frame (Usually an affine transformation or homography transformation). Next, each frame is processed according to... Transform to the coordinate system of the reference frame to obtain the aligned sequence. ,in Finally, let's talk about... Perform a simple average: ; This method significantly improves the averaging effect when there is global motion (such as camera translation and rotation). However, its drawbacks are also very obvious: First, motion estimation is computationally expensive, especially for high-resolution videos or situations requiring fine local alignment. Second, the motion estimation model may be imperfect, and alignment errors can accumulate, especially in low-texture areas or scenes with rapid motion, where mismatches can lead to "ghosting". Furthermore, it can only handle cases that conform to a preset motion model (such as global rigid motion), and it is powerless for objects moving independently in the scene (non-rigid motion, deformable motion), which will still be blurred after averaging.

[0006] 3. Recursive averaging method based on pixel-level weights: To adapt to dynamic scenes and achieve real-time processing, recursive averaging (or exponential moving average) is widely used. It does not require storing multiple frames of historical data; instead, it maintains a continuously updated average image. At each time t, a new input frame... Compared with the current average image According to a fixed weighting factor (generally Mix them together: ; in, It can be initialized to the first frame. The size determines the average "memory length". The smaller the value, the longer the influence of historical frames lasts, the better the noise suppression, but the slower the response to changes; The larger the weight, the more sensitive it is to scene changes, but the noise suppression capability weakens. This method is computationally efficient and has a small memory footprint. However, its main drawback lies in the fixed weights. Unable to handle complex situations: When the scene is static, it is desirable to use small... To achieve adequate average noise reduction; however, when the scene changes abruptly (e.g., objects moving in / out), it is desirable to use large noise reduction. To quickly update the background. Fixed weights cannot make this adaptive adjustment, resulting in either updates that are too slow, causing ghosting, or updates that are too fast, resulting in insufficient noise suppression.

[0007] 4. Weighted average method based on image quality assessment: Some more advanced schemes attempt to assign different weights to each frame or pixel in a sequence to reduce the impact of low-quality frames. Common weights are defined based on image sharpness (such as gradient energy), exposure (histogram distribution), noise level, or similarity to a reference frame. For example, weights can be calculated for each frame. Global Sharpness Score (e.g., the sum of Laplace operator responses), and then normalization is performed to obtain the weights. Finally, a weighted average is calculated: ; Alternatively, we can go even further and assign independent weights to each pixel. The weights may be based on local contrast or the difference between the pixel and the average of its neighborhood. This method can suppress the effects of blurred or abnormally exposed frames to some extent. However, existing solutions typically rely on heuristic rules to design weights, lacking in-depth modeling of noise statistics, and the weight calculation itself may be sensitive to noise. More importantly, they rarely consider the two core factors of scene dynamics (motion / static) and imaging condition reliability (noise level) in a joint and adaptive manner.

[0008] In summary, existing technical solutions are either too simplistic to handle motion and abnormal frames, computationally complex and reliant on potentially failing motion models, or employ fixed or heuristic weighting strategies that fail to achieve an intelligent balance between noise suppression, detail preservation, and dynamic adaptation. In particular, existing methods often fall short of optimal performance when dealing with real-world scenes containing complex noise (such as signal-dependent Poisson-Gaussian mixture noise), intermittent motion, and subtle variations in illumination.

[0009] Based on an in-depth analysis of the aforementioned existing technologies, their main drawbacks can be summarized as follows: The assumptions about noise models are too simplistic: most existing methods implicitly assume that the noise is additive white Gaussian noise (AWGN). However, in real-world imaging systems, noise models are often more complex, typically a mixture of signal-dependent (e.g., Poisson noise) and signal-independent (e.g., Gaussian noise) models. Simple averaging or fixed-weight recursive averaging is not the optimal estimator in such non-uniform noise fields, leading to an imbalance in denoising performance between dark areas (low SNR) and bright areas (high SNR) of the image.

[0010] Rigid motion processing capabilities: Motion compensation-based methods rely on precise alignment, which is computationally burdensome and sensitive to model errors; while methods without motion compensation directly lead to blurring. Existing methods lack a lightweight mechanism capable of distinguishing between static backgrounds and dynamic foregrounds at the pixel level and applying different averaging strategies to the two.

[0011] The weighting strategy lacks joint optimization: In existing weighted averaging methods, the calculation of weights often only considers a single factor (such as sharpness or difference), failing to uniformly and probabilistically model the spatiotemporal consistency of pixels (whether they belong to a static scene), the reliability of the current observation (noise level), and the confidence of historical estimates. This leads to unreasonable weight allocation, which can easily produce artificial traces at edges, textured areas, and motion boundaries.

[0012] Insufficient robustness to outliers and illumination changes: When extreme noise points, salt-and-pepper noise, or localized transient occlusions are present in the sequence, arithmetic averages or simple weighted averages will spread these outliers into the final result. Similarly, slow illumination changes will be smoothed by averaging, while rapid illumination abrupt changes will cause problems.

[0013] The parameters need to be manually adjusted, resulting in poor adaptability: whether it's a recursive averaging mixing factor... The threshold in quality assessment usually needs to be set manually based on experience. It cannot be automatically and online adjusted according to the content and noise characteristics of the input image sequence, which limits the universality of the method. Summary of the Invention

[0014] To overcome the shortcomings of existing technologies, a novel adaptive frame-averaged image generation method and system are proposed. This invention aims to achieve the following objectives without relying on complex and unreliable global motion estimation: It can automatically sense and adapt to complex, signal-dependent imaging noise models, achieving better noise suppression.

[0015] It can intelligently distinguish between static and dynamic regions at the pixel level, perform strong temporal averaging on static regions to maximize noise reduction, and flexibly adjust the mixing strategy for dynamic regions to avoid blurring.

[0016] Design a dynamic weighting mechanism based on a Bayesian inference framework. This weight can simultaneously reflect the spatiotemporal stability of pixel values, the noise confidence of the current observation, and the accuracy of historical estimates.

[0017] It is highly robust to abnormal pixel values ​​and lighting fluctuations, avoiding their contamination of the average results.

[0018] The entire method should be highly efficient, suitable for real-time or near-real-time processing scenarios, and most parameters should be adaptively determined to minimize user intervention. It includes the following steps: S10, Input image sequence, the input is a real-time image stream or a pre-stored image sequence. , ; S20, Noise level estimation, analysis of the current frame. Estimate the noise variance for each pixel or each local region. ; S30, Spatiotemporal consistency detection, based on current observations State estimation at the previous moment Calculate a motion / change likelihood value This indicates the probability that the pixel belongs to the static background. S40, Adaptive weight calculation, comprehensive noise level estimation Change likelihood value and the confidence level of historical estimates. The optimal weights for updating the state are calculated using optimization criteria. ; S50, Bayesian state update, using the calculated weights. Following an update equation similar to but more general than Kalman filtering, the new observations are... Fusion into the state vector In the middle, the updated state is obtained. and confidence level ; S60, Generate output, state vector The "mean" component is directly used as the frame average image output at the current time. .

[0019] Preferably, step S20 employs a fast noise estimation algorithm based on local image statistics, comprising the following steps: S21, assuming the imaging noise model is a general Poisson-Gaussian mixture model: ; in, It is an ideal noise-free signal. Indicates expectation as The Poisson random variable, K is a constant that depends on the sensor gain. It is read noise; this model can be approximated as: ; in, and It is a constant that depends on the sensor characteristics; in a locally uniform region, it can be considered... Approximate local mean ; S22, for the current frame First, calculate its local mean plot. and local variance plot Use a sliding window: ; ; in, Therefore The local neighborhood centered on, It is the number of pixels in the neighborhood; S23, divide the image into several blocks, and within each block, there is a series of... The sample points were analyzed using linear regression. Estimate the current noise parameters of the block. and Use RANSAC to exclude outliers caused by edges or textures; S24 uses bilinear interpolation to smoothly propagate the block-level noise parameters to each pixel, obtaining an instantaneous noise variance estimate for each pixel location: ; in, It is a positive number to prevent the variance from being zero. Quantify current observations The greater the uncertainty and noise variance, the lower the reliability of the observation.

[0020] Preferably, a change likelihood value is introduced in S30. , This indicates that it is most likely a static background; This indicates that a real change has very likely occurred; Compare current observations The average value compared to the previous time step Consider the state vector of the pixel Maintain the mean and the variance of the mean And velocity or gradient information.

[0021] Preferably, step S30 includes the following steps: S31, define the state vector as ,in, It is an estimate of the trend of background brightness change, and the predicted background value is: ; in, Let the frame interval be 1. The prediction uncertainty is: ; in, It is the uncertainty of historical estimates. It is the noise variance of the previous frame, representing the model error, and Q is a process noise variance, used to allow for a slow random walk of the background; S32, Calculate the standardized residuals: ; Where the denominator is the combined standard deviation of the prediction error and the current observation noise; if only Gaussian noise exists, It should follow a standard normal distribution; S33, using a smooth threshold function to... Mapped to change likelihood values : ; Alternatively, you can use sigmoid class functions: ; in, It is a threshold. Control the steepness of the transition; when hour, ;when hour, .

[0022] Preferably, in step S40, an optimal fusion weight is calculated. Used to transmit new observations With historical status The mixing is performed, and the weights are designed to satisfy the following conditions: for high-noise, static regions, i.e., low... , Smaller size allows historical information to dominate, resulting in more comprehensive and even noise reduction; suitable for low-noise, dynamic regions. Or when the confidence level of historical estimates is lower than the preset value, Larger, allowing new observations to update the status.

[0023] Preferably, step S40 includes the following steps: S41, Suppose the desired true background value is... There are two sources of information: one is prediction based on historical information. Its variance is Second, current observations Its variance is If both are true If the unbiased estimators are equal, then the optimal linear unbiased estimators are the weighted average of the inverses of their variances: ; Equivalent to recursive update: ,in ; S42, Introducing the change likelihood value To modulate the variance of the observation noise, if the change is likely to occur, i.e. This means that the current observations are unreliable for background estimation, equivalent to an infinite effective noise variance, causing their weights to approach zero; therefore, an effective observation noise variance is defined as follows: ; in, It is a very small positive number to prevent division by zero, when , ;when , This is equivalent to introducing a variable that depends on the observation equation. Degradation factors; S43, Substituting into the optimal weight formula, we obtain the adaptive weights: .

[0024] Preferably, the adaptive weights of S43 include the following cases: Noise-dominated: In the static region, i.e. If the noise is loud The larger, the better The smaller the size, the slower the update speed, and the stronger the noise reduction. Change-driven: If a change is detected, i.e. ,but ,lead to At this point, current observations are ignored, and the historical background estimate remains unchanged; Confidence balance: If historical estimates themselves are highly uncertain, i.e. Even if the noise level is not high, the value is greater than the preset value. It will also be greater than the preset value, allowing new observations to quickly establish a background model; Low-noise static area: low noise and static. It is a value that does not exceed a preset limit, which can both track possible slow changes and maintain smoothness.

[0025] Preferably, S50 includes the following steps: S51, perform state prediction: ; ; in, It is the state transition matrix, for state A simple model is , It is the velocity decay factor. It is the process noise covariance matrix, representing the uncertainty of the model; from Extract the predicted mean That is, the previous and its variance That is, the previous ; S52, Calculate Kalman gain ,and Directly related, the observation model is ,in , The Kalman gain is: ; For the state vector, the first element of the gain Weight ,Right now ; S53, perform a status update: ; ; in, It is the identity matrix, specifically the mean component: .

[0026] Preferably, the main output of S60 is a continuously updated average value image. In the first few frames of the runtime, due to low historical confidence, It will quickly converge to the input scene; then, for static backgrounds, It will become smooth and clear; for dynamic foregrounds, It will remain as a clear, uncontaminated recent background; it also outputs a confidence plot. This is used to indicate the reliability of average results in different regions.

[0027] Preferably, a confidence-based bilateral filter is added before the output of S60.

[0028] Compared with existing technologies, the image generation method based on dynamic weight adaptive noise-aware frame averaging disclosed in this invention has at least the following beneficial effects: 1. Adaptive denoising for complex noise models: This invention estimates the signal-dependent noise variance online. This allows weight calculations to be based on real, spatially varying noise characteristics, rather than a global constant. This enables the application of stronger smoothing (smaller) smoothing in dark (high-noise) areas of the image. In bright areas (low noise), it can better preserve details (moderate). This achieves optimal or near-optimal noise reduction performance under non-uniform noise fields.

[0029] 2. Precise differentiation between static and dynamic regions and intelligent updates: Utilizing "soft" change likelihood values ​​calculated based on prediction residuals and uncertainties (…). This invention can intelligently determine motion at the pixel level. For static backgrounds, the system performs long-term integration with minimal effective weights to maximize noise reduction benefits; for dynamic foregrounds, the system automatically reduces their update weights, effectively preventing motion blur and "ghosting" without requiring any complex global motion estimation or segmentation.

[0030] 3. Optimal Weight Fusion Based on Bayesian Inference: This invention transforms the frame averaging problem into a Bayesian state estimation problem. The derived adaptive weights... It has strict statistical significance (approximate optimal solution under the MMSE criterion), and it dynamically balances the confidence levels of historical estimates. ), current observed noise level ( And the possibility of scene changes () This balancing process is automatic and real-time, requiring no manual parameter tuning.

[0031] 4. High robustness to outliers and lighting changes: Since change detection is based on statistically significant residuals, extreme noise points (impulse noise) in a single frame typically do not cause [outliers / problems]. Significantly increased (because) They are also relatively large, therefore they will be naturally suppressed by the averaging process. For gradual illumination changes, the trend term in the state vector It can be modeled and tracked, making the predicted values Able to keep up with changes, thus maintaining a small And effective updates. Sudden changes in lighting are identified as "changes," and their weight is reduced to prevent the background from being instantly contaminated. Once the lighting stabilizes, the system will gradually adapt to the background under the new lighting through normal updates.

[0032] 5. High computational efficiency and real-time performance: Although the core algorithm involves multiple computational steps, all operations are pixel-level or local neighborhood-level, allowing for high parallelization. Compared to motion compensation methods that require global optimization or iteration, the recursive form of this invention has a computational complexity of O(N) (where N is the number of pixels), and its memory footprint is only the size of two frames of images plus the state vector, making it ideal for real-time processing on embedded systems, GPUs, or FPGAs.

[0033] 6. Enhanced parameter adaptability: Key thresholds in the method (For change detection) can be set in conjunction with the estimated noise level (e.g.) Process noise It can also adapt to the overall activity of the image sequence. This greatly reduces the number of parameters that users need to manually adjust, improving the practicality and versatility of the method. Attached Figure Description

[0034] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration: Figure 1 This is a flowchart illustrating the steps of the image generation method based on dynamic weight adaptive noise-aware frame averaging in an embodiment of the present invention. Figure 2 This is a flowchart illustrating the pixel and state update process of the image generation method based on dynamic weighted adaptive noise-aware frame averaging in an embodiment of the present invention. Detailed Implementation

[0035] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0036] See Figure 1 The present invention provides a method for generating an image based on dynamically weighted adaptive noise-aware frame averaging, comprising the following steps: S10, Input image sequence, the input is a real-time image stream or a pre-stored image sequence. , ; S20, Noise level estimation, analysis of the current frame. Estimate the noise variance for each pixel or each local region. ; S30, Spatiotemporal consistency detection, based on current observations State estimation at the previous moment Calculate a motion / change likelihood value This indicates the probability that the pixel belongs to the static background. S40, Adaptive weight calculation, comprehensive noise level estimation Change likelihood value and the confidence level of historical estimates. The optimal weights for updating the state are calculated using optimization criteria. ; S50, Bayesian state update, using the calculated weights. Following an update equation similar to but more general than Kalman filtering, the new observations are... Fusion into the state vector In the middle, the updated state is obtained. and confidence level ; S60, Generate output, state vector The "mean" component is directly used as the frame average image output at the current time. .

[0037] Accurate noise perception in S20 is the foundation for reasonable weight allocation. This invention employs a fast noise estimation algorithm based on local image statistics, which can adapt to signal-dependent noise. The algorithm includes the following steps: S21, assuming the imaging noise model is a general Poisson-Gaussian mixture model: ; in, It is an ideal noise-free signal. Indicates expectation as The Poisson random variable, K is a constant that depends on the sensor gain. It is read noise; this model can be approximated as: ; in, and It is a constant that depends on the sensor characteristics; in a locally uniform region, it can be considered... Approximate local mean ; S22, for the current frame First, calculate its local mean plot. and local variance plot Use a sliding window: ; ; in, Therefore The local neighborhood centered on, It is the number of pixels in the neighborhood; S23, divide the image into several blocks, and within each block, there is a series of... The sample points were analyzed using linear regression. Estimate the current noise parameters of the block. and Use RANSAC to exclude outliers caused by edges or textures; S24 uses bilinear interpolation to smoothly propagate the block-level noise parameters to each pixel, obtaining an instantaneous noise variance estimate for each pixel location: ; in, It is a positive number to prevent the variance from being zero. Quantify current observations The greater the uncertainty and noise variance, the lower the reliability of the observation.

[0038] S30 determines the new observation value of the current pixel. Is it more likely that the change originates from a static background (merely contaminated by noise) or from a foreground that has undergone real change (such as object movement or sudden changes in lighting)? We introduce a change likelihood value. . This indicates that it is most likely a static background; This indicates that a real change has very likely occurred.

[0039] We not only compare current observations The average value compared to the previous time step ,because The background itself contains noise and may be lagging. We construct a more reliable "background prediction". This considers the state vector of the pixel. We not only maintained the mean It also maintains the variance of the mean (the inverse of the confidence level). And, possibly, velocity or gradient information (for slowly changing illumination). S30 includes the following steps: S31, define the state vector as ,in, It is an estimate of the trend of background brightness change, and the predicted background value is: ; in, Let the frame interval be 1. The prediction uncertainty is: ; in, It is the uncertainty of historical estimates. It is the noise variance of the previous frame, representing the model error, and Q is a process noise variance, used to allow for a slow random walk of the background; S32, Calculate the standardized residuals: ; Where the denominator is the combined standard deviation of the prediction error and the current observation noise; if only Gaussian noise exists, It should follow a standard normal distribution; S33, using a smooth threshold function to... Mapped to change likelihood values : ; Alternatively, you can use sigmoid class functions: ; in, It is a threshold. Control the steepness of the transition; when hour, ;when hour, This soft decision-making is more robust than a hard threshold (0 or 1) and can smoothly handle edge cases.

[0040] S40 is the core of this invention. We need to calculate an optimal fusion weight. Used to transmit new observations With historical status Perform the mixing. The weight design should meet the following requirements: for high-noise, static regions (low noise)... ), The noise level should be relatively low, allowing historical information to dominate and ensuring sufficient average noise reduction; for low-noise, dynamic areas (high... ), or when the confidence level of the historical estimate itself is very low (e.g., the sequence is just beginning). The value should be relatively large to allow new observations to quickly update the status.

[0041] We derive the method based on the principle of minimum mean square error (MMSE). This includes the following steps: S41, Suppose the desired true background value is... There are two sources of information: one is prediction based on historical information. Its variance is Second, current observations Its variance is If both are true If the unbiased estimators are equal, then the optimal linear unbiased estimators are the weighted average of the inverses of their variances: ; Equivalent to recursive update: ,in ; However, this does not take into account changes in the scene ( ).when When it is very large, the current observation It may no longer be about the background. It is not an unbiased estimate, but includes the foreground signal. Directly using... This will cause an update error.

[0042] S42, Introducing the change likelihood value To modulate the variance of the observation noise, if the change is likely to occur, i.e. This means that the current observations are unreliable for background estimation, equivalent to an infinite effective noise variance, causing their weights to approach zero; therefore, an effective observation noise variance is defined as follows: ; in, It is a very small positive number to prevent division by zero, when , ;when , This is equivalent to introducing a variable that depends on the observation equation. Degradation factors; S43, Substituting into the optimal weight formula, we obtain the adaptive weights: .

[0043] This adaptive weight includes the following cases: Noise-dominated: In the static region, i.e. If the noise is loud The larger, the better The smaller the size, the slower the update speed, and the stronger the noise reduction. Change-driven: If a change is detected, i.e. ,but ,lead to At this point, current observations are ignored, and the historical background estimate remains unchanged; Confidence balance: If historical estimates themselves are highly uncertain, i.e. Even if the noise level is not high, the value is greater than the preset value. It will also be greater than the preset value, allowing new observations to quickly establish a background model; Low-noise static area: low noise and static. It is a value that does not exceed a preset limit, which can both track possible slow changes and maintain smoothness.

[0044] With optimal weights We can then update the state. The update equation follows a form similar to Kalman filtering, but the observation noise covariance is time-varying and determined by... Definition. Specifically, S50 includes the following steps: S51, perform state prediction: ; ; in, It is the state transition matrix, for state A simple model is , It is the velocity decay factor. It is the process noise covariance matrix, representing the uncertainty of the model; from Extract the predicted mean That is, the previous and its variance That is, the previous ; S52, Calculate Kalman gain ,and Directly related, the observation model is ,in , The Kalman gain is: ; For the state vector, the first element of the gain Weight ,Right now ; S53, perform a status update: ; ; in, It is the identity matrix, specifically the mean component: .

[0045] This is the core formula for recursive updates. This is the average frame image output at the current time. Simultaneously, we updated the velocity estimate. and all confidence information This prepares the frame for processing.

[0046] The S60's main output is a continuously updated average image. In the early stages of system operation (the first few frames), due to low historical confidence, It will quickly converge to the input scene. Then, for static backgrounds, It will become extremely smooth and clear; for dynamic foregrounds, It will remain a clear, uncontaminated, recent background.

[0047] In addition, the system can also output confidence maps, for example This plot can be used for downstream tasks. High-confidence regions indicate that the average results are reliable, while low-confidence regions (such as regions of continuous motion) indicate that the average results may not be meaningful static backgrounds.

[0048] Alternatively, a lightweight spatial post-filter, such as a confidence-based bilateral filter, can be added before the final output to further smooth uniform regions with high confidence but still some residual noise, while protecting high-confidence edges.

[0049] See Figure 2 This is a pixel-level state update flowchart. Taking a single pixel as an example, it illustrates the entire state update process from frame t-1 to frame t, including: reading the state. State prediction is obtained Extract from input frame And estimate noise ; Calculate the change likelihood value ; Calculate effective noise and optimal weight Finally, the state is updated to obtain... and output .

[0050] The scope of protection of this invention is not limited to the specific embodiments described above. Based on the beneficial effects described in Part Four, the key points and core points for which this invention is intended to be protected include: 1. Weighting calculation model for joint noise estimation and change detection: The core of protection lies in calculating pixel-level fusion weights. The method, especially its use of formulas Or, in a mathematically equivalent form, to address the uncertainty of historical estimates. Current observed noise level ( ) and pixel change likelihood value ( A computational system that organically combines the three.

[0051] 2. Adaptive mechanism based on effective observation noise variance: protecting the changing likelihood value Transformed into effective observation noise variance This idea. This mechanism is a key technical means to achieve the distinction between dynamic and static states and to prevent dynamic pollution from contaminating static states.

[0052] 3. Online estimation and fusion method for signal-dependent noise: During frame averaging, the noise variance of each pixel or region is estimated in real time. Specific methods for using it as one of the core inputs for weight calculation include Poisson-Gaussian mixture parameter estimation based on local statistics and its application in recursive updates.

[0053] 4. Bayesian state-space model for frame averaging: This model protects the pixel values ​​and their changing trends by modeling them as state vectors. And through the inclusion of time-varying observation noise covariance (by... This refers to a general approach for recursive updates using a Bayesian filtering framework (such as a Kalman filter) defined by the state. This includes the definition of the state, and the specific equations for prediction and updating.

[0054] 5. System Architecture and Module Implementation: Protect the complete system that implements the above method, including the specific connection relationships and data flow of the noise level estimation module, the spatiotemporal consistency detection module, the adaptive weight calculation module, and the Bayesian state update module, as well as the output method of finally generating the frame average image.

[0055] Without departing from the core idea of ​​this invention, there are various alternatives or partial alternatives that can also achieve the purpose of this invention: 1. Alternative definition of state vector: state vector It can include not only the mean and trends It can also include higher-order statistics, such as local texture feature descriptors, or parameters used to describe more complex illumination variations (such as periodic fluctuations). Observation model It can also be expanded accordingly.

[0056] 2. Alternatives for calculating the change likelihood value: The computation can be based not on the residual of a single pixel, but on the features of a small local block (e.g., 3x3) (e.g., SSD or NCC of block matching, combined with the consistency of optical flow vectors). Alternatively, a deep learning model can be used, training a lightweight network directly from... and prediction from contextual features .

[0057] 3. Alternatives to noise estimation methods: Besides linear regression based on local statistics, pre-calibrated sensor noise parameter lookup tables (LUTs) can be used, or frequency domain methods (such as wavelet transform) can be employed to estimate noise levels under the assumption of sparse signal. For scenarios where the noise model is known to be fixed, It can be simplified to a... The relevant known functions.

[0058] 4. Alternative weighting criteria: In addition to the MMSE criterion, the maximum a posteriori probability (MAP) criterion can be adopted to introduce prior constraints (such as total variation prior) into the state update. At this time, the update equation will become nonlinear (such as iterative reweighting), and a small number of iterations can be performed in each frame to further improve the edge preservation capability.

[0059] 5. Alternatives to the Filtering Framework: While the Kalman filter provides an elegant framework, in nonlinear, non-Gaussian cases, particle filtering (importance sampling) can be used to approximate the posterior distribution, especially suitable for multimodal scenes (e.g., a pixel may belong to two alternating backgrounds). Alternatively, a simpler form of exponentially weighted moving average (EWMA) can be used, but with fixed weights. Replace with our calculated dynamic weights ,Right now This is a highly simplified yet effective implementation of the core idea of ​​this invention.

[0060] 6. Multi-scale processing alternative: This method can be applied at different scales of the image pyramid, and then state information can be passed between scales. The coarse scale can capture large-scale motion, while the fine scale can preserve fine details. Cross-scale guidance can improve the ability to handle large-scale motion and complex textures.

[0061] 7. Alternative for Color Image Processing: For color images, this method can be applied independently to each color channel. However, a better alternative is to perform primary state estimation and weight calculation in the luminance channel (e.g., Y in YUV space), and then apply the calculated weights... and change likelihood value They are shared with the chroma channels (U, V) because the human eye is more sensitive to changes in brightness, and the noise characteristics of the chroma channels may differ, but the motion information is consistent.

[0062] In addition to the embodiments described above, the present invention may have other implementations. All technical solutions formed by equivalent substitution or equivalent transformation are within the scope of protection claimed by the present invention.

[0063] The present invention has been described in detail above, but its specific implementation is not limited thereto. Various modifications or alterations can be made by those skilled in the art without departing from the spirit and scope of the claims of this application.

Claims

1. A method for generating image averages based on dynamically weighted adaptive noise-aware frames, characterized in that, Includes the following steps: S10, Input image sequence, the input is a real-time image stream or a pre-stored image sequence. , ; S20, Noise level estimation, analysis of the current frame. Estimate the noise variance for each pixel or each local region. ; S30, Spatiotemporal consistency detection, based on current observations. State estimation at the previous moment Calculate a motion / change likelihood value This indicates the probability that the pixel belongs to the static background. S40, Adaptive weight calculation, comprehensive noise level estimation Change likelihood value and the confidence level of historical estimates. The optimal weights for updating the state are calculated using optimization criteria. ; S50, Bayesian state update, using the calculated weights. Following an update equation similar to but more general than Kalman filtering, the new observations are... Fusion into the state vector In the middle, the updated state is obtained. and confidence level ; S60, Generate output, state vector The "mean" component is directly used as the frame average image output at the current time. .

2. The image generation method based on dynamic weighted adaptive noise-aware frame averaging according to claim 1, characterized in that, S20 employs a fast noise estimation algorithm based on local image statistics, including the following steps: S21, assuming the imaging noise model is a general Poisson-Gaussian mixture model: ； in, It is an ideal noise-free signal. Indicates expectation as The Poisson random variable, K is a constant that depends on the sensor gain. It is read noise; this model can be approximated as: ； in, and It is a constant that depends on the sensor characteristics; in a locally uniform region, it can be considered... Approximate local mean ; S22, for the current frame First, calculate its local mean plot. and local variance plot Use a sliding window: ； ； in, Therefore The local neighborhood centered on, It is the number of pixels in the neighborhood; S23, divide the image into several blocks, and within each block, there is a series of... The sample points were analyzed using linear regression. Estimate the current noise parameters of the block. and Use RANSAC to exclude outliers caused by edges or textures; S24 uses bilinear interpolation to smoothly propagate the block-level noise parameters to each pixel, obtaining an instantaneous noise variance estimate for each pixel location: ； in, It is a positive number to prevent the variance from being zero. Quantify current observations The greater the uncertainty and noise variance, the lower the reliability of the observation.

3. The image generation method based on dynamic weighted adaptive noise-aware frame averaging according to claim 2, characterized in that, A change likelihood value is introduced in S30. , This indicates a static background; This indicates that a real change has occurred; Compare current observations The average value compared to the previous time step Consider the state vector of the pixel Maintain the mean and the variance of the mean And velocity or gradient information.

4. The image generation method based on dynamic weighted adaptive noise-aware frame averaging according to claim 2, characterized in that, S30 includes the following steps: S31, define the state vector as ,in, It is an estimate of the trend of background brightness change, and the predicted background value is: ； in, Let the frame interval be 1. The prediction uncertainty is: ； in, It is the uncertainty of historical estimates. Q is the noise variance of the previous frame, representing the model error, while Q is a process noise variance used to allow for a slow random walk of the background. S32, Calculate the standardized residuals: ； Where the denominator is the combined standard deviation of the prediction error and the current observation noise; if only Gaussian noise exists, It should follow a standard normal distribution; S33, using a smooth threshold function to... Mapped to change likelihood values : ； Alternatively, you can use sigmoid class functions: ； in, It is a threshold. Control the steepness of the transition; when hour, ;when hour, .

5. The image generation method based on dynamic weighted adaptive noise-aware frame averaging according to claim 4, characterized in that, In S40, an optimal fusion weight is calculated. Used to transmit new observations With historical status The mixing is performed, and the weights are designed to satisfy the following conditions: for high-noise, static regions, i.e., low... , Smaller size allows historical information to dominate, resulting in more comprehensive and even noise reduction; suitable for low-noise, dynamic regions. Or when the confidence level of historical estimates is lower than the preset value, Larger, allowing new observations to update the status.

6. The image generation method based on dynamic weighted adaptive noise-aware frame averaging according to claim 5, characterized in that, S40 includes the following steps: S41, Suppose we want to estimate the true background value. There are two sources of information: one is prediction based on historical information. Its variance is Second, current observations Its variance is If both are true If the unbiased estimators are equal, then the optimal linear unbiased estimators are the weighted average of the inverses of their variances: ； Equivalent to recursive update: ,in ； S42, Introducing the change likelihood value To modulate the variance of the observation noise, if the change is likely to occur, i.e. This means that the current observations are unreliable for background estimation, equivalent to an infinite effective noise variance, causing their weights to approach zero; therefore, an effective observation noise variance is defined as follows: ； in, It is a very small positive number to prevent division by zero, when , ;when , This is equivalent to introducing a variable that depends on the observation equation. Degradation factors; S43, Substituting into the optimal weight formula, we obtain the adaptive weights: 。 7. The image generation method based on dynamic weighted adaptive noise-aware frame averaging according to claim 6, characterized in that, The adaptive weights of S43 include the following cases: Noise-dominated: In the static region, i.e. If the noise is loud The larger, the better The smaller the size, the slower the update speed, and the stronger the noise reduction. Change-driven: If a change is detected, i.e. ,but ,lead to At this point, current observations are ignored, and the historical background estimate remains unchanged; Confidence balance: If historical estimates themselves are highly uncertain, i.e. Even if the noise level is not high, the value is greater than the preset value. It will also be greater than the preset value, allowing new observations to quickly establish a background model; Low-noise static area: low noise and static. It is a value that does not exceed a preset limit, which can both track possible slow changes and maintain smoothness.

8. The image generation method based on dynamic weighted adaptive noise-aware frame averaging according to claim 7, characterized in that, S50 includes the following steps: S51, perform state prediction: ； ； in, It is the state transition matrix, for state A simple model is , It is the velocity decay factor. It is the process noise covariance matrix, representing the uncertainty of the model; from Extract the predicted mean That is, the previous and its variance That is, the previous ; S52, Calculate Kalman gain ,and Directly related, the observation model is ,in , The Kalman gain is: ； For the state vector, the first element of the gain Weight ,Right now ; S53, perform a status update: ； ； in, It is the identity matrix, specifically the mean component: 。 9. The image generation method based on dynamic weighted adaptive noise-aware frame averaging according to claim 1, characterized in that, The main output of S60 is a continuously updated average value image. In the first few frames of the runtime, due to low historical confidence, It will quickly converge to the input scene; then, for static backgrounds, It will become smooth and clear; for dynamic foregrounds, It will remain as a clear, uncontaminated recent background; it also outputs a confidence plot. This is used to indicate the reliability of average results in different regions.

10. The image generation method based on dynamic weighted adaptive noise-aware frame averaging according to claim 1, characterized in that, A confidence-based bilateral filter is added before the output of S60.

Images