Smoke detection method based on adaptive time sequence feature fusion and residue self-cleaning

Abstract

The invention provides a smoke detection method based on time sequence feature fusion and residual self-cleaning, and the method comprises the steps: obtaining a background difference feature map, a time sequence change intensity feature map, a multi-scale texture feature map, a time-space gradient feature map and a frequency domain feature map based on a video image frame; and fusing the feature maps to obtain a comprehensive smoke confidence score map. Based on the comprehensive smoke confidence score plot, screening pixel points of which the smoke confidence is greater than a set threshold value as seed points for growth of a smoke area, and adding a preliminary smoke mask; screening out pixel points of which the detection support probability is greater than a set threshold value in a support domain of the seed points as grown smoke points, and adding the smoke points into a primary smoke mask; traversing all seed points to obtain a smog growth area; and eliminating artifacts in the smoke growth region to obtain the smoke mask. According to the invention, the accuracy and robustness of smoke detection can be improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent video analysis technology, and in particular to a smoke detection method based on adaptive temporal feature fusion and residual self-cleaning. Background Technology

[0002] Smoke detection has significant applications in security, industrial safety, and other fields. However, existing smoke detection methods suffer from the following bottlenecks: (1) Problems of insufficient feature utilization and collaborative failure. Traditional methods often rely on single or a few features (such as color or motion), which have limited discriminative power in complex scenes.

[0003] (2) Issues with the controllability of regional growth. Smoke diffusion is a gradual process with blurred boundaries, and existing regional growth methods are prone to overexpansion or undergrowth.

[0004] (3) Temporal remnants and artifact accumulation issues. False detections caused by momentary interference or target movement in video sequences will remain as "historical information," forming "ghost" artifacts. Traditional methods lack an active identification and removal mechanism for this type of invalid historical information, resulting in persistent false detections and seriously affecting detection stability.

[0005] (4) Problems with jitter and continuity of detection results. Single-frame detection results are easily affected by noise, resulting in unstable and discontinuous detection output in the video stream.

[0006] In summary, the shortcomings of existing smoke detection methods, such as insufficient accuracy and high false alarm rate in complex backgrounds, stem from their failure to achieve adaptive temporal feature fusion to cope with dynamic changes in smoke, and also from the lack of effective control over temporal residual artifacts. Therefore, there is an urgent need for a novel smoke detection method capable of dynamically fusing temporal features and possessing self-cleaning capabilities to meet the stringent requirements of high reliability and stability in industrial applications. Summary of the Invention

[0007] In view of this, the present invention provides a smoke detection method based on temporal feature fusion and residual self-cleaning, which can improve the accuracy and robustness of smoke detection.

[0008] To solve the above-mentioned technical problems, the present invention is implemented as follows.

[0009] A smoke detection method based on temporal feature fusion and residual self-cleaning includes: Step 1: Multi-feature fusion: Based on video image frames, background difference feature map, temporal change intensity feature map, multi-scale texture feature map, spatiotemporal gradient feature map and frequency domain feature map are obtained; the feature maps are fused to obtain a comprehensive smoke confidence score map. Step 2: Growth in the smoke area: Based on the comprehensive smoke confidence score map, smoke with a confidence level greater than a set threshold is filtered out. The pixels are used as seed points for the growth of the smoke region, and an initial smoke mask is added. ; For each seed point, a support region is configured that increases with size and confidence level. Within the support region, the probability of detecting support is selected. Greater than the set threshold The pixels are used as the smoke points to create an initial smoke mask. In the middle; traverse all seed points to obtain the smoke growth area. The detection support probability By fusing the comprehensive smoke confidence score map, the temporal variation intensity feature map, the multi-scale texture feature map, and the seed prior confidence maintained in historical smoke detection, this method achieves the desired results. And obtain; Step 3: Target the smoke growth area Eliminate artifacts and obtain smoke mask. : Joint detection supports probability Based on the temporal change intensity feature map, identify whether pixels in video image frames belong to the no-evidence state, which lacks current feature support and continuous motion evidence; and determine the no-evidence state duration based on the... Perform attenuation intensity control; Pixels whose retention time exceeds the specified time are considered artifacts and are removed.

[0010] Preferably, in step 2, when selecting seed points based on the confidence level of the smoke screening, a priori enhanced multi-feature fusion confidence level is used. With threshold This is achieved through comparison;

[0011] in, This is the comprehensive smoke confidence score map. This is the time-series variation intensity feature map. This refers to the multi-scale texture feature map; , , , These are the weighting coefficients; For the prior confidence level of the seed, this The system integrates historical smoke image detection results, which are attenuated during artifact removal in each video frame. After obtaining the final smoke mask through smoke detection, the final smoke mask is Gaussian blurred and then stacked with a fixed gain value. In order to Strengthen, thereby achieving Dynamic maintenance; Prior confidence level of seeds The maximum value;

[0012] Enhance the confidence level of multi-feature fusion based on prior knowledge. Determine the size of the support domain.

[0013] Preferably, in step 2, configuring a support domain for each seed point that increases with size and confidence is as follows: Set three confidence levels: low, medium, and high, and correspondingly set three support domains: 3×3, 5×5, and 7×7. Based on the smoke confidence level of the seed point, assign one of three support domains to the seed point.

[0014] Preferably, in step 2, the detection support probability The method for determining it is as follows:

[0015] in, This is the comprehensive smoke confidence score map. This is the time-series variation intensity feature map. This refers to the multi-scale texture feature map; The penalty factor is the edge density obtained by edge detection and neighborhood edge density statistics of video image frames; , , , These are the weighting coefficients.

[0016] Preferably, in step 2, the diffusion distance of the smoke per unit time is set. As a physical limitation on the growth of smoke points; based on diffusion distance The Euclidean distance between the smoke point and the seed point is limited to ensure that growth does not exceed the physically reasonable range.

[0017] Preferably, step 3 specifically includes: For each pixel in a video image frame, the support probability will be detected. The time-series change intensity feature map is compared with its corresponding no-evidence judgment threshold, and the pixel points whose values ​​are both less than the corresponding threshold are recorded as no-evidence state points. For each pixel in a video image frame, a no-evidence counter is set; if a pixel is determined to be a no-evidence state point, the corresponding no-evidence counter is incremented; if a pixel is not a no-evidence state point, the corresponding no-evidence counter is cleared to zero. When the no-evidence counter is 0, the set normal decay coefficient is used to determine the seed prior confidence level. A smooth decay is employed; when the no-evidence counter is ≥1, a strong decay coefficient greater than the normal decay coefficient is used to determine the seed prior confidence. Rapid decay is performed; when the no-evidence counter accumulates to the residual artifact detection threshold... At that time, the pixel was identified as a residual artifact pixel and removed from the smoke growth area. Remove from the middle and set the corresponding seed prior confidence. Forced reset to zero.

[0018] Preferably, the method further includes: after generating the final smoke mask, selecting the seed prior confidence level. Higher than the preset low threshold Pixels not adopted by the final smoke mask constitute weak residual regions; a morphological dilation operation is performed on the weak residual regions, and the seed prior confidence of the dilated regions is... Zeroing out is used to eliminate isolated weak residual artifacts.

[0019] Preferably, the method further includes: a continuous frame consistency enhancement mechanism based on optical flow extrapolation and temporal voting, specifically including two operations: Continuous frame consistency enhancement operation based on optical flow extrapolation: After step 3, the Farneback optical flow algorithm is used to calculate the motion vectors between adjacent frames. Based on the motion vectors, the smoke mask of the current video image frame is extrapolated to the next frame as a priori candidate region. In step 2, the pixels in the priori candidate region are also added as seed points to the initial smoke mask. ; The continuous frame consistency enhancement operation based on time-series voting is as follows: A sliding window of length M frames is set; when the number of consecutive frames in which the same pixel is detected as smoke within the window exceeds the set continuous threshold upper limit, the smoke confidence of the pixel in the comprehensive smoke confidence score map obtained in step 1 is multiplied by an enhancement coefficient greater than 1 to increase the probability of the pixel being selected as a seed point; if the number of consecutive frames in which the same pixel is detected as smoke within the window is less than the continuous threshold lower limit, the smoke confidence of the pixel in the comprehensive smoke confidence score map is multiplied by a reduction coefficient less than 1 to reduce the probability of the pixel being selected as a seed point; the enhanced-reduction adjusted comprehensive smoke confidence score map is substituted into step 2 to perform the smoke region growth operation.

[0020] Preferably, the background difference feature map is obtained as follows: the first frame of the video is subjected to median filtering to obtain a background model; the difference between the video image frame and the background model is calculated, the larger the difference value, the higher the probability of change; the difference value is then denoised to generate a background difference feature map. ; The temporal variation feature map is obtained as follows: a sliding window is performed in the video, the inter-frame difference of multiple consecutive video images within the sliding window is calculated, and the pixel values ​​at the same spatial location in the inter-frame difference map are arithmetically averaged to obtain an average difference image; the average difference image is then denoised to obtain the temporal variation intensity feature map. ; The method for obtaining the multi-scale texture feature map is as follows: At least two windows of different scales are established, and each pixel position in a single video image frame is processed. Local standard deviations are calculated using windows at all scales to obtain standard deviation maps at different scales; the arithmetic mean method is then used to fuse the standard deviation maps at each scale to obtain a multi-scale texture feature map. .

[0021] The spatiotemporal gradient feature map is obtained by using the Sobel operator to calculate... Calculate the directional gradient and its modulus to obtain the comprehensive gradient direction; then calculate the mean of the comprehensive gradient direction. and standard deviation ; take the mean and standard deviation By comparing the data with predefined multi-segment thresholds, the probability that each pixel belongs to the soft edge of the smoke is determined, and a score is assigned based on the probability, thus obtaining the spatiotemporal gradient feature map. .

[0022] The frequency domain feature map is obtained as follows: the video image frame is divided into multiple local windows, and a two-dimensional fast Fourier transform is performed on each local window to obtain the local power spectrum; the proportion of low-frequency energy within a preset radius with the spectrum center as the origin is calculated in each local window to the total energy of the window; the proportion of low-frequency energy in each local window is interpolated to the pixel level to obtain the frequency domain feature map. .

[0023] Preferably, when calculating the background difference feature map for each video image frame, the background model is first updated: the current video image frame is subjected to median filtering to obtain the current background map; using the current background map, the non-smoke regions finally detected in the previous frame in the background model are fused and updated with a specified learning rate, while the smoke regions remain unchanged.

[0024] Beneficial effects: (1) This invention adopts a feature fusion strategy, which integrates background difference, temporal variation, texture stability, gradient fluctuation and frequency domain features to construct a comprehensive and complementary smoke discrimination criterion. This criterion synergistically utilizes multi-dimensional features in the spatiotemporal domain, including background, temporal, texture, gradient and frequency domain, to form a robust and complementary preliminary discrimination criterion. Based on this criterion, smoke detection can improve accuracy and reduce false alarm rate.

[0025] (2) This invention incorporates artifact removal processing to address the problem of how to actively identify, rapidly attenuate, and forcibly remove pixel regions lacking long-term evidence support, thereby completely eliminating temporal artifacts and ensuring the spatiotemporal purity of the detection results. In a preferred embodiment, an evidence-free counting mechanism and a dual-speed attenuation strategy are constructed to rapidly reduce the weight and forcibly remove pixel regions lacking long-term support, effectively suppressing the "ghost residue" phenomenon. Combined with weak residual erosion and conditional zeroing mechanisms, drift seeds and pseudo-contours are further eliminated, ensuring the purity and consistency of the detection results in the temporal dimension. This mechanism is particularly suitable for scenarios where smoke has gradual changes, drift, and unstable edges, significantly improving the stability of the system in continuous video streams.

[0026] (3) In this invention, when growing smoke points, the support domain is dynamically configured according to different confidence levels, thereby controlling the growth range based on the confidence level. This allows the smoke region to expand adaptively in space without exceeding the boundary, ensuring the rationality of smoke point growth and solving the problem of adaptive and controlled expansion of the smoke region. This effectively avoids false detections and missed detections caused by background interference, changes in illumination, or blurred edges. In addition, when determining the growth point of the smoke point based on the confidence level, the seed prior confidence level is added. The seed prior confidence level incorporates historical judgments on artifacts and the results of the previous frame's smoke mask, improving the reliability and robustness of the grown smoke points. In a preferred embodiment, a growth distance threshold is added when growing smoke points to ensure that the growth does not exceed a physically reasonable range.

[0027] (4) In a preferred embodiment, considering that the edge characteristics of smoke differ significantly from those of ordinary objects, the probability of detecting support is calculated as follows: A penalty factor was added to effectively inhibit the expansion of the growth region towards clear, sharp, non-smoky edges. This coefficient does not change with the scale of the support domain to ensure that the inhibition of clear edges remains consistent.

[0028] (5) The system integrates an optical flow extrapolation and temporal voting fusion mechanism. Cross-frame mask extrapolation is achieved through the Farneback optical flow algorithm, and the N-of-M voting strategy is combined to enhance detection coherence. This design enables the system to maintain stable detection performance under conditions of thin smoke, semi-transparency, dynamic interference, or drastic changes in illumination, and has good generalization ability in industrial scenarios. Attached Figure Description

[0029] Figure 1 This is a flowchart of the smoke detection method based on temporal feature fusion and residual self-cleaning according to the present invention.

[0030] Figure 2 This describes the workflow of the PGDG module of the present invention.

[0031] Figure 3This describes the workflow of the GhostFix module in this invention. Detailed Implementation

[0032] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0033] A preferred embodiment of the present invention provides a smoke detection method based on temporal feature fusion and residual self-cleaning. The core of this preferred solution lies in the collaborative work of a candidate region generation mechanism with multi-feature fusion, a terrain decoupling module (PGDG) with limited distance growth capability, and a ghost residue management module (GhostFix) with dynamic attenuation and forced removal capability, which together constitute a highly robust smoke detection solution.

[0034] See Figure 1 The specific implementation process of this preferred solution includes the following steps: Step 1: Feature extraction and fusion to obtain robust preliminary criteria for smoke detection.

[0035] In this step, background difference feature maps are obtained based on video image frames. Time-series variation intensity characteristic map Multi-scale texture feature map Spatiotemporal gradient feature map and frequency domain feature map The feature maps are then fused to obtain a comprehensive smoke confidence score map. .

[0036] The temporal variation intensity feature map here It is also used to detect the probability of success in subsequent steps. The calculation and the judgment of the state of no evidence in the elimination of artifacts are important features introduced in this invention.

[0037] The extraction of each feature is described in detail below.

[0038] (1) Background difference feature map

[0039] The first frame of the video is median filtered to obtain the background model. The difference between the video frame and the background model is calculated; the larger the difference value, the higher the probability that the point has changed (possibly due to moving objects, including smoke). After the difference value is Gaussian blurred (e.g., 5×5 Gaussian blur) to suppress high-frequency noise, a background difference feature map is generated. .

[0040] To ensure the purity and long-term stability of the background model, this invention further maintains and updates the background model before processing each frame. Specifically, median filtering is applied to the current video image frame to obtain the current background image. Using the current background image, only the non-smoke regions detected in the previous frame are fused and updated with a specified learning rate in the background model, while the smoke regions remain unchanged, thus preventing smoke targets from contaminating the background model. The fusion update with a specified learning rate refers to a weighted summation of pixel values ​​at the same position in the background model and the current background image based on a specified learning rate. The specified learning rate is preferably 0.01 to 0.03 to ensure that the background model can adapt to the slow changes in lighting in industrial scenes.

[0041] (2) Temporal variation intensity characteristic map

[0042] A sliding window is used in the video to calculate the inter-frame difference of multiple consecutive video frames within the window. The pixel values ​​at the same spatial location in the inter-frame difference image are then arithmetically averaged to obtain the average difference image. This average difference image is then subjected to Gaussian filtering or mean filtering to eliminate noise and obtain a smoothed temporal variation intensity feature map. .

[0043] (3) Multi-scale texture feature map

[0044] Establish at least two different window sizes. In this embodiment, three different window sizes are constructed: 3×3, 5×5, and 7×7. For each pixel position in a single video image frame... Local standard deviations were calculated using windows at all scales to capture the microscopic diffusion characteristics, medium-sized cluster structure, and macroscopic distribution patterns of smoke, respectively. This step yielded standard deviation maps at different scales. The local standard deviations at the three scales were used to construct a texture feature pyramid from fine to coarse. Then, the arithmetic mean method was used to fuse the standard deviation maps at each scale to obtain a multi-scale texture feature map. This multi-scale texture feature map This provides a complete description of the multi-scale characteristics of smoke texture, effectively enhancing the robustness of smoke detection at different concentrations and diffusion stages.

[0045] Here, the arithmetic averaging operation has extremely low computational complexity and introduces almost no additional overhead, making it very suitable for real-time operation on resource-constrained embedded platforms. Its simple and stable linear operation avoids overfitting, training requirements, and instability issues that may arise from complex models, meeting the high reliability requirements of industrial systems. Furthermore, in the smoke detection task, the goal of multi-scale information complementarity has been achieved by extracting features at different scales. Simple averaging fusion has been proven in practice to be sufficient to effectively integrate this information and improve robustness to smoke of different concentrations and diffusion stages.

[0046] (4) Spatiotemporal gradient feature map

[0047] Calculate using Sobel operator Calculate the directional gradient and its modulus to obtain the comprehensive gradient direction; then calculate the mean of the comprehensive gradient direction. and standard deviation ; take the mean and standard deviation By comparing the data with predefined multi-segment thresholds, the probability that each pixel belongs to the soft edge of the smoke is determined, and a score is assigned based on the probability, thus obtaining the spatiotemporal gradient feature map. .

[0048] In a specific instance, comparing multiple threshold segments can be done as follows: If 5 < <20 and >2, then This condition matches the characteristics of smoke edges that are soft and dynamically changing.

[0049] If 5 < <20 and ≤2, then This condition applies to areas with soft textures but lacking dynamic variation.

[0050] like ≤5 or ≥20, then This condition excludes flat areas and sharp edges.

[0051] (5) Frequency domain feature map

[0052] The video image frame is divided into multiple local windows. A two-dimensional fast Fourier transform is performed on each local window to obtain the local power spectrum. The proportion of low-frequency energy within a preset radius (with the spectrum center as the origin) in each local window is calculated, reflecting the frequency distribution characteristics of the image within the window. The proportion of low-frequency energy in each local window is interpolated to the pixel level to form a frequency domain feature map of the same size as the original image. This map characterizes the frequency domain characteristics of the local region where each pixel is located, thus obtaining the frequency domain feature map. .

[0053] Smoke regions, due to their gradual grayscale changes and lack of sharp edges, exhibit a higher proportion of low-frequency energy in the frequency domain; while non-smoke regions (such as areas with complex textures and sharp edges) show different frequency domain energy distribution patterns. By setting a low-frequency energy threshold, frequency domain features can be transformed into auxiliary criteria for smoke detection.

[0054] The above process yields feature maps in five dimensions. All features are then normalized using a min-max method to the [0,1] interval to eliminate the influence of unit weight. A feature fusion model based on weight allocation is established, assigning weight coefficients reflecting the discriminative power of different features. The weight configuration can be pre-set according to scene characteristics or dynamically adjusted based on feature quality assessment. The fusion process employs a weighted average method, comprehensively considering the contribution of each feature to generate a comprehensive smoke confidence score map. , where the element value is the smoke confidence level.

[0055] In one option, the comprehensive smoke confidence score obtained from the above steps is shown in the diagram. Further enhancement-reduction adjustments can be made, with the enhancement-reduction coefficients derived from the sequential voting-based consecutive frame consistency enhancement operation in step 4.

[0056] Step 2: Growth in the smoke area: Existing region growing methods typically employ fixed thresholds or morphological dilation for region expansion, lacking an adaptive response to the physical diffusion characteristics of smoke and the confidence level of local features, which can easily lead to over-growing or under-growing. This preferred embodiment uses a growth module based on support domain probabilistic fusion and terrain decoupling (PGDG) to address the problem of adaptive and controlled expansion of smoke regions, preventing over-growing and boundary overflow.

[0057] See Figure 2 The PGDG module performs the following steps: Step 21: Based on the comprehensive smoke confidence score map Filter smoke with a confidence level greater than a set threshold The pixels are used as seed points for the growth of the smoke region, and an initial smoke mask is added. .

[0058] In one approach, the following is adopted: Score (i.e., smoke confidence level) and set threshold The comparison is performed, and pixels with values ​​greater than a threshold are used as seed points for the growth of the smoke region, and an initial smoke mask is added. .

[0059] In another preferred approach, a further fusion of prior information is used to enhance the confidence level of the multi-feature fusion. With set threshold Compare and select seed points.

[0060] By integrating the comprehensive smoke confidence score map Time-series variation intensity characteristic map Multi-scale texture feature map Seed prior confidence The specific calculation is as follows:

[0061] in, The components involved in the integration , , All were normalized using the Sigmoid function. , , , These are the weighting coefficients. The prior confidence level of the seed; Prior confidence level of seeds The maximum value.

[0062] in, The system incorporates historical smoke image detection results. It maintains a seed prior confidence map with the same resolution as the video. Its element values ​​are in the range [0, 255]; Maintenance and updates involve multiple steps, which are summarized here: ① No-evidence region treatment: In step 3, artifact removal processing, based on the situation of no-evidence regions, pixel regions lacking current evidence support are addressed. Perform normal decay, rapid decay, or forced zeroing.

[0063] ② Soft diffusion backfill of detection results: After obtaining the final smoke mask for the current frame, this method performs Gaussian blurring and superimposes it with a fixed gain (obtained from pre-experiments). .

[0064] ③ Removal of isolated weak residual artifacts: After obtaining the final smoke mask for the current frame, this method selects the seed prior confidence. Pixels exceeding a preset low threshold but not adopted by the final smoke mask constitute weak residual regions. A morphological dilation operation is performed on these weak residual regions, and the seed prior confidence of the dilated regions is then calculated. Zeroing out is used to eliminate isolated weak residual artifacts.

[0065] ④ When the algorithm starts, Setting it to an all-zero matrix indicates that there is no historical detection information.

[0066] In a better approach, prior candidate regions are further identified in the subsequent step 4, which involves enhancing frame consistency based on optical flow extrapolation. Seed points from these prior candidate regions are then added to the initial smoke mask. Equivalent to based on , The initial smoke mask is obtained by taking the union of the determined seed points and the prior seed points extrapolated from the optical flow based on the smoke mask of the previous frame. .

[0067] Step 22: Dynamically configure support domains of different scales for each seed point based on the smoke confidence level, and perform region growing inference. The higher the smoke confidence level, the larger the support domain size.

[0068] In this step, the smoke confidence level used to configure the support domain can be directly selected. Alternatively, prior enhancement of multi-feature fusion confidence can be employed. .

[0069] In one specific scheme, three confidence intervals are set: low, medium, and high, and three support domains are set accordingly: 3×3, 5×5, and 7×7. Based on the smoke confidence of the seed point, one of the three support domains is assigned to the seed point.

[0070] Step 23: Filter out the support probability within the support region. Greater than the set threshold Only such a high pixel count. Only the pixels with specific values ​​are used as the generated smoke points, and an initial smoke mask is added. This ensures the reliability and stability of the growth process.

[0071] In this step, the support probability is detected. By fusing the comprehensive smoke confidence score map, temporal variation intensity feature map, multi-scale texture feature map, and seed prior confidence, And thus obtained.

[0072] Detecting support probability The method for determining it is as follows:

[0073] in, , , , These are the weighting coefficients.

[0074] Since the edge characteristics of smoke differ significantly from those of ordinary objects, edge density is introduced as a penalty factor. It effectively inhibits the expansion of the growth region towards clear, sharp, non-smoky edges. This coefficient does not change with the scale of the support domain, ensuring that the inhibition of clear edges remains consistent.

[0075] The penalty factor The edge density map is obtained as follows: First, Canny edge detection is used to extract all obvious edges in the image, including smoke edges and object contours, outputting a binary image with edge pixels at 255 and non-edge pixels at 0. Then, neighborhood averaging filtering is used to transform the discrete binary edge map into a continuous edge density map; that is, for each pixel, the proportion of edge pixels within its 15×15 neighborhood is calculated. In the output edge density map, the value of each pixel is between [0, 1], representing the density of edges around that pixel. A value close to 1 indicates that the pixel is located in a region with very dense edges, while a value close to 0 indicates that the pixel is located in a region with sparse edges. This value is the penalty factor. .

[0076] As can be seen from the above steps, and Each includes a multi-feature fusion component. Used for preliminary judgment at the global level, its main output is candidate smoke regions. Detection support probability. This is used for refined growth control within the candidate region, and its calculation incorporates a penalty mechanism to prevent boundary overflow. These two processes are interconnected in the algorithm flow, together forming a complete decision-making chain from coarse screening to fine judgment.

[0077] Furthermore, when growing smoke points, a growth distance threshold is considered. Since smoke diffusion is a continuous physical process with a finite diffusion distance per unit time, and in a homogeneous medium, smoke diffusion tends to be isotropic (uniform in all directions), while smoke concentration decreases with increasing distance, gradually thinning at the edges, therefore, a growth distance threshold is assumed to be [missing information - likely related to a threshold value]. The diffusion distance within is Then its mathematical model corresponds to:

[0078] Where D is the diffusion coefficient. This is mathematically consistent with the square root property of Euclidean distance, the diffusion distance. As a physical limitation on the growth of smoke points; based on diffusion distance The Euclidean distance between the smoke point and the seed point is limited to ensure that growth does not exceed the physically reasonable range.

[0079] Step 24: Traverse all seed points to obtain the smoke growth region. .

[0080] Step 3: Target the smoke growth area Eliminate artifacts and obtain smoke mask. .

[0081] Traditional smoke detection methods lack proactive mechanisms to address historical detection remnants, leading to persistent "ghost" artifacts caused by target movement, sudden changes in illumination, etc., severely impacting the spatiotemporal purity of the detection results. This invention employs a ghost remnant mitigation module (GhostFix) based on no-evidence counting and dual-speed decay to solve the "ghost" artifact problem caused by temporal remnants, ensuring the spatiotemporal purity of the detection results.

[0082] The GhostFix module jointly detects the support probability. Based on the temporal change intensity feature map, identify whether pixels in video image frames belong to the no-evidence state, which lacks current feature support and continuous motion evidence; and determine the no-evidence state duration based on the... Perform attenuation intensity control; Pixels whose retention time exceeds the specified time are considered artifacts and are removed.

[0083] See Figure 3 The specific working mechanism of this GhostFix module is as follows: For each pixel in a video image frame, the support probability will be detected. The temporal change intensity feature map is compared with its corresponding no-evidence threshold. Pixels whose values ​​are both less than the corresponding threshold are identified and recorded as no-evidence state points. In one example, it can be set... and If the condition is met, the pixel is marked as a "no evidence state point"; otherwise, it is marked as a "evidence state point".

[0084] For each pixel in a video image frame, a no-evidence counter is set; if a pixel is determined to be a no-evidence state point, its corresponding no-evidence counter is incremented (+1); if a pixel is a state point with evidence, its corresponding no-evidence counter is cleared to zero. When the no-evidence counter is 0, the set normal decay coefficient is used to determine the seed prior confidence level. A smooth decay is employed; when the no-evidence counter is ≥1, a strong decay coefficient greater than the normal decay coefficient is used to determine the seed prior confidence. Rapid decay is employed. For example, the seed prior decays rapidly at 0.6, and then steadily at 0.8. This determination is based on a quantitative analysis of the dynamic characteristics of smoke: real smoke must simultaneously possess characteristic responses and evidence of continuous change, while transient interference or "ghost" remnants left by a removed target cannot meet these dual conditions. This dual-mode mechanism ensures a rapid reduction in evidence strength when reliable detection signals are lacking in the short term, while maintaining steady, gradual decay under normal conditions.

[0085] Furthermore, when the no-evidence counter accumulates to the residual artifact detection threshold... Time (in a specific instance) The pixel is identified as a residual artifact pixel and enters the "forced cleanup" phase: the seed prior confidence of the pixel is... Forced to zero, and simultaneously remove the pixel from the smoke growth area. This process clears artifacts, thus achieving artifact removal.

[0086] Furthermore, after generating the final smoke mask in step 4, the GhostFix module can further function by filtering out the seed prior confidence. Higher than the preset low threshold (e.g., 40) Pixels that are not adopted by the final smoke mask constitute weak residual regions; perform a morphological dilation operation on the weak residual regions, preferably a 3×3 small kernel dilation operation, and set the seed prior confidence of the dilated regions. Zero out the pixels to eliminate isolated weak residual artifacts in the internal state. If no matching pixels are found, no sweeping is required, and the current smoke mask output determined in step 4 remains unchanged.

[0087] Step 4: Use optical flow extrapolation and temporal voting to enhance the consistency of consecutive frames and improve the coherence and stability of smoke detection in the video stream.

[0088] Existing methods for cross-frame consistency processing mostly rely on simple inter-frame differencing or fixed sliding window averaging, lacking effective utilization of motion information and dynamic voting mechanisms, resulting in significant temporal jitter in the detection results. This invention introduces continuous frame consistency enhancement operations based on optical flow extrapolation and continuous frame consistency enhancement operations based on temporal voting to solve this problem.

[0089] Continuous frame consistency enhancement operation based on optical flow extrapolation: First, the Farneback optical flow algorithm is used to calculate the motion vectors between adjacent frames. Based on the motion vectors, the smoke mask of the current video image frame is extrapolated to the next frame, with the extrapolation error controlled within 2 pixels. The extrapolated mask position is used as a priori candidate region in the image processing of the next frame. In step 2, the pixels in the priori candidate region are also added as seed points to the initial smoke mask. .

[0090] The sequential voting-based consecutive frame consistency enhancement operation involves setting a sliding window of length M frames (e.g., M=5). When the number of consecutive frames in which the same pixel is detected as smoke within the window exceeds a set consecutive threshold (e.g., consecutive frames ≥ 3), the smoke confidence score of that pixel in the comprehensive smoke confidence score obtained in step 1 is multiplied by an enhancement coefficient greater than 1 (e.g., 1.2) to increase the probability of that pixel being selected as a seed point. If the number of consecutive frames in which the same pixel is detected as smoke within the window is less than the consecutive threshold (e.g., consecutive frames ≤ 1), the smoke confidence score of that pixel in the comprehensive smoke confidence score is multiplied by a reduction coefficient less than 1 (e.g., 0.8) to decrease the probability of that pixel being selected as a seed point. This mechanism effectively suppresses false detections in single frames and enhances the continuous signal of real smoke.

[0091] To implement the above solution, a smoke detection system can be constructed, which specifically includes: Input module: Adopting industrial-grade high-definition acquisition equipment, it supports simultaneous input of 4 channels of 1080P@30fps video streams, is compatible with protocols such as HDMI and GigE Vision, and has built-in preprocessing units such as Gamma correction, white balance and bad pixel repair to reduce noise and optimize image quality of the original acquired images, ensuring brightness consistency and detail integrity of the input images, and providing a high-quality data foundation for subsequent feature extraction.

[0092] Feature extraction module: Used to perform feature calculation and fusion.

[0093] PGDG module: used to perform the terrain decoupling and region growing.

[0094] GhostFix module: Used to perform the remediation of the ghost remnants.

[0095] Rendering module: OpenGL accelerated rendering technology is used to visualize the detection results, including highlighting of smoke areas (red semi-transparent overlay, 60% transparency), display of confidence scores, and drawing of smoke diffusion trend curves.

[0096] AI Embedded Processing Platform: Based on the ARM Cortex-A57 architecture, equipped with 4GB DDR4 memory and 128GB SSD storage, it supports fast loading of algorithm models and real-time storage of detection data. The platform power consumption is controlled within 30W, meeting the low-power deployment requirements of industrial sites.

[0097] By organically combining the above methods and systems, this invention constitutes a complete, efficient, and reliable industrial-grade smoke detection solution, which has the following beneficial effects: (1) Detection accuracy is significantly improved and the false alarm rate is greatly reduced. This invention adopts a feature fusion strategy, integrating background difference, temporal variation, texture stability, gradient fluctuation, and frequency domain features to construct a comprehensive and complementary basis for smoke discrimination. Soft diffusion terrain reconstruction and distance-limited growth are achieved through the PGDG module. Combined with edge penalty factors and dynamic support domain control, the smoke area expands adaptively in space without exceeding boundaries, effectively avoiding false detections and missed detections caused by background interference, illumination changes, or edge blurring. Experiments show that compared with traditional methods, this method improves the detection accuracy by approximately 15%~20% and reduces the false alarm rate by more than 30% in complex industrial scenarios.

[0098] (2) Thoroughly resolve temporal residue and artifact issues. By introducing the GhostFix module, an evidence-free counting mechanism and a dual-speed attenuation strategy are constructed to rapidly reduce the weight and forcibly remove pixel regions that have been without support for a long time, effectively suppressing the "ghost residue" phenomenon. Combined with weak residue erosion and conditional zeroing mechanism, drift seeds and pseudo-contours are further eliminated, ensuring the purity and consistency of detection results in the temporal dimension. This mechanism is particularly suitable for scenarios where smoke has gradual changes, drift, and unstable edges, significantly improving the stability of the system in continuous video streams.

[0099] (3) Strong robustness and environmental adaptability. The system integrates optical flow extrapolation and temporal voting fusion mechanisms. Cross-frame mask extrapolation is achieved through the Farneback optical flow algorithm, and the N-of-M voting strategy is combined to enhance detection coherence. This design enables the system to maintain stable detection performance under conditions of thin smoke, semi-transparency, dynamic interference, or drastic changes in illumination, and has good generalization ability in industrial scenarios.

[0100] (4) Strong real-time performance, suitable for embedded deployment. The algorithm fully considers computational efficiency and resource consumption during design, and the feature extraction and processing flow adopt a lightweight design. The system supports simultaneous processing of multiple high-definition video streams and has been deployed on the ARM Cortex-A57 embedded platform. The measured power consumption is less than 30W, which meets the requirements of low power consumption and high real-time performance in industrial sites.

[0101] (5) Visualization and system integration friendliness. The rendering module is based on OpenGL acceleration technology and provides visualization functions such as highlighting smoke areas, displaying confidence levels and diffusion trend curves, which facilitates intuitive judgment by monitoring personnel. The system has a clear modular design, standardized interfaces, and supports seamless integration with existing industrial video surveillance systems, demonstrating good engineering implementation capabilities.

[0102] The specific embodiments described above only illustrate the design principles of the present invention. The shapes and names of the components in this description may differ and are not limited. Therefore, those skilled in the art can modify or make equivalent substitutions to the technical solutions described in the foregoing embodiments; and these modifications and substitutions do not depart from the inventive spirit and technical solutions of the present invention, and should all fall within the protection scope of the present invention.

Claims

1. A smoke detection method based on temporal feature fusion and residual self-cleaning, characterized in that, include: Step 1: Multi-feature fusion: Based on video image frames, background difference feature map, temporal change intensity feature map, multi-scale texture feature map, spatiotemporal gradient feature map and frequency domain feature map are obtained; the feature maps are fused to obtain a comprehensive smoke confidence score map. Step 2: Growth in the smoke area: Based on the comprehensive smoke confidence score map, smoke with a confidence level greater than a set threshold is filtered out. The pixels are used as seed points for the growth of the smoke region, and an initial smoke mask is added. ; For each seed point, a support region is configured that increases with size and confidence level. Within the support region, the probability of detecting support is selected. Greater than the set threshold The pixels are used as the smoke points to create an initial smoke mask. In the middle; traverse all seed points to obtain the smoke growth area. The detection support probability By fusing the comprehensive smoke confidence score map, the temporal variation intensity feature map, the multi-scale texture feature map, and the seed prior confidence maintained in historical smoke detection, this method achieves the desired results. And obtain; Step 3: Target the smoke growth area Eliminate artifacts and obtain smoke mask. : Joint detection supports probability Based on the temporal change intensity feature map, identify whether pixels in video image frames belong to the no-evidence state, which lacks current feature support and continuous motion evidence; and determine the no-evidence state duration based on the... Perform attenuation intensity control; Pixels whose retention time exceeds the specified time are considered artifacts and are removed.

2. The method as described in claim 1, characterized in that, In step 2, when selecting seed points based on the confidence level of the smoke screening, a priori enhancement of the multi-feature fusion confidence level is used. With threshold This is achieved through comparison; in, This is the comprehensive smoke confidence score map. This is the time-series variation intensity feature map. This refers to the multi-scale texture feature map; , , , These are the weighting coefficients; For the prior confidence level of the seed, this The system integrates historical smoke image detection results, which are attenuated during artifact removal in each video frame. After obtaining the final smoke mask through smoke detection, the final smoke mask is Gaussian blurred and then stacked with a fixed gain value. In order to Strengthen, thereby achieving Dynamic maintenance; Prior confidence level of seeds The maximum value; Enhance the confidence level of multi-feature fusion based on prior knowledge. Determine the size of the support domain.

3. The method as described in claim 1, characterized in that, In step 2, configuring a support domain for each seed point that increases with size and confidence is as follows: Set three confidence levels: low, medium, and high, and correspondingly set three support domains: 3×3, 5×5, and 7×7. Based on the smoke confidence level of the seed point, assign one of three support domains to the seed point.

4. The method as described in claim 1, characterized in that, In step 2, the detection support probability The method for determining it is as follows: in, This is the comprehensive smoke confidence score map. This is the time-series variation intensity feature map. This refers to the multi-scale texture feature map; The penalty factor is the edge density obtained by edge detection and neighborhood edge density statistics of video image frames; , , , These are the weighting coefficients.

5. The method as described in claim 1, characterized in that, In step 2, the diffusion distance of the smoke per unit time is set. As a physical limitation on the growth of smoke points; based on diffusion distance The Euclidean distance between the smoke point and the seed point is limited to ensure that growth does not exceed the physically reasonable range.

6. The method as described in claim 2, characterized in that, Step 3 specifically includes: For each pixel in a video image frame, the support probability will be detected. The time-series change intensity feature map is compared with its corresponding no-evidence judgment threshold, and the pixel points whose values ​​are both less than the corresponding threshold are recorded as no-evidence state points. For each pixel in a video image frame, a no-evidence counter is set; if a pixel is determined to be a no-evidence state point, the corresponding no-evidence counter is incremented; if a pixel is not a no-evidence state point, the corresponding no-evidence counter is cleared to zero. When the no-evidence counter is 0, the set normal decay coefficient is used to determine the seed prior confidence level. A smooth decay is employed; when the no-evidence counter is ≥1, a strong decay coefficient greater than the normal decay coefficient is used to determine the seed prior confidence. Rapid decay is performed; when the no-evidence counter accumulates to the residual artifact detection threshold... At that time, the pixel was identified as a residual artifact pixel and removed from the smoke growth area. Remove from the middle and set the corresponding seed prior confidence. Forced reset to zero.

7. The method as described in claim 6, characterized in that, The method further includes: after generating the final smoke mask, selecting the seed prior confidence. Higher than the preset low threshold Pixels not adopted by the final smoke mask constitute weak residual regions; a morphological dilation operation is performed on the weak residual regions, and the seed prior confidence of the dilated regions is... Zeroing out is used to eliminate isolated weak residual artifacts.

8. The method as described in claim 1, characterized in that, The method further includes: a continuous frame consistency enhancement mechanism based on optical flow extrapolation and temporal voting, specifically comprising two operations: Continuous frame consistency enhancement operation based on optical flow extrapolation: After step 3, the Farneback optical flow algorithm is used to calculate the motion vectors between adjacent frames. Based on the motion vectors, the smoke mask of the current video image frame is extrapolated to the next frame as a priori candidate region. In step 2, the pixels in the priori candidate region are also added as seed points to the initial smoke mask. ; The continuous frame consistency enhancement operation based on time-series voting is as follows: A sliding window of length M frames is set; when the number of consecutive frames in which the same pixel is detected as smoke within the window exceeds the set continuous threshold upper limit, the smoke confidence of the pixel in the comprehensive smoke confidence score map obtained in step 1 is multiplied by an enhancement coefficient greater than 1 to increase the probability of the pixel being selected as a seed point; if the number of consecutive frames in which the same pixel is detected as smoke within the window is less than the continuous threshold lower limit, the smoke confidence of the pixel in the comprehensive smoke confidence score map is multiplied by a reduction coefficient less than 1 to reduce the probability of the pixel being selected as a seed point; the enhanced-reduction adjusted comprehensive smoke confidence score map is substituted into step 2 to perform the smoke region growth operation.

9. The method as described in claim 1, characterized in that, The background difference feature map is obtained as follows: the first frame of the video is subjected to median filtering to obtain a background model; the difference between the video image frame and the background model is calculated, and the larger the difference value, the higher the probability of change; the difference value is then denoised to generate a background difference feature map. ; The method for obtaining the temporal variation feature map is as follows: a sliding window is performed in the video, the inter-frame difference of multiple consecutive video images within the sliding window is calculated, and the pixel values ​​at the same spatial position in the inter-frame difference map are arithmetically averaged to obtain the average difference image. After denoising the average difference image, a temporal variation intensity feature map is obtained. ; The method for obtaining the multi-scale texture feature map is as follows: At least two windows of different scales are established, and each pixel position in a single video image frame is processed. Local standard deviations are calculated using windows at all scales to obtain standard deviation maps at different scales; the arithmetic mean method is then used to fuse the standard deviation maps at each scale to obtain a multi-scale texture feature map. . The spatiotemporal gradient feature map is obtained by using the Sobel operator to calculate... Calculate the directional gradient and its modulus to obtain the comprehensive gradient direction; then calculate the mean of the comprehensive gradient direction. and standard deviation ; take the mean and standard deviation By comparing the data with predefined multi-segment thresholds, the probability that each pixel belongs to the soft edge of the smoke is determined, and a score is assigned based on the probability, thus obtaining the spatiotemporal gradient feature map. . The frequency domain feature map is obtained as follows: the video image frame is divided into multiple local windows, and a two-dimensional fast Fourier transform is performed on each local window to obtain the local power spectrum; the proportion of low-frequency energy within a preset radius with the spectrum center as the origin is calculated in each local window to the total energy of the window; the proportion of low-frequency energy in each local window is interpolated to the pixel level to obtain the frequency domain feature map. .

10. The method as described in claim 9, characterized in that, When calculating the background difference feature map for each video image frame, the background model is first updated: median filtering is performed on the current video image frame to obtain the current background map; Using the current background image, the non-smoke regions detected in the previous frame in the background model are fused and updated with a specified learning rate, while the smoke regions remain unchanged.

Images