An edge flame and smoke intelligent identification method based on deep learning

Abstract

This invention discloses a deep learning-based intelligent edge flame and smoke recognition method, comprising: acquiring continuous video frames, performing normalization, correction, and noise reduction to generate a preprocessed video sequence; constructing a background static reference frame, performing pixel difference with the current frame to generate a geometric refraction potential field code; performing refraction phase mapping on the geometric refraction potential field code to form a refraction phase perturbation tensor; inputting an improved SlowFast model, dynamically adjusting the three-branch sampling, and outputting preliminary candidate regions; extracting refraction, phase, and energy evolution sequences, constructing a coupled sequence, and correcting the recognition results; calculating the risk level, marking high-risk areas, and outputting a fire warning on the edge device. This invention achieves rapid, accurate, and stable edge-side intelligent recognition and warning of flames and smoke by constructing refraction potential field features and phase perturbation features combined with an improved multi-branch SlowFast deep learning model.

Description

Technical Field

[0001] This invention relates to the field of fire detection and fire monitoring technology, and in particular to a method for intelligent identification of edge flames and smoke based on deep learning. Background Technology

[0002] Existing flame and smoke detection technologies primarily rely on traditional image processing methods and single feature threshold judgments, such as color models, optical flow analysis, and edge detection. While these methods can achieve basic fire identification in uniformly lit and stable background environments, they are susceptible to external noise and environmental disturbances in complex real-world monitoring environments. This leads to unstable detection results and a high false positive rate, especially under conditions of outdoor sunlight interference, indoor light reflection, thin smoke diffusion, and rapidly changing flame patterns. Traditional detection methods often struggle to incorporate temporal information, failing to effectively capture the high-frequency changes in flame characteristics and the slow diffusion patterns of smoke, resulting in limited recognition capabilities and an inability to meet the demands of practical fire early warning systems.

[0003] With the development of deep learning technology, more and more research is attempting to apply convolutional neural networks, temporal networks, or lightweight models to flame and smoke detection. However, most existing deep learning methods directly process video frames or use fixed-structure temporal models, lacking targeted structural design for dynamic changes in videos, especially lacking differentiated modeling of flame and smoke across time scales. Existing models struggle to simultaneously handle the rapid changes in flames and the slow spread of smoke, and the information interaction between multi-branch models is limited, resulting in insufficient utilization of temporal features. Existing deep learning methods are weak in judging the temporal consistency and physical plausibility of candidate regions, easily generating discontinuous or erroneous detection results under noise interference.

[0004] For practical applications, existing technologies still suffer from high computational consumption, slow inference speed, and inability to dynamically adapt model structures when applied to edge devices. Existing models cannot achieve a balance between high accuracy and real-time performance on resource-constrained devices, particularly in their inability to jointly analyze refraction disturbances, phase changes, and energy changes in monitoring scenarios, leading to insufficient recognition accuracy.

[0005] Therefore, how to provide a deep learning-based intelligent identification method for edge flames and smoke is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a deep learning-based intelligent identification method for edge flames and smoke. This invention fully utilizes geometric refractive potential field modeling, refractive phase perturbation analysis, and a SlowFast model improved by a three-phase time lattice, time topological gating, and self-generated characteristic flow structure to differentially model and jointly identify the rapid flickering characteristics of flames and the slow diffusion characteristics of smoke. Combined with refractive-phase-energy three-domain consistency analysis, it generates stable and reliable identification results. This method enables real-time detection of flames and smoke on edge devices with limited computing power, possessing advantages such as strong environmental adaptability, high identification accuracy, and rapid early warning response.

[0007] According to an embodiment of the present invention, a deep learning-based intelligent identification method for edge flames and smoke includes:

[0008] Collect continuous video frames from the monitored scene, perform preprocessing on the continuous video frames, and obtain a preprocessed video sequence;

[0009] A background static reference frame is constructed using a preprocessed video sequence. Pixel difference is performed between each frame image and the background static reference frame to generate a geometric refraction potential field code.

[0010] Perform refraction phase mapping on the geometric refraction potential field encoding of adjacent frames to obtain the phase sequence of refraction direction change and refraction gradient change, and form the refraction phase perturbation tensor;

[0011] The refractive phase perturbation tensor is input into the improved SlowFast model, which consists of a three-phase time lattice module, a time topology gating module, and a self-generated feature flow module. The time sampling parameters of the fast branch, slow branch, and phase branch of the improved SlowFast model are adjusted according to the refractive phase perturbation tensor to generate preliminary identification results of flame candidate regions and smoke candidate regions.

[0012] Based on the preliminary identification results of the flame and smoke candidate regions, the refraction evolution sequence, phase evolution sequence, and energy evolution sequence are extracted, a refraction-phase-energy coupling sequence is constructed, global consistency correction is performed, and flame and smoke identification results are generated.

[0013] The risk level is calculated based on the flame and smoke recognition results. Areas with a recognition probability exceeding a preset threshold are marked as risk areas, and corresponding fire warning signals are output at the edge device.

[0014] Optionally, the continuous video frames refer to a sequence of multiple images acquired in real time by the monitoring equipment at a preset frame rate and arranged in chronological order.

[0015] Optionally, the preprocessing of consecutive video frames includes resolution normalization, brightness correction, and noise suppression.

[0016] Optionally, the generation of the geometric refractive potential field encoding includes:

[0017] The preprocessed video sequence is analyzed frame by frame. Based on the brightness change amplitude, color component change amplitude and local texture stability index, the dynamics of each pixel position in each frame is determined. Pixel regions that meet one of the three change indices are marked as dynamic candidate regions, and pixel regions that do not meet any of the change indices are marked as static candidate regions.

[0018] Pixel samples belonging to static candidate regions are collected within a preset time window, with pixel positions as the unit. Samples belonging to dynamic candidate regions are removed. Clustering is performed based on the principle of similar sample values. The pixel cluster with the smallest range of change and the highest time continuity is selected from the clustering results as the background candidate dataset for pixel positions.

[0019] For each pixel location, the background candidate dataset is filtered for stability, and short-term isolated mutation samples are removed. The remaining pixel samples are averaged or the median is taken. The pixel value is used as the background static reference value of the pixel location. The background static reference values ​​of all pixel locations are combined to construct the background static reference frame.

[0020] Each frame of the preprocessed video sequence is compared pixel by pixel with the background static reference frame. An initial offset map is generated by comparing the magnitude and direction of the difference between the current pixel value and the background static reference value.

[0021] Joint consistency enhancement processing is performed on the initial offset map in both spatial and temporal dimensions. In the spatial dimension, local structures with continuous offset directions and similar amplitudes are strengthened, while in the temporal dimension, offset patterns that persist across multiple consecutive frames are strengthened, and isolated offsets that only occur in a single frame are suppressed. Geometric refraction potential field coding is generated based on the enhanced offset features.

[0022] Optionally, forming the refractive phase perturbation tensor includes:

[0023] The geometric refraction potential field encoding is aligned between adjacent frames. At each pixel position and in the preset spatial neighborhood, the direction change and intensity change between the current frame and the previous frame are calculated to obtain the initial direction field and the initial intensity field.

[0024] Multi-scale orientation anchor point detection is performed on the initial orientation field. A fixed number of orientation responses are selected from each pixel position as phase seed orientations. All orientations are discretized into a finite number of orientation levels with a preset orientation interval to obtain an orientation level map and the corresponding orientation confidence map.

[0025] Based on the orientation level map, phase unrolling and temporal continuity tracking are performed in the time dimension. The orientation level sequence of each pixel is unambiguously numbered to generate a monotonically increasing or decreasing phase trajectory index. The signed phase increment and phase accumulation between adjacent frames are calculated to obtain the phase trajectory map, phase increment map and phase accumulation map.

[0026] The phase trajectory map is subjected to topological consistency constraints and bidirectional consistency checks. In the spatial dimension, isolated phase segments are merged or removed based on connectivity and boundary continuity. In the temporal dimension, forward and backward phase tracking results are compared simultaneously. Pixel positions that do not meet the consistency requirements are marked as phase unstable, and phase stability mask and orientation consistency mask are output.

[0027] The geometric refraction potential field encoding, phase trajectory diagram, phase increment diagram, phase accumulation diagram, direction confidence diagram, phase stability mask and direction consistency mask are combined in the channel dimension to generate the refraction phase perturbation tensor.

[0028] Optionally, the preliminary identification results of the generated flame candidate region and smoke candidate region include:

[0029] The refracted phase perturbation tensor is input into the improved SlowFast model. The three-phase time lattice module includes a slow branch, a fast branch, and a phase branch. The time sampling rate, time step size, and start frame index are set for the three branches respectively, and the time axes of the three branches are aligned to output the corresponding slow branch features, fast branch features, and phase branch features.

[0030] Within the three-phase time lattice module, based on the inter-frame variation of the refractive phase perturbation tensor, the time sampling rate and time step of the slow branch are set to a low-frequency long window, the time sampling rate and time step of the fast branch are set to a high-frequency short window, and the time sampling method of the phase branch is set to fixed frame shift difference sampling. This completes the synchronization of time markers and feature alignment within the three branches, forming an aligned multi-timescale feature set.

[0031] The multi-timescale feature set and the refraction phase perturbation tensor are input into the time topology gating module. According to the order of refraction direction change and the magnitude of refraction intensity change, the activation and deactivation states of the horizontal convolution branch, the vertical convolution branch, the temporal convolution branch, and the multi-scale convolution branch are determined. The directional convolution selection parameters and receptive field selection parameters are generated. Gating is performed on the slow branch features, fast branch features, and phase branch features respectively to obtain the gated slow branch features, fast branch features, and phase branch features.

[0032] The gated slow branch features and fast branch features are input into the self-generated feature flow module, and an independent spatiotemporal intermediate feature flow is generated in the feature generation unit with a fixed topology. The spatiotemporal intermediate feature flow is fused with the gated phase branch features at the channel level, and then jointly aggregated with the multi-timescale output of the three-phase time lattice module to form a joint feature map.

[0033] Candidate region decoding and confidence calculation are performed on the joint feature map, and preliminary identification results of flame candidate region and smoke candidate region are output.

[0034] Optionally, the generation of flame recognition results and smoke recognition results includes:

[0035] For each flame candidate region and smoke candidate region, a region trajectory is established in consecutive frames according to the region identifier. The refraction direction, refraction amplitude and refraction persistence marker are extracted from the geometric refraction potential field encoding and local offset information frame by frame. They are arranged in chronological order to form a refraction evolution sequence. A timestamp is recorded for each element in the refraction evolution sequence.

[0036] Based on the phase trajectory map, phase increment map, and phase accumulation map, the phase direction level, phase change amplitude, and phase continuity marker are extracted frame by frame along the regional trajectory, arranged in chronological order to form a phase evolution sequence, and a correspondence is established with the timestamp of the refraction evolution sequence.

[0037] In each frame corresponding to the region trajectory, the energy feature set of the region is calculated. The energy feature set includes boundary roughness intensity, brightness gradient intensity and texture change intensity, which are arranged in chronological order to form an energy evolution sequence.

[0038] Using candidate region keyframes as time anchors, bidirectional time alignment and missing frame interpolation are performed on the refraction evolution sequence, phase evolution sequence, and energy evolution sequence. At each alignment time step, the refraction features, phase features, and energy features are combined at the channel level, and interactive indicators between the three types of features are added.

[0039] Based on the temporal order of the refraction-phase-energy coupling sequence, consistency analysis is performed on the coupling features of multiple consecutive frames. By evaluating the degree of matching between the refraction change trend, phase change trend and energy change trend in the time dimension, the recognition confidence of the flame candidate region and the smoke candidate region is updated, and the flame recognition result and the smoke recognition result are obtained.

[0040] Optionally, the step of outputting a corresponding fire warning signal at the edge device includes:

[0041] Receive flame recognition results and smoke recognition results, establish a fixed-length time window for each candidate region within consecutive frames, and record the existence probability, region area, boundary length, number of continuous frames, centroid displacement, orientation confidence, phase stability marker and energy change marker frame by frame to form a time series feature set;

[0042] The time series feature set is weighted and summarized according to the pre-configured weight table to obtain the comprehensive risk score sequence of each candidate region within the time window. The comprehensive risk score sequence is then smoothed and outlier is removed.

[0043] Set up multi-level risk thresholds arranged in ascending order of risk level and corresponding uplink / downlink slack parameters. Compare the comprehensive risk score sequence with the multi-level risk thresholds, combine the uplink / downlink slack parameters to complete the jitter determination, and output the risk level label corresponding to each time window.

[0044] For candidate areas that are determined to reach any level of risk, event objects are generated and written to the local event queue and audit log respectively. Based on the risk level, edge-end audio-visual prompts, switch control or network reporting are triggered, and the corresponding frame image or compressed video clip is output.

[0045] For candidate areas of generated event objects, continuous tracking and alarm withdrawal determination are performed. When the comprehensive risk score is continuously lower than the alarm withdrawal threshold in consecutive frames and the phase stability is marked as stable and the minimum number of continuous frames is met, an alarm withdrawal record is generated, the closing time and final state of the event object are updated, and the warning output of the area is terminated.

[0046] The beneficial effects of this invention are:

[0047] This invention effectively overcomes the shortcomings of existing technologies in feature representation under complex lighting, background perturbation, and sparse smoke conditions by introducing geometric refraction potential field encoding and refraction phase perturbation modeling. By constructing a static background reference frame, dynamic region culling, and offset information enhancement, subtle changes in flames and smoke within the video sequence can be stably captured, thereby improving the accuracy of dynamic feature representation at the source level. This makes the robustness of this method significantly superior to traditional image analysis methods and ordinary deep learning methods under strong interference environments.

[0048] This invention employs a SlowFast model improved by incorporating a three-phase time lattice module, a time topology gating module, and a self-generated feature flow module. This allows the model to automatically adjust sampling strategies at different time scales based on changes in refractive phase perturbation, dynamically control the convolution path, and generate a more discriminative intermediate spatiotemporal feature flow through self-generated feature flow. This enables differentiated modeling of rapid flame fluctuations and slow smoke diffusion. This structure significantly improves the efficiency of utilizing temporal features, fundamentally solving the problem of existing models' inability to simultaneously handle high-frequency and low-frequency variations, resulting in more stable and consistent recognition results.

[0049] This invention further constructs a refraction-phase-energy coupled sequence to perform cross-time period consistency evaluation on candidate regions of flames and smoke. Through joint analysis of refraction direction trends, phase change trends, and energy change trends, it effectively reduces false triggering phenomena caused by environmental noise, illumination changes, and local motion artifacts, significantly improving the reliability of the recognition results. This invention can achieve higher detection accuracy, lower false alarm rate, and faster early warning response speed in complex scenarios, possessing significant practical and engineering application value. Attached Figure Description

[0050] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0051] Figure 1 This is a flowchart of a deep learning-based intelligent identification method for edge flames and smoke proposed in this invention;

[0052] Figure 2 This is a schematic diagram of the improved SlowFast model structure of the intelligent edge flame and smoke recognition method based on deep learning proposed in this invention. Detailed Implementation

[0053] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0054] refer to Figure 1 and Figure 2 A deep learning-based intelligent method for edge flame and smoke recognition includes:

[0055] Collect continuous video frames from the monitored scene, perform preprocessing on the continuous video frames, and obtain a preprocessed video sequence;

[0056] A background static reference frame is constructed using a preprocessed video sequence. Pixel difference is performed between each frame image and the background static reference frame to generate a geometric refraction potential field code.

[0057] Perform refraction phase mapping on the geometric refraction potential field encoding of adjacent frames to obtain the phase sequence of refraction direction change and refraction gradient change, and form the refraction phase perturbation tensor;

[0058] The refractive phase perturbation tensor is input into the improved SlowFast model, which consists of a three-phase time lattice module, a time topology gating module, and a self-generated feature flow module. The time sampling parameters of the fast branch, slow branch, and phase branch of the improved SlowFast model are adjusted according to the refractive phase perturbation tensor to generate preliminary identification results of flame candidate regions and smoke candidate regions.

[0059] Based on the preliminary identification results of the flame and smoke candidate regions, the refraction evolution sequence, phase evolution sequence, and energy evolution sequence are extracted, a refraction-phase-energy coupling sequence is constructed, global consistency correction is performed, and flame and smoke identification results are generated.

[0060] The risk level is calculated based on the flame and smoke recognition results. Areas with a recognition probability exceeding a preset threshold are marked as risk areas, and corresponding fire warning signals are output at the edge device.

[0061] In this embodiment, the continuous video frame refers to a sequence of multiple images that are acquired in real time by the monitoring device at a preset frame rate and arranged in chronological order.

[0062] In this embodiment, the preprocessing of consecutive video frames includes resolution normalization, brightness correction, and noise suppression.

[0063] In this embodiment, the generation of geometric refractive potential field encoding includes:

[0064] Frame-by-frame analysis is performed on the preprocessed video sequence. Based on the magnitude of brightness change, the magnitude of color component change, and local texture stability indicators, the dynamics of each pixel position in each frame are determined. Pixel regions that simultaneously meet one of the three change indicators are marked as dynamic candidate regions, while pixel regions that do not meet any of the change indicators are marked as static candidate regions.

[0065] The magnitude of brightness change is obtained by comparing the brightness difference of pixels in the current frame with that in the previous frame;

[0066] The variation range of color components is calculated based on the color channel differences of pixels in consecutive frames;

[0067] Local texture stability is obtained by analyzing the consistent changes in texture features within the neighborhood of a pixel across consecutive frames.

[0068] Pixel samples belonging to static candidate regions are collected within a preset time window, using pixel positions as the unit. Samples belonging to dynamic candidate regions are removed. Clustering is performed based on the principle of similar sample values. From the clustering results, the pixel cluster with the smallest range of change and the highest temporal continuity is selected as the background candidate dataset for pixel positions. The clustering based on the principle of similar sample values ​​specifically involves:

[0069] Based on the numerical differences in brightness and color components of pixel samples, static samples at the same pixel position within a preset time window are sorted in ascending order of numerical distance to form an ordered sample sequence.

[0070] In an ordered sample sequence, based on the principle that the difference between adjacent samples is less than a preset similarity threshold, continuous samples are automatically divided into several sample clusters with similar pixel values, so that the pixel value changes within each sample cluster remain highly consistent.

[0071] Statistical analysis was performed on the internal variation range and temporal continuity of all sample clusters. The sample cluster with the smallest range of numerical variation and the highest continuity in the temporal dimension was selected as the final background candidate dataset for pixel location.

[0072] For each pixel location, the background candidate dataset undergoes stability filtering to remove short-term isolated mutation samples. The remaining pixel samples are then averaged or the median is taken, and the pixel value is used as the background static reference value for that pixel location. The background static reference values ​​for all pixel locations are combined to construct a background static reference frame. Specifically, the stability filtering for each pixel location's background candidate dataset involves:

[0073] In the background candidate dataset, the magnitude of change of each sample is examined in chronological order. Mutation values ​​with a magnitude of change higher than the average level of the adjacent sample groups are marked, and samples that appear only at a single time point and do not have continuity before and after are regarded as short-term isolated mutation samples.

[0074] Samples marked as short-term isolated mutations are removed, and only pixel samples with stable change amplitudes and continuous distribution over time are retained;

[0075] The temporal continuity of the background candidate dataset after removing mutant samples is re-examined. When there are large intervals within a certain period, the data is smoothed to keep the remaining samples stable in the time dimension, which serves as the final sample set for calculating the static reference value of the background.

[0076] Each frame of the preprocessed video sequence is compared pixel-by-pixel with a background static reference frame. An initial offset map is generated by comparing the magnitude and direction of the difference between the current pixel value and the background static reference value. Specifically, this comparison involves:

[0077] In each frame of the image, the luminance component value and color component value of the current pixel are read for each pixel position, and numerical difference processing is performed between them and the luminance reference value and color reference value of the corresponding pixel position in the background static reference frame.

[0078] The brightness difference results and color difference results are analyzed in a preset calculation order to obtain amplitude values ​​representing the amplitude of brightness change and color change.

[0079] After amplitude analysis is completed, the positive and negative directions of the brightness difference results and color difference results are marked, and the offset direction symbol is generated according to the positive or negative change.

[0080] At each pixel location, the brightness change amplitude, color change amplitude, and offset direction sign are combined in a predetermined order to form an offset feature record of the pixel location.

[0081] The offset features of all pixel positions are recorded and arranged in spatial order to form an initial offset map containing offset direction information and offset magnitude information of the entire frame;

[0082] Joint consistency enhancement processing is performed on the initial offset map in both spatial and temporal dimensions. Spatially, local structures with continuous offset directions and similar amplitudes are strengthened; temporally, persistent offset patterns across multiple frames are enhanced, while isolated offsets occurring only in a single frame are suppressed. Geometric refraction potential field encoding is generated based on the enhanced offset features. Specifically, strengthening local structures with continuous offset directions and similar amplitudes in the spatial dimension involves:

[0083] In the initial offset map, a fixed-size spatial neighborhood is constructed with the pixel as the center. The offset directions of each pixel in the spatial neighborhood are compared, and pixels with the same or similar offset direction values ​​are grouped into a set of pixels with continuous direction.

[0084] In a set of directionally continuous pixels, amplitude consistency processing is performed on pixels with similar offset amplitude values. By increasing the amplitude consistency-related values, the spatial structure that is directionally continuous and has similar amplitudes is enhanced.

[0085] For pixels with significant changes in offset direction or large differences in offset magnitude within the spatial neighborhood, the numerical weights for processing are reduced, and for pixels outside the set of directionally continuous pixels, a weakening process is performed to form a spatial offset structure with continuous directional distribution and clear offset magnitude hierarchy.

[0086] The enhancement of the persistent offset pattern across multiple consecutive frames in the time dimension specifically includes:

[0087] In consecutive video frames, the offset values ​​of the same pixel position are extracted in chronological order to construct a time series and record the offset direction and offset magnitude at each time point.

[0088] In a time series, the differences in the direction and magnitude of the offset are compared between adjacent time points, and time periods with continuous offset direction or stable offset magnitude are divided into continuous offset segments.

[0089] Temporal smoothing is performed on the offset values ​​in the continuous offset segments. By increasing the temporal correlation weight of values ​​at adjacent time points in the sequence, the offset mode that maintains continuity across multiple frames is enhanced.

[0090] For offset records that appear at a single time point and do not repeat at previous or subsequent time points, the numerical weights for processing are reduced, and they are treated as time-isolated offset samples, thus weakening their impact on the overall offset structure.

[0091] After completing time smoothing and isolated offset weakening, the offset direction sequence and offset amplitude sequence are output in time order to form an offset structure with a stable trend in the time dimension.

[0092] In this embodiment, forming the refractive phase perturbation tensor includes:

[0093] The geometric refraction potential field encoding undergoes two-frame alignment processing between adjacent frames. At each pixel position and within a preset spatial neighborhood, the direction change and intensity change between the current frame and the previous frame are calculated to obtain the initial direction field and initial intensity field. Specifically, the calculation of the direction change and intensity change between the current frame and the previous frame involves:

[0094] In the geometric refraction potential field encoding of the current frame, the offset direction value and offset amplitude value are read for each pixel position, and the offset direction value and offset amplitude value of the corresponding pixel position are read in the geometric refraction potential field encoding of the previous frame, forming a value pair of the current direction value and the previous frame direction value, as well as a value pair of the current amplitude value and the previous frame amplitude value.

[0095] In the direction value pair, the direction difference calculation is performed between the current direction value and the direction value of the previous frame. The direction difference result is recorded as the direction change amount. The direction change amount record reflects the range of numerical change of the offset direction between the two frames.

[0096] In the amplitude value pair, the amplitude difference calculation is performed between the current amplitude value and the amplitude value of the previous frame. The amplitude difference result is recorded as the intensity change, and the intensity change record reflects the numerical change of the offset amplitude between the two frames.

[0097] The preset spatial neighborhood is a local pixel region defined by a fixed window size with the pixel position as the center. The shape of the local pixel region is a square window, and the window size is set to 5×5 according to the spatial resolution and offset feature scale of the application scenario.

[0098] Multi-scale orientation anchor point detection is performed on the initial orientation field. A fixed number of orientation responses with the largest values ​​are selected from each pixel position as phase seed orientations. All orientations are discretized into a finite number of orientation levels with a preset orientation interval to obtain an orientation level map and a corresponding orientation confidence map. The preset orientation interval is a discrete interval divided into a fixed angle difference within the orientation value range. The orientation value range is defined as 0° to 360°. The fixed angle difference is set to 10° according to the orientation change sensitivity of the application scenario. An orientation level is formed by dividing continuous orientation values ​​into several intervals with a fixed angle difference.

[0099] Based on the orientation hierarchy map, phase unrolling and temporal continuity tracking are performed in the time dimension. The orientation hierarchy sequence of each pixel is unambiguously numbered, generating a monotonically increasing or decreasing phase trajectory index. The signed phase increment and phase accumulation between adjacent frames are calculated to obtain the phase trajectory map, phase increment map, and phase accumulation map. Specifically, the phase unrolling and temporal continuity tracking based on the orientation hierarchy map in the time dimension, and the unambiguous numbering of the orientation hierarchy sequence of each pixel, are as follows:

[0100] In the orientation level map, the orientation level values ​​of consecutive frames are read in chronological order for each pixel position, and all orientation level values ​​are arranged in time to form an orientation level sequence. Cross-level changes in orientation level between adjacent frames in the sequence are marked.

[0101] In the orientation level sequence, the jump situation of the orientation level change between adjacent frames is expanded. When the orientation level jumps from the highest level to the lowest level or from the lowest level to the highest level, the orientation level sequence is expanded in the forward or reverse direction, and the orientation level sequence is adjusted into a number sequence that changes continuously in one direction.

[0102] In the unfolded orientation level sequence, all time points are renumbered in chronological order, and the unfolded orientation level sequence is recorded as a monotonically increasing number sequence or a monotonically decreasing number sequence, forming the phase trajectory index of each pixel position in the time dimension.

[0103] The calculation of the signed phase increment and phase accumulation between adjacent frames is specifically as follows:

[0104] At each pixel location, the phase trajectory numbers expanded in the time dimension are arranged in spatial order to form a two-dimensional array containing the phase trajectory numbers of all pixel locations. The two-dimensional array is then expanded by frame number to obtain the phase trajectory map.

[0105] At each pixel position, directional difference is performed between the phase trajectory number of the next frame and the phase trajectory number of the previous frame. The difference results with positive and negative signs are arranged into a two-dimensional array according to their spatial positions and recorded frame by frame according to the frame number to form a phase increment map.

[0106] At each pixel location, the phase increment is accumulated in chronological order. The accumulated phase values ​​are arranged into a two-dimensional array according to their spatial location. A frame sequence is then established in chronological order to form a phase accumulation map.

[0107] Topological consistency constraints and bidirectional consistency checks are applied to the phase trajectory map. In the spatial dimension, isolated phase segments are merged or removed based on connectivity and boundary continuity. In the temporal dimension, both forward and backward phase tracking results are compared. Pixel positions that do not meet consistency requirements are marked as phase unstable, and a phase stability mask and a direction consistency mask are output.

[0108] In terms of spatial dimension, isolated phase segments are merged or eliminated based on connectivity and boundary continuity, specifically as follows:

[0109] In the phase trajectory map, the connectivity analysis of pixels with the same or consecutive phase numbers is performed according to the spatial neighborhood relationship. The connected region is divided into multiple phase segments, and the spatial position and area of ​​each phase segment are recorded.

[0110] In the set of phase segments, phase segments with small areas and obvious breaks at the boundary positions with surrounding phase segments are isolated segments and marked. The boundary directions between phase segments are compared to identify pairs of phase segments with small differences in boundary directions and spatially adjacent.

[0111] In adjacent phase segment pairs, phase segments with continuous boundary directions and similar phase numbers are merged, and phase segments with extremely small area or high degree of abrupt change in boundary direction are removed from the phase trajectory diagram to form a phase trajectory distribution with continuous spatial structure.

[0112] The results of both forward and backward phase tracking are compared simultaneously in the time dimension, specifically as follows:

[0113] In a continuous time frame, forward phase tracking is performed for each pixel position, and the phase number at the current time point is associated with the phase number at the subsequent time points frame by frame to form a forward phase tracking sequence;

[0114] Backward phase tracking is performed at the same pixel position. The phase number at the current time point is associated with the phase number at the previous time point frame by frame to form a backward phase tracking sequence. The phase number change trend is recorded for both sequences.

[0115] At each pixel location, the phase number change trend of the forward phase tracking sequence and the backward phase tracking sequence is compared point by point. When the two tracking results show inconsistent numbering, opposite change direction, or significant difference in change magnitude in the time direction, the time point is recorded as a phase instability point and marked as an unstable state in the phase stability mask.

[0116] The geometric refraction potential field encoding, phase trajectory diagram, phase increment diagram, phase accumulation diagram, direction confidence diagram, phase stability mask and direction consistency mask are combined in the channel dimension to generate the refraction phase perturbation tensor.

[0117] In this embodiment, the preliminary identification results of generating flame candidate regions and smoke candidate regions include:

[0118] The refracted phase perturbation tensor is input into the improved SlowFast model. The three-phase time lattice module includes a slow branch, a fast branch, and a phase branch. The time sampling rate, time step size, and start frame index are set for each of the three branches, and the time axes of the three branches are aligned. The corresponding slow branch features, fast branch features, and phase branch features are output. Specifically, the output of the corresponding slow branch features, fast branch features, and phase branch features is as follows:

[0119] In the slow branch, based on the set low-frequency time sampling rate and long time step, the refraction phase perturbation tensor is sampled at intervals in the time dimension. Convolution or time-domain processing operations are performed on the sampled time series to generate slow branch features that characterize low-frequency refraction changes and output them in chronological order.

[0120] In the fast branch, the refraction phase perturbation tensor is densely sampled according to the set high-frequency time sampling rate and short time step. Convolution or time-domain processing operations are performed on the sampled high-frequency time series to generate fast branch features that characterize rapid refraction fluctuations and output them in time order.

[0121] In the phase branch, based on a fixed phase sampling strategy and a preset starting frame index, the part of the refracted phase perturbation tensor that represents the phase trajectory change is sampled, the sampling results are subjected to phase correlation calculation operations, and phase branch features that characterize the rhythm of phase change are generated and output in chronological order.

[0122] Within the three-phase time lattice module, based on the inter-frame variation of the refractive phase perturbation tensor, the time sampling rate and time step of the slow branch are set to a low-frequency long window, the time sampling rate and time step of the fast branch are set to a high-frequency short window, and the time sampling method of the phase branch is set to fixed frame shift difference sampling. This completes the synchronization of time markers and feature alignment within the three branches, forming an aligned multi-timescale feature set.

[0123] The multi-timescale feature set and the refraction phase perturbation tensor are input into the time topology gating module. Based on the order of refraction direction change and the magnitude of refraction intensity change, the activation and deactivation states of the horizontal, vertical, temporal, and multi-scale convolution branches are determined. Directional convolution selection parameters and receptive field selection parameters are generated. Gating is applied to the slow-branch, fast-branch, and phase-branch features respectively, resulting in gated slow-branch, fast-branch, and phase-branch features. Specifically, the generation of directional convolution selection parameters and receptive field selection parameters is as follows:

[0124] In the multi-timescale feature set, the numerical values ​​of the refraction direction change and the refraction intensity change are compared in time order. Based on the magnitude of the refraction direction change in the horizontal, vertical and time directions, the horizontal convolution, vertical convolution and time domain convolution are marked as enabled or disabled respectively to form the directional convolution selection parameters.

[0125] In the record of refractive intensity variation, the intensity variation range of continuous time period is divided into ranges, and the intensity variation range is classified into small-scale range, medium-scale range or large-scale range respectively. The corresponding convolution receptive field size is marked according to different ranges to form receptive field selection parameters.

[0126] Gated slow-branch and fast-branch features are input into the self-generated feature flow module. An independent spatiotemporal intermediate feature flow is generated within a fixed-topology feature generation unit. This spatiotemporal intermediate feature flow is then fused with the gated phase-branch features at the channel level, and subsequently jointly aggregated with the multi-timescale output of the three-phase time lattice module to form a joint feature map. Specifically, the generation of an independent spatiotemporal intermediate feature flow within the fixed-topology feature generation unit involves:

[0127] In the feature generation unit with fixed topology, the gated slow branch features and the gated fast branch features are stacked according to the preset channel arrangement order. The stacked feature matrix is ​​then subjected to spatial convolution and temporal convolution operations according to a fixed convolution path to generate an intermediate feature sequence containing spatial structure information and temporal change information.

[0128] In the intermediate feature sequence, the spatial convolution output and the temporal convolution output are fused channel by channel. The fused multidimensional features are arranged into a continuous spatiotemporal intermediate feature stream according to the connection rules of a fixed topology, and output in chronological order for subsequent channel-level fusion.

[0129] Candidate region decoding and confidence calculation are performed on the joint feature map to output preliminary identification results for flame and smoke candidate regions. Specifically, the process of performing candidate region decoding and confidence calculation on the joint feature map is as follows:

[0130] In the joint feature map, feature aggregation processing is performed on each spatial location to synthesize the corresponding values ​​from the slow branch features, fast branch features and phase branch features to form a decoding input matrix containing spatial information, temporal information and phase information.

[0131] In the decoding input matrix, category response estimation is performed for all spatial locations, and flame response values ​​and smoke response values ​​are calculated independently. The category response values ​​are then arranged into a two-dimensional response distribution map.

[0132] In the two-dimensional response distribution map, the flame response value and smoke response value are numerically determined according to a preset threshold. Spatial locations that exceed the threshold are marked as flame candidate regions or smoke candidate regions. At the same time, the response intensity of each candidate region is recorded as the confidence value of the candidate region, forming the corresponding preliminary identification results.

[0133] In this embodiment, generating flame recognition results and smoke recognition results includes:

[0134] For each flame candidate region and smoke candidate region, a region trajectory is established in consecutive frames according to the region identifier. The refraction direction, refraction amplitude and refraction persistence marker are extracted from the geometric refraction potential field encoding and local offset information frame by frame. They are arranged in chronological order to form a refraction evolution sequence. A timestamp is recorded for each element in the refraction evolution sequence.

[0135] Based on the phase trajectory map, phase increment map, and phase accumulation map, the phase direction level, phase change amplitude, and phase continuity marker are extracted frame by frame along the regional trajectory, arranged in chronological order to form a phase evolution sequence, and a correspondence is established with the timestamp of the refraction evolution sequence.

[0136] In each frame corresponding to the region trajectory, an energy feature set for the region is calculated. This energy feature set includes boundary roughness intensity, brightness gradient intensity, and texture change intensity, arranged chronologically to form an energy evolution sequence, wherein:

[0137] Boundary roughness intensity: At the boundary of the region, the contour line of the region boundary is read pixel by pixel. The directional change value and position offset value between consecutive boundary pixels are differentially calculated. The differential results are arranged in spatial order to form a boundary change sequence. The boundary roughness intensity value is recorded according to the fluctuation amplitude of the boundary change sequence.

[0138] Brightness gradient intensity: Within the region, directional difference calculation is performed on the brightness component value of each pixel and the brightness component value of the adjacent pixels. The difference results are arranged in the horizontal and vertical directions to form a brightness gradient matrix. The brightness gradient intensity value is recorded according to the average level of the gradient amplitude in the brightness gradient matrix.

[0139] Texture change intensity: Within the region, texture difference calculation is performed on the texture response value of each pixel and the texture response value of the adjacent pixels. The texture difference results in consecutive time frames are combined into a texture change sequence. The texture change intensity value is recorded according to the amplitude of texture response fluctuation in the texture change sequence.

[0140] Using candidate region keyframes as time anchors, bidirectional time alignment and missing frame interpolation are performed on the refraction evolution sequence, phase evolution sequence, and energy evolution sequence. At each alignment time step, refraction features, phase features, and energy features are combined at the channel level, and interaction indicators between the three types of features are added. These interaction indicators include refraction and phase in-direction consistency indicators, phase and energy synchronization enhancement indicators, and refraction and energy dominance indicators. Weight labels obtained from the direction confidence map and phase stability mask are superimposed to construct a refraction-phase-energy coupled sequence. Specifically, the bidirectional time alignment and missing frame interpolation on the refraction evolution sequence, phase evolution sequence, and energy evolution sequence are as follows:

[0141] In the refraction evolution sequence, phase evolution sequence, and energy evolution sequence, the characteristic values ​​of each time step are recorded in chronological order. Missing time steps in the three sequences are marked, and sequence alignment relationships are established in the forward and backward directions based on the time index.

[0142] After establishing the bidirectional alignment relationship, the missing time steps in the three sequences are numerically interpolated according to the forward and backward characteristic trends. The interpolated refraction values, phase values, and energy values ​​are arranged into a continuous sequence in chronological order, providing a complete time feature input basis for channel-level combination.

[0143] Based on the temporal order of the refraction-phase-energy coupling sequence, consistency analysis is performed on the coupling features of multiple consecutive frames. By evaluating the degree of matching between the refraction change trend, phase change trend, and energy change trend in the time dimension, the recognition confidence of the flame candidate region and the smoke candidate region is updated, resulting in flame recognition results and smoke recognition results. Specifically, the evaluation of the degree of matching between the refraction change trend, phase change trend, and energy change trend in the time dimension involves:

[0144] In the refraction-phase-energy coupling sequence, the numerical sequences of refraction change, phase change, and energy change in each frame are compared in chronological order. The differences in the direction, magnitude, and continuity of change of the three types of values ​​at the same time position are measured, and the results of the difference measurement are recorded as the time matching degree value.

[0145] Among all time-matching scores, continuous segments are identified for time periods with high matching scores. The average matching level of these continuous segments is recorded as the final matching score. This final matching score is then used as the input for updating the confidence scores of flame and smoke candidate regions, thus forming a matching evaluation sequence.

[0146] In this embodiment, the step of outputting a corresponding fire warning signal at the edge device includes:

[0147] Receive flame recognition results and smoke recognition results, establish a fixed-length time window for each candidate region within consecutive frames, and record the existence probability, region area, boundary length, number of continuous frames, centroid displacement, orientation confidence, phase stability marker and energy change marker frame by frame to form a time series feature set;

[0148] The time series feature set is weighted and summarized according to a pre-configured weight table to obtain the comprehensive risk score sequence of each candidate region within the time window. Smoothing and outlier removal are performed on the comprehensive risk score sequence. The pre-configured weight table is a fixed weight mapping table set according to the sensitivity level of refraction change feature value, phase change feature value and energy change feature value in historical samples. The mapping table assigns refraction weight to refraction change feature value, phase weight to phase change feature value and energy weight to energy change feature value. The three types of weights are recorded according to a fixed ratio relationship. The recording method adopts a tabular storage structure, where each row corresponds to a feature type and each column records the weight value corresponding to different sensitivity intervals.

[0149] Set up multi-level risk thresholds arranged in ascending order of risk level and corresponding uplink / downlink slack parameters. Compare the comprehensive risk score sequence with the multi-level risk thresholds, combine the uplink / downlink slack parameters to complete the jitter determination, and output the risk level label corresponding to each time window.

[0150] For candidate areas that are determined to reach any level of risk, event objects are generated and written to the local event queue and audit log respectively. Based on the risk level, edge-end audio-visual prompts, switch control or network reporting are triggered, and the corresponding frame image or compressed video clip is output.

[0151] For candidate areas of generated event objects, continuous tracking and alarm withdrawal determination are performed. When the comprehensive risk score is continuously lower than the alarm withdrawal threshold in consecutive frames and the phase stability is marked as stable and the minimum number of continuous frames is met, an alarm withdrawal record is generated, the closing time and final state of the event object are updated, and the warning output of the area is terminated.

[0152] Example 1:

[0153] To verify the feasibility of this invention in practice, it was applied to a smart fire monitoring system project. The experimental site was a warehouse-style supermarket that had been built for over eight years. The building's interior features dense shelving, narrow aisles, and is characterized by numerous light reflections, complex background textures, and frequent dust disturbances, posing a significant challenge to the accuracy of flame and smoke detection. To achieve real-time early warning and accurate identification, this invention was deployed in an edge computing terminal on-site. The terminal was configured with a quad-core ARM processor and 2GB of memory, and connected to eight 1080p network camera video streams. Continuous testing was conducted at night and during peak operating hours.

[0154] In practical applications, the continuous video frames captured by the camera are first input into a preprocessing unit to complete resolution normalization, brightness correction, and noise suppression. This invention analyzes the brightness changes, color changes, and texture stability of pixels across multiple frames to eliminate dynamic interference areas such as forklift movement, glare, and personnel movement. The remaining static candidate pixels are then aggregated within a time window to construct a stable static background reference frame. During a morning goods placement operation, the highly reflective metal frame at the entrance of the passageway flickered violently multiple times under strong light, causing multiple misjudgments in traditional background subtraction methods. In this scenario, this invention effectively eliminates flicker interference through a static candidate pixel aggregation mechanism, ensuring the stability of the background reference frame.

[0155] After generating the background static reference frame, this invention performs pixel-level difference extraction between each image frame and the background static reference frame. Based on the magnitude, direction, and duration of the difference, a local geometric offset map is generated, and after spatial and temporal consistency enhancement, a geometric refractive potential field code is obtained. This code accurately reflects the background refraction changes caused by smoke or hot air. In a test conducted on the morning of May 27, 2025, there was slight hot air rising behind the shelf, resulting in very low refraction intensity. However, the geometric refractive potential field code generated by this invention was still able to capture the faint offset pattern at the background edge, ensuring the usability of subsequent phase mapping.

[0156] This invention performs refraction direction analysis, direction-level discretization, phase trajectory construction, and phase continuity correction on the geometric refraction potential field encoding of adjacent frames to form a refraction phase perturbation tensor. This tensor accurately reflects the directional fluctuations caused by the rapid flickering of flames and the continuous offset trend caused by the slow diffusion of smoke. During a nighttime inspection, the reflection of light at the end of the passage caused local bright spots to flicker randomly. However, due to the discontinuity of its refraction direction and the break in its phase trajectory, this invention successfully excluded it from the next stage as a candidate flame region.

[0157] Subsequently, the refractive phase perturbation tensor is used to improve the SlowFast model, which consists of a three-phase time lattice module, a time topology gating module, and a self-generated feature flow module. In a simulated smoke test in the shelf area, the phase branch captures the slowly rising refractive direction sequence; the fast branch identifies features without obvious jumps; and the slow branch steadily extracts low-frequency diffusion trends. The time topology gating module automatically closes high-frequency convolution paths and enhances the weights of convolutions with large receptive fields, enabling the model to more accurately identify smoke features. The self-generated feature flow module generates an intermediate spatiotemporal flow from the features of the slow and fast branches according to rules, making the smoke region more clearly identified. Under this collaborative mechanism, the present invention obtains the correct candidate region in the 5th frame after the smoke appears, while the control model does not identify the target until the 13th frame.

[0158] This invention extracts the refraction direction trend, phase change trend, and energy change trend frame by frame from candidate regions of flames and smoke, constructs a refraction-phase-energy coupled sequence, and performs consistency analysis. In a flame simulation test, flame jumping caused rapid changes in refraction direction, frequent fluctuations in phase increment, and a continuous increase in energy value. This invention successfully confirmed the real flame by utilizing the consistency of these three factors. While light reflection showed brightness changes, its refraction direction was unstable and its phase trajectory was discontinuous, thus it was excluded as a false trigger.

[0159] Ultimately, this invention generates a risk score and risk level based on the identification results. In a real-world incident, at 2:16 AM on June 3, 2025, a small piece of paper smoldered in a corner of the warehouse, producing very thin smoke. This invention detected a refractive disturbance 1.6 seconds after the smoke appeared and issued a low-level risk warning before the smoke became obvious. The system prompted personnel to take timely action, thus preventing further escalation.

[0160] Table 1. Comparison of recognition performance of different methods in warehouse monitoring scenarios.

[0161]

[0162] As shown in Table 1, this invention demonstrates outstanding performance in smoke detection. Whether dealing with thin smoke or gray smoke, the accuracy rate remains above 92%, outperforming the 74%–77% of the standard SlowFast model and the 55%–59% of BaselineCNN. This indicates that the invention effectively captures the subtle refractive changes in the early stages of smoke diffusion through geometric refraction potential field encoding and refraction phase perturbation modeling, enabling the system to maintain high accuracy even under conditions of weak smoke, complex backgrounds, and unstable lighting. Furthermore, the average detection time of this invention is between 1.5 and 1.8 seconds, significantly faster than SlowFast and CNN, validating its real-time advantages when deployed at edge environments.

[0163] In flame detection tasks, this invention achieves recognition accuracy of 91.6% for weak flickering flames and 95.2% for medium-intensity flames, representing an overall improvement of over 20% compared to the ordinary SlowFast model and over 30% compared to BaselineCNN. This invention improves the SlowFast model through a three-phase temporal lattice, temporal topological gating, and a self-generated feature flow structure, enabling it to distinguish the temporal feature differences between illumination flicker and flame flickering, significantly enhancing its ability to capture rhythmic changes in flames. Furthermore, this invention achieves a detection latency as low as 0.19 seconds, approximately 40% and 60% faster than SlowFast and CNN, respectively, enabling rapid response.

[0164] In complex environmental interference tests, the false alarm rate of this invention was only 1.8%, far lower than SlowFast's 6.2% and CNN's 11.4%. In real-world scenarios such as warehouses and supermarkets, reflected light, dust movement, and human interference are very common. This invention comprehensively judges the perturbation trend of continuous frames through a refraction-phase-energy coupling sequence, effectively eliminating false triggers caused by spurious changes. This invention demonstrates significant advantages in recognition accuracy, response speed, and anti-interference performance, providing more reliable technical support for early fire warning at the edge.

[0165] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for intelligent recognition of edge flames and smoke based on deep learning, characterized in that, include: Collect continuous video frames from the monitored scene, perform preprocessing on the continuous video frames, and obtain a preprocessed video sequence; A background static reference frame is constructed using a preprocessed video sequence. Pixel difference is performed between each frame image and the background static reference frame to generate a geometric refraction potential field code. Perform refraction phase mapping on the geometric refraction potential field encoding of adjacent frames to obtain the phase sequence of refraction direction change and refraction gradient change, and form the refraction phase perturbation tensor; The refractive phase perturbation tensor is input into the improved SlowFast model, which consists of a three-phase time lattice module, a time topology gating module, and a self-generated feature flow module. The time sampling parameters of the fast branch, slow branch, and phase branch of the improved SlowFast model are adjusted according to the refractive phase perturbation tensor to generate preliminary identification results of flame candidate regions and smoke candidate regions. Based on the preliminary identification results of the flame and smoke candidate regions, the refraction evolution sequence, phase evolution sequence, and energy evolution sequence are extracted, a refraction-phase-energy coupling sequence is constructed, global consistency correction is performed, and flame and smoke identification results are generated. Based on the flame and smoke recognition results, the risk level is calculated, and areas with a recognition probability exceeding a preset threshold are marked as risk areas. Corresponding fire warning signals are then output at the edge device. The generation of geometric refractive potential field encoding includes: performing frame-by-frame analysis on the preprocessed video sequence, and making dynamic determination of the position of each pixel in each frame based on the brightness change amplitude, color component change amplitude and local texture stability index, marking the pixel region that simultaneously meets one of the three change indices as a dynamic candidate region, and marking the pixel region that does not meet any change index as a static candidate region. Pixel samples belonging to static candidate regions are collected within a preset time window, with pixel positions as the unit. Samples belonging to dynamic candidate regions are removed. Clustering is performed based on the principle of similar sample values. The pixel cluster with the smallest range of change and the highest time continuity is selected from the clustering results as the background candidate dataset for pixel positions. For each pixel location, the background candidate dataset is filtered for stability, and short-term isolated mutation samples are removed. The remaining pixel samples are averaged or the median is taken. The pixel value is used as the background static reference value of the pixel location. The background static reference values ​​of all pixel locations are combined to construct the background static reference frame. Each frame of the preprocessed video sequence is compared pixel by pixel with the background static reference frame. An initial offset map is generated by comparing the magnitude and direction of the difference between the current pixel value and the background static reference value. Joint consistency enhancement processing is performed on the initial offset map in both spatial and temporal dimensions. In the spatial dimension, local structures with continuous offset directions and similar amplitudes are strengthened, while in the temporal dimension, offset patterns that persist across multiple consecutive frames are strengthened, and isolated offsets that only occur in a single frame are suppressed. Geometric refraction potential field coding is generated based on the enhanced offset features. The formation of the refractive phase perturbation tensor includes: performing two-frame alignment processing on the geometric refractive potential field encoding between adjacent frames, calculating the direction change and intensity change between the current frame and the previous frame at each pixel position and in a preset spatial neighborhood, and obtaining the initial direction field and the initial intensity field. Multi-scale orientation anchor point detection is performed on the initial orientation field. A fixed number of orientation responses are selected from each pixel position as phase seed orientations. All orientations are discretized into a finite number of orientation levels with a preset orientation interval to obtain an orientation level map and the corresponding orientation confidence map. Based on the orientation level map, phase unrolling and temporal continuity tracking are performed in the time dimension. The orientation level sequence of each pixel is unambiguously numbered to generate a monotonically increasing or decreasing phase trajectory index. The signed phase increment and phase accumulation between adjacent frames are calculated to obtain the phase trajectory map, phase increment map and phase accumulation map. The phase trajectory map is subjected to topological consistency constraints and bidirectional consistency checks. In the spatial dimension, isolated phase segments are merged or removed based on connectivity and boundary continuity. In the temporal dimension, forward and backward phase tracking results are compared simultaneously. Pixel positions that do not meet the consistency requirements are marked as phase unstable, and phase stability mask and orientation consistency mask are output. The geometric refraction potential field encoding, phase trajectory diagram, phase increment diagram, phase accumulation diagram, direction confidence diagram, phase stability mask and direction consistency mask are combined in the channel dimension to generate the refraction phase perturbation tensor.

2. The method for intelligent recognition of edge flames and smoke based on deep learning according to claim 1, characterized in that, The continuous video frames refer to a sequence of multiple images that are captured in real time by monitoring equipment at a preset frame rate and arranged in chronological order.

3. The method for intelligent recognition of edge flames and smoke based on deep learning according to claim 1, characterized in that, The preprocessing of consecutive video frames includes resolution normalization, brightness correction, and noise suppression.

4. The method for intelligent recognition of edge flames and smoke based on deep learning according to claim 1, characterized in that, The preliminary identification results of the generated flame candidate region and smoke candidate region include: The refracted phase perturbation tensor is input into the improved SlowFast model. The three-phase time lattice module includes a slow branch, a fast branch, and a phase branch. The time sampling rate, time step size, and start frame index are set for the three branches respectively, and the time axes of the three branches are aligned to output the corresponding slow branch features, fast branch features, and phase branch features. Within the three-phase time lattice module, based on the inter-frame variation of the refractive phase perturbation tensor, the time sampling rate and time step of the slow branch are set to a low-frequency long window, the time sampling rate and time step of the fast branch are set to a high-frequency short window, and the time sampling method of the phase branch is set to fixed frame shift difference sampling. This completes the synchronization of time markers and feature alignment within the three branches, forming an aligned multi-timescale feature set. The multi-timescale feature set and the refraction phase perturbation tensor are input into the time topology gating module. According to the order of refraction direction change and the magnitude of refraction intensity change, the activation and deactivation states of the horizontal convolution branch, the vertical convolution branch, the temporal convolution branch, and the multi-scale convolution branch are determined. The directional convolution selection parameters and receptive field selection parameters are generated. Gating is performed on the slow branch features, fast branch features, and phase branch features respectively to obtain the gated slow branch features, fast branch features, and phase branch features. The gated slow branch features and fast branch features are input into the self-generated feature flow module, and an independent spatiotemporal intermediate feature flow is generated in the feature generation unit with a fixed topology. The spatiotemporal intermediate feature flow is fused with the gated phase branch features at the channel level, and then jointly aggregated with the multi-timescale output of the three-phase time lattice module to form a joint feature map. Candidate region decoding and confidence calculation are performed on the joint feature map, and preliminary identification results of flame candidate region and smoke candidate region are output.

5. The method for intelligent recognition of edge flames and smoke based on deep learning according to claim 1, characterized in that, The generation of flame recognition results and smoke recognition results includes: For each flame candidate region and smoke candidate region, a region trajectory is established in consecutive frames according to the region identifier. The refraction direction, refraction amplitude and refraction persistence marker are extracted from the geometric refraction potential field encoding and local offset information frame by frame. They are arranged in chronological order to form a refraction evolution sequence. A timestamp is recorded for each element in the refraction evolution sequence. Based on the phase trajectory map, phase increment map, and phase accumulation map, the phase direction level, phase change amplitude, and phase continuity marker are extracted frame by frame along the regional trajectory, arranged in chronological order to form a phase evolution sequence, and a correspondence is established with the timestamp of the refraction evolution sequence. In each frame corresponding to the region trajectory, the energy feature set of the region is calculated. The energy feature set includes boundary roughness intensity, brightness gradient intensity and texture change intensity, which are arranged in chronological order to form an energy evolution sequence. Using candidate region keyframes as time anchors, bidirectional time alignment and missing frame interpolation are performed on the refraction evolution sequence, phase evolution sequence, and energy evolution sequence. At each alignment time step, the refraction features, phase features, and energy features are combined at the channel level, and interactive indicators between the three types of features are added. Based on the temporal order of the refraction-phase-energy coupling sequence, consistency analysis is performed on the coupling features of multiple consecutive frames. By evaluating the degree of matching between the refraction change trend, phase change trend and energy change trend in the time dimension, the recognition confidence of the flame candidate region and the smoke candidate region is updated, and the flame recognition result and the smoke recognition result are obtained.

6. The method for intelligent recognition of edge flames and smoke based on deep learning according to claim 1, characterized in that, The step of outputting a corresponding fire warning signal at the edge device includes: Receive flame recognition results and smoke recognition results, establish a fixed-length time window for each candidate region within consecutive frames, and record the existence probability, region area, boundary length, number of continuous frames, centroid displacement, orientation confidence, phase stability marker and energy change marker frame by frame to form a time series feature set; The time series feature set is weighted and summarized according to the pre-configured weight table to obtain the comprehensive risk score sequence of each candidate region within the time window. The comprehensive risk score sequence is then smoothed and outlier is removed. Set up multi-level risk thresholds arranged in ascending order of risk level and corresponding uplink / downlink slack parameters. Compare the comprehensive risk score sequence with the multi-level risk thresholds, combine the uplink / downlink slack parameters to complete the jitter determination, and output the risk level label corresponding to each time window. For candidate areas that are determined to reach any level of risk, event objects are generated and written to the local event queue and audit log respectively. Based on the risk level, edge-end audio-visual prompts, switch control or network reporting are triggered, and the corresponding frame image or compressed video clip is output. For candidate areas of generated event objects, continuous tracking and alarm withdrawal determination are performed. When the comprehensive risk score is continuously lower than the alarm withdrawal threshold in consecutive frames and the phase stability is marked as stable and the minimum number of continuous frames is met, an alarm withdrawal record is generated, the closing time and final state of the event object are updated, and the warning output of the area is terminated.

Images