Multi-scene fire early warning and identification method and system

Abstract

The invention discloses a multi-scene fire early warning and identification method and system. The method comprises the following steps: acquiring a fire monitoring video stream, and decoding the video stream into a continuous frame sequence; performing multi-scale preprocessing on the frame sequence; according to the preprocessed frame sequence, fire feature time sequence enhancement processing is carried out, and a standardized feature set D is obtained; performing multi-level feature coding and aggregation on the standardized feature set D to obtain a fire feature vector set; performing fire detection of different scenes according to the fire feature vector set V; wherein the different scenes comprise a closed space, an open space, a semi-open space, a strong light environment, a weak light environment and a normal light environment; in order to solve the problem that early warning cannot be realized due to the fact that existing fire detection is difficult to effectively recognize weak features such as spark splashing and rarefied smoke in the early stage of a fire, the method and the device specially recognize and enhance the features of flames and rarefied smoke in the early stage, fully utilize time sequence information to enhance weak signals and the like, and improve the detection precision of small targets in the early stage of the fire.

Description

Technical Field

[0001] The present application relates to the field of fire detection, and in particular to a multi-scenario fire warning and identification method and system. Background Art

[0002] Fire is one of the most serious disasters threatening people's lives and property. Early detection and early warning are crucial for minimizing losses. With the development of computer vision and artificial intelligence technologies, automatic fire detection systems based on video surveillance have become a research hotspot in the field of fire safety. However, the key to early fire warning is the ability to detect subtle features such as small flames, sparks, and thin smoke, which places extremely high demands on detection technology.

[0003] The existing fire detection technology has the following main technical defects: First, the ability to detect small targets is seriously insufficient. The existing technology is mainly applicable to large and medium-sized fire target data samples, and it is difficult to effectively detect the key features of the early stage of fire. Specifically, (1) for the detection of small flames in the early stage, because the pixel area occupied by them in the image is very small, conventional detection methods can easily ignore them; (2) for the spark splash phenomenon, it is characterized by high-light points and fast movement, and the existing methods lack the ability to capture such instantaneous small targets; (3) for early smoke, it is grayish white, translucent, and slowly diffuses. The existing technology is difficult to accurately identify such low-contrast thin smoke from a complex background. Secondly, the adaptability to multiple scenes is poor. In actual applications, fires may occur in various environments, including closed spaces and open spaces, strong light environments and weak light environments, etc. When facing multi-scene fire detection, the existing technology generally has the problem of high missed detection rate and false detection rate, resulting in low average detection accuracy. In particular, in scenes with complex lighting conditions or special spatial structures, the detection performance drops sharply. In addition, the existing fire detection model based on deep neural networks has the problem of structural rigidity. Its network depth and width cannot be flexibly adjusted, and it is difficult to optimize and deploy it according to the computing power of different hardware devices, which limits the practical application scope of the technology.

[0004] Therefore, there is an urgent need for a fire warning and identification method that can effectively detect the characteristics of small targets in the early stages of fire, adapt to changes in multi-scenario environments, and have good deployability. Summary of the Invention

[0005] In view of the fact that existing fire detection has difficulty in effectively identifying weak features such as sparks and thin smoke in the early stages of a fire, resulting in the inability to achieve early warning, this application provides a multi-scenario fire warning and identification method and system, which improves the accuracy of small target detection in the early stages of a fire by specifically identifying and enhancing the features of initial flames and thin smoke, and making full use of timing information to enhance weak signals.

[0006] One aspect of the present application provides a multi-scenario fire warning and identification method, comprising: obtaining a fire monitoring video stream, decoding the video stream into a continuous frame sequence ; Perform multi-scale preprocessing on the frame sequence; Based on the preprocessed frame sequence, perform fire feature temporal enhancement processing to obtain a standardized feature set D; Among them, fire features include: flame flickering features, smoke diffusion features and flame rising features; Perform multi-level feature encoding and aggregation on the standardized feature set D to obtain a fire feature vector set , where each Corresponding to a detection time window; performing fire detection in different scenarios based on the fire feature vector set V; wherein the different scenarios include: closed space / open space / semi-open space, strong light environment / weak light environment / normal light environment;

[0007] Furthermore, multi-scale preprocessing is performed on the frame sequence, including: Each frame in the image is decomposed into multiple resolutions to obtain a multi-scale image pyramid. ,in, Indicates the i-th level scale The image of degree, the scale factor is ; For each scale image Detect candidate areas of small fire targets by calculating flame color features , smoke transparency characteristics , identifying potential incipient flames and areas of thin smoke ; Adaptively crop and enhance the identified candidate region R to generate an enhanced multi-scale feature set ; Set multi-scale features Normalization is performed to obtain a preprocessed multi-scale frame sequence.

[0008] Furthermore, fire feature temporal enhancement processing is performed based on the preprocessed frame sequence to obtain a standardized feature set D; wherein the fire features include: flame flickering features, smoke diffusion features, including: from the preprocessed multi-scale feature set Extract the time window sequence of continuous T frames to form a time series data set ; According to the time series data set , calculate the brightness time series of each pixel through Fourier transform to generate the flame flickering feature map ; According to the time series data set , the optical flow algorithm is used to calculate the motion vector of each scale feature, extract the vertical upward motion component, and generate the flame rising feature map ; Calculate adjacent frames and The local texture descriptor between them is used to calculate the texture similarity using the gray level co-occurrence matrix ; Through pixel-level difference operation Calculate the transparency change map and mark it as the smoke diffusion area; perform morphological dilation operation on the marked area to obtain the smoke diffusion boundary; calculate the area growth rate and center of mass movement distance of the diffusion area to generate the smoke diffusion feature map ; The original multi-scale features Perform weighted fusion with flame flickering features, smoke diffusion features, and flame rise features to generate time series enhancement features ; Enhanced features for timing Perform normalization to obtain a standardized feature set , where each represents the standardized fire characteristic data of the i-th time window;

[0009] Furthermore, the original multi-scale features Perform weighted fusion with flame flickering features, smoke diffusion features, and flame rise features to generate time series enhancement features : ;in, are weight coefficients respectively.

[0010] Furthermore, the standardized feature set D is multi-level encoded and aggregated to obtain the fire feature vector set , where each Corresponding to a detection time window, including: for each feature data in the standardized feature set D Perform hierarchical feature encoding to obtain local feature maps , Regional feature map , global feature map ; Local feature map , Regional feature map , global feature map The dimension is reduced to the preset size H×W by maximum pooling and average pooling respectively to obtain a set of pooled feature maps ; The pooled feature map set is channel-joined to form a feature tensor , the dimension is [3×2×H×W], where 3 represents three levels and 2 represents two pooling methods; the feature tensors of consecutive k time windows are By combining the sliding window method, the mean, variance and maximum value of the time series features are calculated to generate time series aggregate features. The time series aggregate features are mapped into a feature vector of fixed dimension through the fully connected layer to obtain the feature vector vi of each detection time window.

[0011] Furthermore, for each feature data in the standardized feature set D Perform hierarchical feature encoding, including: using N1×N1 convolution kernel to extract local texture and edge features and generate local feature maps ; Use N2×N2 convolution kernel to extract the shape and structural features of the medium receptive field and generate regional feature maps ; Use N3×N3 convolution kernel to extract large-scale spatial context features and generate global feature maps ; Among them, N3 is greater than N2, and N2 is greater than N1.

[0012] Furthermore, the feature tensors of k consecutive time windows are By combining the sliding window method, the mean, variance and maximum value of the time series features are calculated to generate the time series aggregation features, including: setting the sliding window parameters, the window size is k, the sliding step size is s, and obtaining the time series feature sequence ; Perform channel-by-channel statistical calculations on the k feature tensors in each sliding window to obtain the time series mean features , time series variance characteristics and the time series maximum feature ; Perform spatial pooling on the spatial dimension H×W of the feature tensor to obtain the spatial mean feature ; The time series mean feature , time series variance characteristics and the time series maximum feature , and spatial mean characteristics Splice to obtain time series aggregation features ;

[0013] Furthermore, the k feature tensors in each sliding window are statistically calculated channel by channel, including: calculating the time series mean feature: , j ranges from i to i+k-1 to capture the average intensity of fire characteristics; calculate the time series variance characteristics: , to reflect the dynamic change of fire development; calculate the maximum value characteristics of the time series: , j ranges from i to i+k-1 to capture the peak intensity of fire characteristics;

[0014] Furthermore, fire detection is performed in different scenarios based on the fire feature vector set V; wherein different scenarios include: closed space / open space / semi-open space, strong light environment / weak light environment / normal light environment, including: Each eigenvector in Perform scene recognition: extract ambient light intensity features, and divide the scene into strong light environment, weak light environment, and normal light environment based on the image average brightness value and brightness variance; extract spatial structure features, and divide the scene into closed space, open space, and semi-open space based on edge density and connected domain analysis; adjust the weight coefficient according to the recognized scene 、 and , and adopt different feature enhancement strategies according to different lighting conditions; train the SVM classifier for each scene type separately, and obtain 6 classification functions {f_closed, f_open, f_semi-open, f_strong light, f_weak light, f_normal}; select the corresponding classification function according to the scene recognition results and threshold ; Calculate fire confidence ,when When a fire occurs, it is determined to be a suspected fire; the fire detection results are output, including the fire type, fire location, confidence score and scene type identification.

[0015] Another aspect of the present application also provides a multi-scenario fire warning and identification system, including: a data acquisition module that acquires a fire monitoring video stream and decodes the video stream into a continuous frame sequence ;

[0016] The preprocessing module performs multi-scale preprocessing on the frame sequence, including: multi-resolution decomposition unit, for each frame Perform multi-resolution decomposition to generate a multi-scale image pyramid ; Small target detection unit, based on each layer of scale image , respectively calculate the flame color features and smoke transparency features , identify the candidate areas of initial flames and thin smoke; the feature enhancement unit adaptively crops and enhances the candidate areas to generate a multi-scale feature set ;

[0017] The time series enhancement module performs time series enhancement processing on fire characteristics, including: time series feature unit, which generates flame flicker feature map through Fourier transform , generate flame rising feature map through optical flow algorithm , generate smoke diffusion feature map through texture analysis and morphological processing ; Feature fusion unit, flame flicker feature map , flame rising characteristic diagram , smoke diffusion characteristic diagram Perform weighted fusion to obtain the standardized feature set D;

[0018] Feature aggregation module, which performs multi-level feature encoding and aggregation on the standardized feature set D, including: hierarchical coding unit, using N1×N1, N2×N2, N3×N3 convolution kernels to generate local feature maps respectively , Regional feature map and global feature maps ; Time series aggregation unit, performs maximum pooling and average pooling on the three-layer feature map, generates a set of pooled feature maps, and calculates the time series mean through the sliding window ,variance and maximum value Features, and generate a feature vector of preset dimensions through a fully connected layer ;

[0019] The detection module performs fire detection in different scenes based on the fire feature vector set V, including: a scene recognition unit that identifies the scene type based on the light intensity and spatial structure characteristics; a parameter adjustment unit that adjusts the weight coefficient according to the scene type 、 and , and select the corresponding feature enhancement strategy;

[0020] The classifier group includes 6 SVM classifiers trained for different scenarios;

[0021] The detection module calculates the fire confidence level, performs time sequence consistency verification, and outputs the fire detection results.

[0022] Compared with the existing technology, the advantages of this application are:

[0023] The existing fire detection technology has insufficient small target detection capabilities, especially the difficulty in effectively identifying weak features such as small flames, sparks and thin smoke in the early stages of a fire, which leads to technical defects that make early warning impossible.

[0024] This application first constructs a multi-resolution image pyramid (scale factor ), detects candidate regions of small fire targets at different scale levels, and specifically identifies and enhances initial flames (HSV color space H∈[0°, 60°] and S>0.5) and thin smoke (semi-transparent areas with S<0.3 and brightness change rate>0.2).

[0025] Then, targeting the dynamic characteristics of the early fire features, the flame flicker features of 2 to 10 Hz are extracted through Fourier transform, the flame rising motion of 0.5 to 3.0 m / s is captured through the optical flow algorithm, and the smoke diffusion boundary is detected through texture similarity and transparency changes, making full use of temporal information to enhance weak signals.

[0026] Finally, a layered convolution kernel with sizes of N1×N1 (local), N2×N2 (regional), and N3×N3 (global) is used to extract features from different receptive fields, ensuring that it can capture small point-like targets such as sparks while also identifying regional features such as smoke diffusion. Feature weights and enhancement strategies are dynamically adjusted based on different scenarios (enclosed / open spaces, strong / low light environments). In particular, temporal accumulation enhancement is used to improve the signal-to-noise ratio in low-light environments, while differential enhancement is used in strong light environments to highlight dynamic changes. This effectively enhances the detection of small targets in the early stages of a fire, enabling reliable identification of small flames, sparks, and thin smoke, significantly improving the timeliness and accuracy of fire warnings. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The present application will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same numbers represent the same structures, wherein:

[0028] Figure 1 is an exemplary flow chart of a multi-scenario fire warning and identification method according to some embodiments of the present application;

[0029] Figure 2 is an exemplary flow chart of performing multi-scale processing according to some embodiments of the present application;

[0030] Figure 3 is an exemplary flow chart of constructing a standardized feature set according to some embodiments of the present application;

[0031] Figure 4 is an exemplary flow chart of a feature vector method for constructing a detection time window according to some embodiments of the present application;

[0032] Figure 5 This is an exemplary flowchart of obtaining fire detection results according to some embodiments of the present application. DETAILED DESCRIPTION

[0033] The method and system provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0034] like Figure 1 As shown, obtain the fire monitoring video stream and decode the video stream into a continuous frame sequence ; Perform multi-scale preprocessing on the frame sequence; Based on the preprocessed frame sequence, perform fire feature temporal enhancement processing to obtain a standardized feature set D; Among them, fire features include: flame flickering features, smoke diffusion features and flame rising features; Perform multi-level feature encoding and aggregation on the standardized feature set D to obtain a fire feature vector set , where each Corresponding to a detection time window; performing fire detection in different scenarios according to the fire feature vector set V; wherein the different scenarios include: closed space / open space / semi-open space, strong light environment / weak light environment / normal light environment.

[0035] Specifically, it receives real-time video stream data from monitoring devices. The input format supports RTSP, RTMP, or local video files (MP4, AVI, etc.). The original data format of the video stream is a compressed byte stream using the H.264 / H.265 encoding standard.

[0036] The video stream is read through the Video Capture interface of FFmpeg or Open CV. The decoder decodes the compressed data into raw RGB frame data. Each frame data is represented as a three-dimensional matrix. , where H is the height, W is the width, and 3 is the RGB channel; output continuous frame sequence , the frame rate remains the original video frame rate (usually 25-30fps).

[0037] like Figure 2 As shown, the frame sequence is preprocessed at multiple scales, including: Construct an n-layer Gaussian pyramid: Layer 1: , maintain the original resolution H×W; i-th layer: ; Gaussian filtering: σ = 1.6, kernel size is 5 × 5; downsampling: each dimension is reduced to 1 / 2 of the original; resolution: the i-th layer is ,pyramid , .

[0038] RGB space Convert to HSV space: ;H channel: hue, range [0°, 360°]; S channel: saturation, range [0, 1]; V channel: brightness, range [0, 255].

[0039] By calculating the flame color characteristics : For each pixel (x, y): ;

[0040] Smoke transparency characteristics : Calculate the brightness change rate of adjacent frames: ;

[0041] For each pixel (x, y): .

[0042] right and Perform connected domain analysis, use the 8-neighborhood connectivity labeling algorithm to filter connected domains with an area smaller than a threshold (20 pixels), and output a set of candidate regions. , each represents the bounding box, where Extract the area with hue H∈[0°, 60°] and saturation S>0.5 in the HSV color space. Detect semi-transparent areas with saturation S<0.3 and brightness change rate>0.2;

[0043] Adaptively crop and enhance the identified candidate region R to generate an enhanced multi-scale feature set ; Set multi-scale features Normalization is performed to obtain a preprocessed multi-scale frame sequence.

[0044] In particular, this application constructs a scale factor of The multi-scale image pyramid achieves comprehensive detection of fire targets of varying sizes. While a small, initial flame occupies only a few pixels in the original resolution image, its relative size increases at lower-resolution pyramid levels, increasing the likelihood of detection. Furthermore, multi-scale representation increases the probability of capturing small, transient, point-like targets such as sparks.

[0045] Furthermore, for incipient flames, the specific distribution of flames in the HSV color space (H∈[0°, 60°] and S>0.5) enables accurate identification of flame regions as small as a few pixels. For thin smoke, the detection of semi-transparent regions with low saturation (S<0.3) and a brightness change rate greater than 0.2 overcomes the difficulty of detecting incipient smoke due to its grayish-white, translucent nature. This physical property-based detection method offers greater targeting and sensitivity than traditional general-purpose target detection.

[0046] like Figure 3 As shown, according to the pre-processed frame sequence, the fire feature temporal enhancement processing is performed to obtain the standardized feature set D; wherein the fire features include: flame flickering features, smoke diffusion features, including: from the pre-processed multi-scale feature set Extract the time window sequence of continuous T frames from the dataset, set the time window length T = 20 (adjustable range 10-30), and form a time series data set. ; According to the time series data set , calculate the brightness time series of each pixel through Fourier transform, extract the periodic variation component in the frequency range of 2-10Hz, and generate the flame flicker characteristic map ΔF_flicker(t);

[0047] According to the time series data set , the optical flow algorithm is used to calculate the motion vectors of features at each scale, extract the vertical upward motion component, and mark it as the flame rising feature when the rising speed is within the range of 0.5 - 3.0 m / s, generating a flame rising feature map ;

[0048] Calculate the local texture descriptors between adjacent frames and . Calculate the texture similarity using the gray-level co-occurrence matrix with a 3×3 window . Specifically, for the 3×3 window, calculate the gray-level co-occurrence matrix GLCM in 4 directions (0°, 45°, 90°, 135°): , where d = 1 is the pixel spacing

[0049] Texture feature extraction: Contrast: ; Homogeneity: . Texture similarity: .

[0050] Calculate the transparency change map through pixel-level differential operations . When the change value is within the range of 0.2 - 0.8 and , mark it as the smoke diffusion area; perform morphological dilation operation on the marked area with a kernel size of 5×5 and iterate 2 times to obtain the smoke diffusion boundary; calculate the area growth rate and centroid movement distance of the diffusion area, generating a smoke diffusion feature map ;

[0051] Weightedly fuse the original multi-scale features with the flame flicker feature and the smoke diffusion feature to generate a time series enhanced feature , where are the weight coefficients respectively. The flame flicker weight at the shallow scale is larger, and the smoke diffusion weight at the deep scale is larger;

[0052] The weight coefficient is dynamically adjusted according to different scale levels i: when i ≤ n / 3, set , and preferentially detect the flame flicker feature at the shallow layer; when n / 3 < i ≤ 2n / 3, set , and balance the detection of various fire features in the middle layer; when i > 2n / 3, set , and preferentially detect the smoke diffusion feature at the deep layer;

[0053] Perform data normalization processing on the time series enhanced feature E(t), including calculating the mean μ and standard deviation σ for each feature dimension, through the formula Normalize and map the eigenvalues ​​to the [0, 1] interval to obtain a standardized feature set , where each Represents the standardized fire characteristic data of the i-th time window.

[0054] In particular, small targets in the early stages of a fire often appear as extremely weak signals in a single-frame image, easily drowned out by noise or overlooked. Even tiny flames have stable flickering frequencies. This application transforms small targets, which are difficult to detect in the spatial domain, into strong features in the temporal domain, leveraging the physical characteristics of fire (flickering, diffusion, and rise) for targeted enhancement.

[0055] On the one hand, this application uses Fourier transform to extract this frequency component within a T-frame (10-30 frames) time window, accumulating small, almost invisible flame flickers in a single frame into a distinct periodic signal. By tracking the vertical motion component using an optical flow algorithm, the ascending trajectory of even small, transient targets like sparks can be effectively detected in time series.

[0056] On the other hand, although the transparency of the early thin smoke is extremely low in a single frame (the change value is only 0.2-0.8), its diffusion movement is continuous in time sequence. The joint judgment of transparency change and morphological dilation operation can accurately capture the dynamic diffusion boundary of smoke.

[0057] In addition, this application dynamically adjusts feature weights according to different scale levels: Shallow scale (i≤n / 3): , prioritize flame flicker detection, because small flames have the most obvious flickering characteristics at low resolution; deep scale (i>2n / 3): , giving priority to detecting smoke diffusion, because the large-scale diffusion characteristics of smoke are easier to identify at high resolution. This design ensures that different types of small targets can be effectively enhanced at their most suitable scale level.

[0058] like Figure 4 As shown, the standardized feature set D is multi-level encoded and aggregated to obtain the fire feature vector set , where each Corresponding to a detection time window, including:

[0059] For each feature data in the standardized feature set D Perform hierarchical feature coding, including: Local feature coding: Use 3×3 convolution kernel to extract local texture and edge features to generate local feature maps ; Regional feature encoding: Use 5×5 convolution kernel to extract the shape and structural features of the medium receptive field to generate a regional feature map ; Global feature encoding: Use 7×7 convolution kernel to extract large-scale spatial context features and generate global feature maps ;

[0060] Perform spatial pooling operation on the feature maps of the three levels. 、 and Dimensionality reduction is performed to a fixed size H×W by maximum pooling and average pooling, where H=16 and W=16 respectively;

[0061] The multi-level features after pooling are channel-joined to form a comprehensive feature tensor , the dimension is [3×2×H×W], where 3 represents three levels and 2 represents two pooling methods;

[0062] Set the sliding window parameters, window size k = 5-10 time windows, sliding step s = 1-3 time windows, to form a time series feature sequence ;

[0063] Perform channel-by-channel statistical calculations on the k feature tensors within each sliding window: Calculate the time series mean feature: , j from i to i+k-1, capturing the average intensity of fire characteristics; calculate the time series variance characteristics: , reflecting the dynamic change degree of fire development; calculating the maximum value characteristics of the time series: , j ranges from i to i+k-1, capturing the peak intensity of fire characteristics;

[0064] Perform weighted aggregation on the spatial dimension H×W of the feature tensor: Fire features are spatially distributed with dense centers and sparse edges. The flame core and smoke source are usually located in the center of the image, while the edge areas are mostly thin features after diffusion or background interference. Therefore, this application designs a spatial attention mechanism with high weights at the center and low weights at the edges. For a 16×16 feature map, a weight matrix of the same size is constructed. : Set the center point of the matrix: (For a 16x16 matrix); calculate the distance from each position (i, j) to the center: ; Apply the Gaussian function: , where σ = 4 controls the decay rate. Normalize the Gaussian function value to the range [0, 1] and linearly map it to the target weight range: , ensuring that the center weight is close to 1.0 and the edge weight is close to 0.3. The center area (radius ≤ 4): weight value is 0.8-1.0; the transition area (radius 4-8): weight value is 0.5-0.8; the edge area (radius > 8): weight value is 0.3-0.5. The weight distribution shows a smooth transition to avoid sudden changes. For each spatial position (i, j) of the feature map feature_map, perform element-by-element multiplication: ;symbol Represents the Hadamard product (element-wise multiplication). Sum the weighted eigenvalues ​​of all spatial locations .

[0065] Generate time series aggregate feature vector: Concatenate the time series statistical features (mean, variance, maximum value) with the spatial weighted features to form an aggregate feature vector ; Perform L2 normalization on the aggregated feature vector to ensure feature scale consistency;

[0066] Introducing temporal consistency constraints: Calculating feature similarity between adjacent windows When sim > 0.7, a continuous fire signature is considered, increasing detection confidence. This indicates that a real, ongoing fire, such as the continuous smoke produced by stable combustion, is detected. When sim < 0.3, a sudden scene change or false detection may occur, reducing detection confidence. This could be due to accidental sparks or environmental interference. Lowering the confidence level can help prevent false alarms.

[0067] In particular, among the weak features of the early stage of a fire, thin smoke is a continuous but weak signal, and sparks are instantaneous but obvious signals. This application sets three statistics: time series mean : Specialized to accumulate weak signals from thin smoke. Even if the smoke is almost transparent in a single frame, the signal strength can be increased by 5-10 times by accumulating the average value of 5-10 frames; time series variance : Capture the dynamic characteristics of spark splashes. The sudden appearance and rapid disappearance of spark splashes will produce large variance values, and even very short durations can be detected; time series maximum : Ensure that the peak signal of Mars splash is not diluted by the averaging operation, protecting the instantaneous strong features;

[0068] In addition, this application focuses on the target area through spatial weighting: the design of the central area weight 0.8-1.0 and the edge weight 0.3-0.5 reflects the spatial characteristics of fire development; for spark splashes, they usually appear in the center of the fire source, and the high weight ensures that they are detected first; for smoke diffusion, although it will diffuse to the edge, the concentration in the center is the highest, and the weighted design conforms to the laws of physics.

[0069] The time series aggregation features are mapped into a fixed-dimensional feature vector through the fully connected layer. The dimension is set to 256 and L2 normalized. The processing flow of S41-S45 is repeated for the entire video sequence, and a 256-dimensional feature vector is generated for each detection time window. , and finally obtain the fire feature vector set , where m is the total number of detection windows in the video sequence.

[0070] In particular, the faint features of the early stages of a fire have great differences in spatial distribution: sparks appear as isolated bright spots, while thin smoke has a diffuse regional distribution.

[0071] This application uses hierarchical feature encoding: Local features (3×3 convolution kernel): specially designed to capture small point-like targets such as sparks. Sparks usually only occupy a few pixels, and the small 3×3 receptive field can accurately extract their high-brightness, fast-moving local features without being diluted by the surrounding background. Regional features (5×5 convolution kernel): designed for smoke edges and small flames, capable of capturing the local diffusion pattern and texture changes of thin smoke. Global features (7×7 convolution kernel): used to understand the overall diffusion trend and spatial distribution of smoke, providing contextual information to reduce false detections.

[0072] In addition, this application uses maximum pooling and average pooling in parallel: Maximum pooling: retains highlight point features such as Mars splash, and these peak signals will not be lost even in the 16×16 dimensionality reduction process; Average pooling: retains the overall distribution information of thin smoke, avoiding ignoring diffuse features due to local extreme values.

[0073] like Figure 5 As shown, fire detection is performed in different scenarios based on the fire feature vector set V; wherein the different scenarios include: closed space / open space / semi-open space, strong light environment / weak light environment / normal light environment, including:

[0074] Fire feature vector collection Each eigenvector in Perform scene recognition: extract ambient light intensity features, convert the original RGB image to grayscale space; calculate the average brightness of the entire image: , N is the total number of pixels; calculate the brightness standard deviation: .

[0075] when and When the light is high, it is judged as a strong light environment (such as a scene with direct sunlight);

[0076] when or When the light is low, it is determined to be a low-light environment (such as at night or in a backlit scene);

[0077] when and When , it is determined to be a normal lighting environment;

[0078] Extract spatial structural features, apply the Canny edge detection algorithm to extract edge maps, and calculate the edge pixel ratio: The edge density reflects the structural complexity of the scene. Perform a morphological closing operation on the edge map, connect broken edges, use the connected component labeling algorithm to identify closed contours, and detect whether there are closed areas with an area greater than a threshold:

[0079] when If there is a closed outline, it is considered a closed space (such as indoors or warehouses);

[0080] when When it is located in an open space (such as a field or square),

[0081] when When it is located in a semi-open space (such as a corridor or balcony),

[0082] Adjust the weight coefficient according to the recognized scene type 、 and , select the corresponding fire detection strategy:

[0083] Enclosed Space Scene: Setup ; Smoke accumulation in closed spaces is obvious and diffusion is limited, so the smoke diffusion feature has the highest weight. .

[0084] Open Space Scene: Setup In open environments, flame characteristics are obvious, smoke is easy to spread, and the flame flickering feature has the highest weight. .

[0085] Semi-open space scene: setting , balance the characteristics;

[0086] Strong light environment scene: Based on the original weight, a differential enhancement strategy is used to calculate the enhanced feature vector ; Through time series difference, dynamic changes are highlighted, static strong light background interference is suppressed, and the discrimination of dynamic characteristics of fire is enhanced.

[0087] Low-light environment scene: Based on the original weight, the cumulative enhancement strategy is used to calculate the enhanced feature vector , j from i-k+1 to i; through time series accumulation, the signal-to-noise ratio is improved, the influence of random noise is smoothed, and the stability of weak fire signals is enhanced.

[0088] Normal lighting environment: keep the original weight coefficient unchanged and use the eigenvector directly ;

[0089] Construct a multi-scene fusion classifier for fire determination:

[0090] An independent SVM classifier is trained for each scene type, and six classification functions {f_closed, f_open, f_semi-open, f_strong light, f_weak light, f_normal} are obtained. Among them, the SVM classifier adopts the RBF kernel function.

[0091] Select the corresponding classification function based on the scene recognition results and threshold ; The optimal threshold is determined by ROC curve analysis. Specifically, the specific forms of the six scene classification functions are:

[0092] Closed space classification function f_closed, decision function: f_closed(v): ; Number of support vectors: , which mainly come from the training samples of smoke accumulation.

[0093] Open space classification function f_open, decision function: f_open(v): ; Number of support vectors: The training samples mainly come from the dynamic characteristics of flames.

[0094] Semi-open space classification function f_semi-open, decision function: f_semi-open(v): ; Number of support vectors: A blend of flame and smoke characteristics.

[0095] Strong light environment classification function f_strong light, decision function: f_strong light (v): ; Number of support vectors: The main focus is on learning dynamic difference patterns.

[0096] Low-light environment classification function f_weak light, decision function: f_weak light (v): ; Number of support vectors: More support vectors are needed to process low signal-to-noise ratio data.

[0097] Normal lighting classification function f_normal, decision function: f_normal(v): Number of support vectors: , standard configuration.

[0098] Construct a fusion classification function based on the identified space type and lighting conditions:

[0099] Closed space + strong light environment, f_scene(v) = 0.6×f_closed(v) + 0.4×f_strong light(v), ;

[0100] Closed space + low light environment, f_scene(v) = 0.7×f_closed(v) + 0.3×f_lowlight(v), ;

[0101] Closed space + normal lighting, f_scene(v) = 0.8×f_closed(v) + 0.2×f_normal(v), ;

[0102] Open space + strong light environment, f_scene(v) = 0.5×f_open(v) + 0.5×f_strong light(v), ;

[0103] Open space + low light environment, f_scene(v) = 0.7×f_open(v) + 0.3×f_lowlight(v), ;

[0104] Open space + normal lighting, f_scene(v) = 0.75×f_open(v) + 0.25×f_normal(v), ;

[0105] Semi-open space + strong light environment, f_scene(v) = 0.55×f_semi-open(v) + 0.45×f_strong light(v), ;

[0106] Semi-open space + low-light environment, f_scene(v) = 0.65×f_semi-open(v) + 0.35×f_low-light(v), ;

[0107] Semi-open space + normal lighting, f_scene(v) = 0.7×f_semi-open(v) + 0.3×f_normal(v), .

[0108] Calculating fire confidence ,when When the fire is detected, it is determined to be a suspected fire;

[0109] Introducing timing consistency verification, maintaining a detection queue of length 5, and confirming a real fire when there are more than 3 suspected fire detection results in the queue;

[0110] Output fire detection results, including fire type, fire location, confidence score, and scene type identification.

[0111] In particular, sparks and thin smoke in the early stages of a fire present distinct visual characteristics in different scenarios. This application sets different detection strategies for different scenarios:

[0112] Low light environment ( or ) cumulative enhancement strategy: In low-light conditions, although the splash of Mars is relatively bright, its absolute brightness value is low and it is easily drowned by noise; thin smoke is almost invisible against a dark background and has extremely low contrast; through cumulative enhancement , the weak signals of consecutive k frames are superimposed, and the signal-to-noise ratio is improved by k times.

[0113] Strong light environment ( and ) differential enhancement strategy: under strong light, the background brightness is high, and the relative brightness difference of the Mars splash is compressed; the thin smoke appears as a faint shadow change under strong light; through differential enhancement , highlighting the characteristics of temporal changes; even if the static features are not obvious, dynamic changes can be effectively captured.

[0114] Enclosed space ( )'s smoke priority strategy: Set (Smoke has the highest weight) because smoke easily accumulates in enclosed spaces, and even extremely thin smoke can form a detectable concentration gradient; spark splashes have enhanced reflections in enclosed spaces and are more easily captured by basic detection, so That's it.

[0115] Open Space ( )'s flame priority strategy: Set (Flame flicker has the highest weight), (The weight of upward movement is second); smoke in open space quickly diffuses and dilutes, making it difficult to form a detectable concentration, so ;The focus is on detecting the flashing and rising characteristics of Mars splashes.

[0116] Six independent SVM classifiers are trained, each optimized for the distribution of faint features in a specific scenario: the low-light environment classifier is more sensitive to low-brightness signals and has a correspondingly lower threshold; the enclosed space classifier assigns higher weight to slowly changing smoke features; and the bright light environment classifier focuses on learning dynamic differential features. For the same spark splash, cumulative enhancement improves visibility in low light, while differential enhancement highlights dynamic characteristics in bright light. For the same thin smoke, higher weighting prioritizes detection in enclosed spaces, while lower weighting prevents false positives in open spaces.

[0117] The invention of the present application and its implementation methods are described schematically above. This description is not restrictive. Without departing from the spirit or basic features of the present application, the present application can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention of the present application, and the actual structure is not limited to this. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the invention, a structural method and embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of the present application. In addition, the word "including" does not exclude other elements or steps, and the word "one" before an element does not exclude the inclusion of "multiple" elements. Words such as first and second are used to indicate names and do not indicate any specific order.

Claims

1. A multi-scenario fire warning and identification method, characterized in that: include: Obtain fire monitoring video stream and decode the video stream into a continuous frame sequence ; Perform multi-scale preprocessing on the frame sequence; According to the pre-processed frame sequence, fire feature temporal enhancement processing is performed to obtain a standardized feature set D; wherein the fire features include: flame flickering features, smoke diffusion features and flame rising features; Perform multi-level feature encoding and aggregation on the standardized feature set D to obtain the fire feature vector set , where each Corresponding to a detection time window; Fire detection is performed in different scenarios based on the fire feature vector set V; wherein the different scenarios include: closed space / open space / semi-open space, strong light environment / weak light environment / normal light environment.

2. The multi-scenario fire warning and identification method according to claim 1, characterized in that: Perform multi-scale preprocessing on the frame sequence, including: For a continuous frame sequence Each frame in the image is decomposed into multiple resolutions to obtain a multi-scale image pyramid. ,in, Indicates the i-th level scale The image of degree, the scale factor is ; For each scale image Detect candidate areas of small fire targets by calculating flame color features , smoke transparency characteristics , identifying potential incipient flames and areas of thin smoke ; Adaptively crop and enhance the identified candidate region R to generate an enhanced multi-scale feature set ; The multi-scale feature set Normalization is performed to obtain a preprocessed multi-scale frame sequence.

3. The multi-scenario fire warning and identification method according to claim 2, characterized in that: According to the pre-processed frame sequence, the fire feature temporal enhancement processing is performed to obtain the standardized feature set D, which includes: From the preprocessed multi-scale feature set Extract the time window sequence of continuous T frames to form a time series data set ; According to the time series data collection , calculate the brightness time series of each pixel through Fourier transform to generate the flame flicker feature map ; According to the time series data collection , the optical flow algorithm is used to calculate the motion vector of each scale feature, extract the vertical upward motion component, and generate the flame rising feature map ; Calculate adjacent frames and The local texture descriptor between them is used to calculate the texture similarity using the gray level co-occurrence matrix ; Through pixel-level difference operation Calculate the transparency change map and mark the smoke diffusion area; Perform morphological dilation operation on the marked area to obtain the smoke diffusion boundary; Calculate the area growth rate and centroid moving distance of the diffusion area to generate the smoke diffusion characteristic map ; The original multi-scale features Perform weighted fusion with flame flickering features, smoke diffusion features, and flame rise features to generate time series enhancement features ; Timing Enhancement Features Perform normalization to obtain a standardized feature set , where each Represents the standardized fire characteristic data of the i-th time window.

4. The multi-scenario fire warning and identification method according to claim 3, characterized in that: Generate timing enhancement features : ;in, are weight coefficients respectively.

5. The multi-scenario fire warning and identification method according to claim 3, characterized in that: Get the fire feature vector set ,include: For each feature data in the standardized feature set D Perform hierarchical feature encoding to obtain local feature maps , Regional feature map , global feature map ; The local feature map , Regional feature map , global feature map The dimension is reduced to the preset size H×W by maximum pooling and average pooling respectively to obtain a set of pooled feature maps ; The pooled feature map set is channel-joined to form a feature tensor , the dimension is [3×2×H×W], where 3 represents three levels and 2 represents two pooling methods; The feature tensors of k consecutive time windows are By combining the sliding windows, the mean, variance, and maximum of the time series features are calculated to generate time series aggregate features. The time series aggregation features are mapped into a fixed-dimensional feature vector through the fully connected layer to obtain the feature vector of each detection time window .

6. The multi-scenario fire warning and identification method according to claim 5, characterized in that: For each feature data in the standardized feature set D Perform hierarchical feature encoding, including: Use N1×N1 convolution kernel to extract local texture and edge features and generate local feature maps ; Use N2×N2 convolution kernel to extract the shape and structural features of the medium receptive field and generate regional feature maps ; Use N3×N3 convolution kernel to extract large-scale spatial context features and generate global feature maps ; Among them, N3 is greater than N2, and N2 is greater than N1.

7. The multi-scenario fire warning and identification method according to claim 5, characterized in that: Generate time series aggregation features, including: Set the sliding window parameters, the window size is k, the sliding step is s, and the time series feature sequence is obtained ； For the k feature tensors in each sliding window, perform channel-by-channel statistical calculations to obtain the time series mean features , time series variance characteristics and the time series maximum feature ; Perform spatial pooling on the spatial dimension H×W of the feature tensor to obtain the spatial mean feature ; The time series mean feature , time series variance characteristics and the time series maximum feature , and spatial mean characteristics Splice to obtain time series aggregation features .

8. The multi-scenario fire warning and identification method according to claim 7, characterized in that: For each k feature tensor in the sliding window, channel-by-channel statistical calculations are performed, including: Calculate the time series mean characteristics: , j ranges from i to i+k-1 to capture the average intensity of fire characteristics; Calculate the time series variance characteristics: , to reflect the dynamic change degree of fire development; Calculate the maximum feature of the time series: , j ranges from i to i+k-1 to capture the peak intensity of fire characteristics.

9. The multi-scenario fire warning and identification method according to claim 7, characterized in that: Fire detection in different scenarios is performed based on the fire feature vector set V, including: Fire feature vector collection Each eigenvector in Perform scene recognition: Extract the ambient light intensity features and divide the scene into strong light environment, weak light environment and normal light environment according to the average brightness value and brightness variance of the image; Extract spatial structural features and divide the scene into closed space, open space and semi-open space based on edge density and connected domain analysis; Adjust the weight coefficient according to the recognized scene 、 and , and adopt different feature enhancement strategies according to different lighting conditions; Train the SVM classifier for each scene type separately and obtain 6 classification functions {f_closed, f_open, f_semi-open, f_strong light, f_weak light, f_normal}; Select the corresponding classification function based on the scene recognition results and threshold ; Calculating fire confidence ,when When the fire is detected, it is determined to be a suspected fire; Output fire detection results, including fire type, fire location, confidence score, and scene type identification.

10. A multi-scenario fire warning and identification system, characterized in that: include: Data acquisition module, which acquires fire monitoring video stream and decodes the video stream into a continuous frame sequence ; Preprocessing module, performs multi-scale preprocessing on the frame sequence; Time series enhancement module, which performs time series enhancement processing on fire characteristics; Feature aggregation module, which performs multi-level feature encoding and aggregation on the standardized feature set D; The detection module performs fire detection in different scenarios based on the fire feature vector set V; The classifier group includes 6 SVM classifiers trained for different scenarios; The detection module calculates the fire confidence level and performs timing consistency verification, and outputs the fire detection results.

Images