Intelligent fire detection method based on artificial intelligence video analysis

Abstract

The application discloses a fire intelligent detection method based on artificial intelligence video analysis, and particularly relates to the technical field of computer vision; a video monitoring stream is collected, spatial color features and morphological features of images are extracted, and a multi-scale time sequence feature map group is constructed; a double-channel attention neural network is used to jointly model color disturbance and structure disturbance, and output a suspected fire area probability graph; candidate fire source areas are screened in combination with boundary disturbance consistency analysis and red channel high-frequency fluctuation detection; a multi-modal fusion discrimination model is used to comprehensively consider texture stability, disturbance directionality and historical smoke evolution features, so that a fire state is finally judged and an alarm result is output; simultaneously, the system can mark a fire starting position and a development trend path, and visual tracking is realized. The method has high robustness, high sensitivity and low false alarm rate, and is suitable for early fire intelligent early warning in complex environments.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a fire intelligent detection method based on artificial intelligence video analysis. BACKGROUND

[0002] With the improvement of automation and unmanned level in closed or semi-closed environments such as urban underground space, warehouse logistics center, and large-scale unmanned logistics park, the traditional fire detection technology relying on smoke and temperature sensors gradually exposes many limitations in practical application. For example, in the unattended closed scene, smoke is difficult to diffuse to the sensor sampling point in time, and false alarms are prone to occur in high-temperature environments (such as industrial drying and logistics oven), causing unnecessary production stoppage and resource waste.

[0003] In recent years, the rapid development of artificial intelligence image recognition technology makes it possible to identify fire based on video. However, most of the existing fire identification methods based on video only rely on convolutional neural network models to classify image features, lack modeling of the dynamic fire evolution process in the time dimension, and it is especially difficult to identify weak open flames and subtle smoke in the initial flame stage, which is the key early warning opportunity before fire spreads. In addition, there are complex factors such as strong light interference, mechanical motion artifacts, and hot gas disturbance in the video, which makes the traditional image algorithm have high false detection rate and large missed detection rate in early identification, and it is difficult to meet the high reliability requirements of industrial applications. Especially in special scenes such as automatic stereoscopic warehouse, logistics conveyor fire source monitoring, and unmanned power distribution room, the existing technology has almost no effective solution. SUMMARY

[0004] The purpose of the present application is to provide a fire intelligent detection method based on artificial intelligence video analysis to solve the problems in the background art.

[0005] In order to achieve the above purpose, the present application provides the following technical scheme: a fire intelligent detection method based on artificial intelligence video analysis, comprising:

[0006] S100: collecting video monitoring stream of a target area, extracting spatial color feature vector C and morphological feature vector M of each frame of image;

[0007] S200: performing time sequence difference analysis on consecutive frames of images, obtaining change amplitude vectors AC and AM based on C and M, and establishing an initial dynamic feature set D;

[0008] S300: constructing a multi-scale time sequence feature map group T, which is composed of three-dimensional tensors formed by D of consecutive N frames of images under different time windows;

[0009] S400: Time sequence feature extraction is performed on T by using a dual-channel neural network model based on an attention mechanism, and a suspected fire area probability map P is output.

[0010] S500: Boundary stability analysis is performed on the suspected area in P, it is judged whether there is a boundary sequence S with flame shape disturbance characteristics, and a disturbance consistency coefficient R is calculated.

[0011] S600: If R exceeds a set threshold, a candidate fire source marking area F is generated by further combining the high-frequency fluctuation characteristics of the red channel in C and AC.

[0012] S700: The F is input into a fire multi-modal fusion discrimination model, the texture stability, disturbance directionality and historical smoke evolution model of the F area under different illumination channels are fused, and a final fire judgment result L is output.

[0013] S800: If L determines that the fire is established, an alarm signal is sent, and the fire starting position and development trend path in the video frame are marked.

[0014] Preferably, the S200 comprises:

[0015] S201: Inter-frame registration processing is performed on a plurality of continuous image frames in a preset time window;

[0016] S202: Based on the registered image frame sequence, frame-by-frame difference operation is performed on the spatial color feature vector C and the shape feature vector M of each image, respectively, to calculate the color change amplitude AC and the shape change amplitude AM between adjacent frames, and the difference operation includes Euclidean distance or structural similarity calculation;

[0017] S203: The AC and AM are subjected to filtering and smoothing processing;

[0018] S204: The smoothed AC and AM are combined to construct a dynamic feature set D, which is used to describe the evolution characteristics of the suspected flame or smoke area in the video frame sequence.

[0019] Preferably, the S300 comprises:

[0020] S301: A plurality of time window scale groups are set, n is the total number of windows, each time window corresponds to a frame number interval, and is used to extract image sequence features under different time lengths from the dynamic feature set D;

[0021] S302: For each time window, a feature map sequence of continuous frames is intercepted from the dynamic feature set D, and the sequence is stacked in time sequence to form a primary three-dimensional tensor Ti, whose size is Wi×H×W, wherein H and W are the image height and width;

[0022] S303: performing time series normalization processing on each primary three-dimensional tensor Ti to generate a time series normalized tensor Ti';

[0023] S304: combining the time series normalized tensors Ti' under all scales to form a multi-scale time series feature map group T, which is used to depict the dynamic evolution characteristics of the fire at different time granularities.

[0024] Preferably, the S400 comprises:

[0025] S401: constructing a dual-channel neural network model with a parallel structure, which respectively receives color dynamic information and morphological dynamic information in the multi-scale time series feature map group T, and independently extracts corresponding time series spatial feature tensors;

[0026] S402: performing key frame weight calculation on the input three-dimensional tensor in the time dimension within each channel;

[0027] S403: performing channel fusion processing on the feature tensors output by the two channels to generate a fused multi-dimensional fire feature map;

[0028] S404: inputting the fused feature map into a spatial saliency decoding network, and gradually restoring to a suspected fire area probability map P with the same size as the original image by using convolutional layers and deconvolutional layers, wherein each pixel point value of the P represents the confidence probability that the position is a fire area.

[0029] Preferably, the S500 comprises:

[0030] S501: performing binaryzation processing on the continuous multi-frame suspected fire area probability map P, extracting the region boundary with a confidence higher than a first threshold, and extracting a boundary contour set B t ;

[0031] S502: performing inter-frame matching on the boundary contour set B t in the time sequence, constructing a boundary correspondence relationship based on the contour geometric barycenter position, boundary shape similarity, and area change ratio, and forming a candidate boundary sequence S;

[0032] S503: quantifying the disturbance behavior of each boundary in the boundary sequence S to construct a disturbance feature vector group, including a boundary deformation rate, a barycenter drift vector, and a local edge irregularity index;

[0033] S504: calculating a disturbance consistency coefficient R based on the disturbance feature vector group, wherein the R is a standard deviation normalization result of the disturbance features in the boundary change process over time, and if the R is less than a second threshold, it is determined that the boundary sequence has flame disturbance consistency.

[0034] Preferably, the S600 comprises:

[0035] S601: Extract the spatial color feature C of each frame from the image region corresponding to the boundary sequence S, and focus on extracting the pixel distribution of the red channel value when the disturbance consistency coefficient R exceeds the set threshold.

[0036] S602: Perform frequency domain analysis on the time sequence of the red channel in the continuous frame image, extract the high-frequency component of the red channel, and construct the red high-frequency energy spectrum E.

[0037] S603: Analyze the spatial distribution of the high-frequency energy spectrum E in the region corresponding to the boundary sequence, identify the high-frequency fluctuation aggregation region, and filter out the red high-frequency abnormal region R_high.

[0038] S604: Perform spatial overlap calculation on the boundary sequence S and R_high, and extract the intersection region as the candidate fire source marker region F.

[0039] Preferably, the S700 includes:

[0040] S701: Extract the texture stability index of the candidate fire source marker region F under multiple illumination channels, and calculate the local texture consistency score of F in the original image, grayscale image, and gamma correction image, respectively.

[0041] S702: Based on the contour deformation trajectory of the F region in the continuous frame, construct a disturbance directionality vector group, obtain the main disturbance direction through principal component analysis, and calculate the direction disturbance deviation value.

[0042] S703: Call the historical smoke evolution model to perform time reverse feature comparison on the F region, evaluate whether there is a continuous smoke diffusion process in the surrounding region of F, and output a matching score.

[0043] S704: Input the texture stability, disturbance directionality, and historical smoke matching score as input into the fire multi-modal fusion discrimination model for joint decision, and output the final fire judgment result L, which is a binary result or a confidence score.

[0044] Preferably, the S800 includes:

[0045] S801: When the fire judgment result L is true, generate an alarm time, a target region position, and an alarm level, and send an alarm signal.

[0046] S802: From the time point determined as a fire, backtrack the continuous video frames, extract the geometric center position of the candidate fire source marker region F in each frame image, and construct a fire starting path trajectory P_start.

[0047] ​S803: In the continuous frames after the fire occurs, the area expansion, boundary movement trend and center drift direction of the F region are dynamically tracked, and a fire expansion path trajectory P_expand is calculated;

[0048] S804: P_start and P_expand are spliced to generate a fire development trend path P_total, and the fire source starting point, movement direction and development trend are marked in the alarm video frame with a trajectory line.

[0049] In the above technical solution, the technical effects and advantages provided by the present application are as follows:

[0050] 1. The present application provides a fire intelligent detection method based on artificial intelligence video analysis, which can effectively break through the limitations of traditional dependence on smoke sensors or static image recognition algorithms, and combines multi-scale time sequence feature extraction, boundary disturbance modeling, color frequency domain analysis and multi-modal intelligent discrimination mechanism, and has strong early fire source recognition capability. By introducing a double-channel neural network structure and an attention mechanism, the present application can realize high-confidence determination of the fire area in a complex background, high interference or weak flame stage, greatly reducing the false alarm rate and the missed detection rate, and significantly improving the adaptability and reliability of the system in unmanned and high-risk scenarios.

[0051] 2. The present application combines texture stability analysis, disturbance direction modeling and historical smoke evolution trajectory backtracking to build a global-local combined fire behavior modeling mechanism, which not only realizes intelligent judgment of the fire standing state, but also can output the fire starting position, diffusion trend and complete trajectory clues in real time, providing visual decision basis for fire dispatching, emergency response and accident backtracking. The overall scheme has the advantages of high intelligence, flexible deployment and strong compatibility, and is especially suitable for smart cities, industrial parks, underground facilities, logistics and warehousing and other complex scenes with high requirements for fire warning accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0053] Figure 1 The method flowchart of the present application. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] For examples, please refer to Figure 1 As shown in this embodiment, the intelligent fire detection method based on artificial intelligence video analysis includes:

[0056] S100: Acquires video surveillance stream of the target area and extracts the spatial color feature vector C and morphological feature vector M of each frame image;

[0057] S200: Perform temporal difference analysis on consecutive frame images to obtain the change magnitude vectors ΔC and ΔM based on C and M, and establish an initial dynamic feature set D;

[0058] S300: Construct a multi-scale temporal feature map group T, wherein T is composed of a three-dimensional tensor formed by the D of N consecutive frames of images under different time windows;

[0059] S400: Use a dual-channel neural network model based on attention mechanism to extract temporal features from T and output a probability map P of suspected fire areas;

[0060] S500: Perform boundary stability analysis on the suspected region in P to determine whether there is a boundary sequence S with flame-shaped perturbation characteristics, and calculate its perturbation consistency coefficient R.

[0061] S600: If R exceeds the set threshold, further combine the high-frequency fluctuation characteristics of the red channel in C and ΔC to generate candidate fire source marking regions F;

[0062] S700: Input F into the fire multimodal fusion discrimination model, fuse the texture stability, perturbation directionality and historical smoke evolution model of region F under different lighting channels, and output the final fire judgment result L;

[0063] S800: If L determines that a fire has occurred, an alarm signal is sent and the fire's starting position and development path are marked in the video frame.

[0064] In this embodiment, step S100 aims to extract basic image feature information from the video source for subsequent fire detection analysis, serving as input data for the entire intelligent detection process. Specifically, it includes the following steps:

[0065] Firstly, real-time or stored video monitoring stream is acquired from the set target monitoring area, the video monitoring stream being a continuous frame image sequence, the frame rate range being configurable according to the actual application scene, preferably between 15-30 frames / second, to ensure the integrity and real-time of the timing information.

[0066] Next, pre-processing operation is performed on each frame image in the video monitoring stream, the pre-processing including image normalization, noise suppression, brightness balance, resolution unification and the like, to eliminate the interference of different device acquisition conditions on the image quality and ensure the accuracy of subsequent feature extraction.

[0067] After the pre-processing is completed, two types of core feature information are extracted from each frame image:

[0068] Spatial color feature vector C: the C is a vector set representing the color distribution of each pixel point in the image, preferably using HSV (Hue-Saturation-Value) color space representation, wherein H represents hue component, S represents saturation component, and V represents brightness component. The color feature can effectively reflect the typical red-orange fluctuation characteristics of the flame region in the image, and has good adaptability to different lighting conditions. Optionally, to enhance the stability of color recognition, C can also include the mean value and high-order statistical features of the red channel (R) in the RGB space, such as skewness and kurtosis.

[0069] Morphological feature vector M: the M is a feature set describing the structural morphological changes of the image, preferably using edge detection operators (such as Canny or Sobel) to obtain the edge contour information of the image, and combining the image gradient direction and texture density to calculate the morphological features. The feature is used to assist in identifying the jumping boundary and irregular disturbance behavior of the flame, and can provide structural stability analysis basis, especially in low brightness or smoke shielding scenes.

[0070] In an embodiment of the present application, step S200 is used to extract spatial color feature changes and morphological structural changes from the continuous frame images, to establish an initial dynamic feature set D reflecting the dynamic development of the fire, and to provide data support for subsequent time series modeling and intelligent discrimination. The step specifically includes the following sub-steps:

[0071] S201: performing inter-frame registration processing on the continuous multiple frame images within the preset time window.

[0072] Preferably, the dense optical flow method (Dense Optical Flow) is used to estimate the motion vector of all pixel points in the adjacent image frames, and the subsequent frames are inversely resampled to align with the reference frame. Taking the Farneback algorithm as an example, the core idea of the dense optical flow method is to estimate the pixel movement trend by expanding the polynomial in the image window, and then generate the optical flow map between the continuous frames.

[0073] In a specific implementation, the tthframe image is the reference frame and the t+1thframe image is the frame to be registered. The offset vector (dx, dy) between the two frames is calculated for each pixel point (x, y). An affine transformation or perspective transformation is performed on the t+1thframe image according to the vector, and the image alignment processing is completed.

[0074] S202: Based on the registered image frame sequence, frame-by-frame difference operation is performed on the spatial color feature vector C and the morphological feature vector M of each frame image, respectively, to calculate the color change amplitude ΔC and the morphological change amplitude ΔM between adjacent frames. The difference method includes Euclidean distance or structural similarity calculation.

[0075] After the image frame registration is completed, the spatial color feature vector C and the morphological feature vector M of each frame image are extracted, respectively, and the difference calculation is performed on the feature vectors between adjacent frames to obtain the color change amplitude ΔC and the morphological change amplitude ΔM.

[0076] Specifically, the color change amplitude ΔC is calculated as follows:

[0077] Let the spatial color feature of the tthframe image be and the t+1thframe image be Then The distance function preferably adopts a weighted Euclidean distance, which has the form:

[0078] ,

[0079] where H, S, and V are the components of the HSV color space, and the weighted coefficients are empirically set to 0.4, 0.3, and 0.3.

[0080] Similarly, the morphological change amplitude ΔM is calculated based on the structural similarity (SSIM) of the image edge gradient: Let the edge map of the tthframe image be and the t+1thframe image be Then ,

[0081] where SSIM is the structural similarity index, and its value ranges from 0 to 1. The smaller the value, the greater the difference. This processing method ensures that the key dynamic information of the flame and smoke changing over time is extracted.

[0082] S203: To suppress the feature errors caused by short-time noise, light jitter, or slight background disturbance, time smoothing processing is required for ΔC and ΔM.

[0083] Preferably, a weighted average method based on a sliding window is used. The specific method is as follows:

[0084] Let the sliding window length be k, then the smoothed ΔC at the t time is:

[0085]

[0086] wherein is a weighting coefficient;

[0087] The weighting coefficient can be set according to a time increasing function, preferably in the form of exponential decay, such as: wherein α ∈ (0, 1), and preferably the value is 0.2.

[0088] Correspondingly, the same sliding weighted average processing is also performed on ΔM to generate the smoothed ΔM'.

[0089] S204: Jointly encode the smoothed color change vector ΔC' and the shape change vector ΔM' to form an initial dynamic feature set D.

[0090] In a specific implementation: the dynamic feature set D is a sequence of two-dimensional feature maps, wherein each frame D t is a set of numerical matrices representing the distribution of ΔC' and ΔM' in the image space; ΔC' and ΔM' can be proportionally superimposed and encoded by pixel-level fusion,

[0091] For example: wherein λ is a fusion weight factor, and preferably set to 0.6.

[0092] In an embodiment of the present application, step S300 adopts a multi-scale modeling idea to stack the aforementioned dynamic feature set D in time sequence to build a multi-scale time sequence feature map group T in order to enhance the dynamic modeling ability in time dimension for the fire occurrence process. The process includes the following steps:

[0093] S301: Specifically, set a time window scale group {W1, W2,..., Wn}, n is the total number of windows, wherein each Wi represents a time window length, and the unit is frame number. Preferably, W1= 3 (short time window), W2= 5 (medium time window), and W3= 7 (long time window) can be set; each time scale Wᵢ corresponds to a window that will be used to extract the image feature sequence of consecutive Wᵢ frames from the dynamic feature set D for subsequent construction of a three-dimensional tensor.

[0094] S302: For each time window Wi, the feature map sequence of the consecutive Wi frames at the current time from the dynamic feature set D is obtained, denoted as ​​Each frame D in the sequence is a two-dimensional matrix, representing the combined result of the smoothed color change amplitude ΔC and the shape change amplitude ΔM. Stacking this Wi-frame sequence in chronological order on the first dimension of a tensor forms a three-dimensional tensor. Its shape is: Where H represents the image height and W represents the image width. Let R be the number of time frames and R be the set of real numbers. This three-dimensional tensor reflects the dynamic changes at this time scale and is an important foundation for time series modeling.

[0095] S303: Because the degree of change of image frames varies in different time periods, the tensor obtained by direct stacking may have an imbalance in time components. Therefore, the tensor needs to be... Temporal normalization and attention enhancement processes are performed to improve the saliency of key temporal features. This process consists of two parts:

[0096] Inter-frame difference normalization processing: for tensors Any two adjacent frames and Calculate the difference Then for all Normalization ensures that the difference ranges between 0 and 1. Min-max normalization can be used as a normalization method. Where min and max are respectively The minimum and maximum values ​​in.

[0097] Enhanced Channel Attention Mechanism: To highlight keyframe information in the temporal dimension of the tensor, a channel attention mechanism, such as the Squeeze-and-Excitation (SE) module, is introduced.

[0098] The implementation steps are as follows: For tensors Perform global average pooling in the spatial dimension to obtain... A 3D vector; this vector is compressed and activated through a two-layer fully connected network, outputting a weighted coefficient vector. Use this coefficient to Frame-by-frame weighting is performed along the time dimension, meaning each frame is multiplied by its corresponding attention weight. The resulting output tensor This will highlight important frames in the time sequence and suppress background noise interference.

[0099] S304: Tensor normalized across all scales Joint encoding is performed to form the final multi-scale temporal feature map group T.

[0100] The specific method is as follows: for all scales Concatenate in the "time dimension" or "channel dimension" to obtain a tensor input with a unified shape, which is used for subsequent deep learning network models, such as 3D convolutional neural network (3D-CNN) or convolutional long short-term memory network (ConvLSTM) model. The final multi-scale feature map group T can comprehensively reflect the dynamic evolution characteristics of the flame under different time lengths, including: flame profile change frequency; smoke diffusion directionality; brightness intensity periodicity.

[0101] In the embodiment, step S400 is used for deep learning processing of the aforementioned constructed multi-scale time sequence feature map group T, high confidence recognition of the fire area is realized by introducing an attention mechanism and a double-channel structure, and finally a probability map P of the suspected fire area is output. The specific steps are as follows:

[0102] S401: This step constructs a double-channel convolutional neural network model with a parallel structure, which includes:

[0103] Color dynamic channel: used for receiving and processing input containing color change characteristics (such as ΔC, HSV channel change, etc.) in the feature map group T; morphological dynamic channel: used for processing feature input containing morphological change (such as edge disturbance, texture jump) (such as ΔM, gradient graph, etc.).

[0104] Each channel internally includes a feature extraction subnetwork composed of a stack of 3D convolutional layers (3D-CNN). 3D convolution can simultaneously extract spatial (image structure) and temporal (inter-frame evolution) features, which is an ideal choice for analyzing dynamic events of fire type.

[0105] Taking the color channel as an example, the model structure is as follows: input size: , indicating that the number of time frames is n, and the image size is HxW; the first layer: the 3D convolution kernel size is 3x3x3, the step is 1, and the number of channels is 32; the second layer: 3D convolution + batch normalization + ReLU activation; the third layer: time dimension pooling (MaxPooling, kernel=2x1x1) reduces the time sequence dimension. The morphological channel structure is the same as the color channel, except that the input feature map type is different. After independent processing of the two channels, each outputs a time sequence feature tensor, respectively denoted as F_c (color) and F_m (morphology).

[0106] S402: In order to enhance the response of the model to key frames (such as initial flame frames or smoke sudden disturbance frames) in the time dimension, a lightweight time sequence self-attention mechanism module is introduced in each channel. This module adopts the TemporalSqueeze-and-Excitation (TSE) principle, which specifically includes:

[0107] Temporal dimension compression (Squeeze): Spatial average pooling is performed on each channel feature tensor to obtain a temporal vector ; ;

[0108] Attention weight calculation (Excitation): The V input is input into a two-layer fully connected network, and a temporal attention vector A ∈ R n is output, which is normalized by a Sigmoid function and represents the importance weight of each frame; Weighted enhancement (Recalibration): A is applied to the original feature tensor F, that is, , to enhance the features of high importance frames. This mechanism can make the network actively focus on key frames with typical fire dynamic performance, and improve the recognition sensitivity.

[0109] S403: After completing the dual-channel feature extraction, the color channel output F_c' and the morphology channel output F_m' are fused to generate a joint multi-dimensional fire feature map F_joint.

[0110] Preferably, the fusion method is feature-level weighted splicing fusion, which specifically includes:

[0111] The feature tensors of the two channels are spliced in the channel dimension to obtain a tensor F_concat;

[0112] F_concat is input into a channel attention module (Channel Attention, such as CBAM) to automatically learn the importance weight of each channel;

[0113] The output fused tensor is , where c is the number of fused channels.

[0114] The fused F_joint has the expression ability of color disturbance features and morphology disturbance features, and provides rich semantic information for subsequent spatial positioning.

[0115] S404: The fused feature map F_joint is input into a spatial saliency decoding network to gradually restore the spatial resolution and generate a probability map P consistent with the size of the original image.

[0116] The decoding network adopts a U-Net style upsampling structure, which mainly includes:

[0117] Convolution layer group: semantic integration is performed using multiple 3x3 convolution+ReLU structures;

[0118] Deconvolution layer group (Deconvolution): gradually restore image size (such as upsample 2x);

[0119] Sigmoid activation layer: output single-channel probability map where each pixel value P(i, j) e [0, 1] represents the probability confidence that the corresponding image location is a fire region.

[0120] In a preferred embodiment of the present application, in order to further determine whether there is a region with typical flame disturbance behavior in the suspected fire region probability map P output by the neural network, step S500 introduces a boundary stability analysis mechanism, and realizes the structured recognition of the flame dynamic characteristics by calculating the boundary disturbance consistency coefficient R. Specifically, the following steps are included:

[0121] S501: This step first performs region extraction processing on the continuous multiple frames of suspected fire region probability map P, so as to obtain the spatial boundary of the high-confidence suspected region.

[0122] The specific steps are as follows: for each frame of probability map P , threshold segmentation processing is performed, and a first confidence threshold is set, which is preferably 0.6. The pixel points satisfying are regarded as fire region pixels to form a binary image .

[0123] On the binary image , a contour extraction algorithm is performed, and the findContours function in OpenC is preferably used to extract all closed or semi-closed boundaries to obtain a boundary contour set , where each b represents a boundary curve of a connected region; for each boundary , its boundary coordinate point set, geometric center of gravity, contour length and area are recorded for subsequent matching analysis. After processing, the boundary contour sequence in the continuous several frames of images can be obtained, which lays the foundation for subsequent disturbance analysis.

[0124] S502: Since the flame boundary is a non-rigid target, the boundary shape changes over time, so this step pairs the boundary contours between consecutive frames by a structural similarity algorithm to construct a boundary sequence S.

[0125] The matching process is as follows: let the t-th frame boundary set be , the t+1-th frame set be , and enumerate the corresponding relationship between each boundary in and each boundary in .

[0126] Define the comprehensive matching score function , which is composed of the following three parts: the center of gravity distance error Euclidean distance between the barycenter of two borders; area ratio error Deviation of area ratio from 1; contour similarity Measure the similarity of contour shape using Hausdorff distance or Fourier shape descriptor; wherein: , preferably the weights are a1=0.4, a2=0.3, a3=0.3;

[0127] Set matching threshold When M is less than (e.g. set to 20.0), consider the corresponding body of the same border in consecutive frames as corresponding bodies, and add the same border sequence .

[0128] After this processing, a plurality of border contour sequence sets with time consistency are obtained , which are used for dynamic disturbance feature extraction.

[0129] S503: To determine whether the border has typical flame disturbance behavior, the time-varying behavior of the border is structured into a disturbance feature vector group in this step, which includes the following three types of disturbance indicators:

[0130] Deformation rate vector : represents the degree of change of the contour area or perimeter; for each border , calculate its area , define the deformation rate as ;

[0131] Barycenter shift vector : represents the moving trend of the barycenter of the border over time, set the geometric center of , then ;

[0132] Edge disturbance indicator : represents the fluctuation of the edge complexity of the contour, and the border fractal dimension or curvature change standard deviation is preferably used as the measurement indicator.

[0133] The above indicators are respectively composed into time sequence feature vector sequences, i.e. , , .

[0134] S504: Finally, a comprehensive disturbance consistency coefficient R is calculated according to the above disturbance feature vector group, which is used to determine whether the border sequence has typical flame disturbance characteristics. The calculation method is as follows: for each disturbance vector , calculate its standard deviation ; normalize the standard deviation to avoid errors caused by different dimensions of the feature quantities;

[0135] The disturbance consistency coefficient R is defined as a weighted average form: , preferably β1=0.4, β2=0.4, β3=0.2;

[0136] A flame disturbance consistency determination threshold is set , preferably 0.15, when R is less than , the boundary sequence is considered to have flame disturbance consistency, and has a high possibility of fire occurrence.

[0137] In a preferred embodiment of the present application, when the boundary disturbance consistency coefficient R exceeds the set threshold (such as 0.15), it indicates that the region has unstable and strong disturbance boundary behavior, which has potential flame characteristics. However, since some non-fire source disturbances (such as reflection, light spot, etc.) may also trigger R abnormalities, this step further combines the high-frequency disturbance characteristics of the color channel of the image, especially the frequency domain fluctuation analysis of the red channel, to generate a more reliable fire source candidate marker region F. The specific steps are as follows:

[0138] S601: In this step, the color channel features C of the continuous frame images are extracted from the image space region corresponding to the identified boundary sequence S, and the change behavior of the red component is analyzed.

[0139] The specific processing flow is as follows: for each frame image, the red color tone mapping under the HSV color space is obtained, or the red channel of the RGB image is directly extracted; for the spatial region covered by the boundary sequence S, the time sequence formed by the change of the red channel value of each pixel point (i, j) with time is recorded in the next N frames: ; a red channel tensor in the time dimension is constructed for subsequent frequency domain analysis.

[0140] S602: In order to capture the rapid jump characteristics of the flame in color (such as frequency flashing, burning boundary fluctuation), this step performs frequency domain analysis on the above-mentioned red channel time sequence, extracts its high-frequency energy component, and constructs a high-frequency energy spectrum E.

[0141] The processing method is as follows: for each pixel point (i, j) corresponding to R_seq(i, j), short-time Fourier transform (STFT) or discrete wavelet transform (DWT) is performed; if STFT is used, the energy density of the high-frequency band (such as frequency ≥ 0.4×Fs, Fs is the frame rate) in the frequency spectrum is extracted within the window length k (preferably 5); the energy spectrum E(i, j) is obtained, which represents the intensity of the obvious high-frequency disturbance in the red channel at this position; optionally, Gaussian smoothing is performed on the spectrum E to reduce single-point errors. Finally, a two-dimensional image , called red high-frequency energy map, is used to identify the possible combustion source area.

[0142] S603: This step filters out the spatial area with high-intensity color disturbance by analyzing the local aggregation characteristics of the energy map E.

[0143] The implementation is as follows: set a high-frequency energy determination threshold , the value of which is dynamically set according to the statistical distribution of the training set, and is preferably the energy image mean plus 2 times the standard deviation:

[0144] On the energy map E, mark the pixel points satisfying as red high-frequency abnormal points;

[0145] Use the connected region analysis method (such as 8-neighbor connectivity) to aggregate adjacent abnormal pixels to form regions, and obtain a red high-frequency abnormal region set .

[0146] S604: To ensure the consistency of spatial and temporal information, this step performs a spatial intersection operation between the color high-frequency abnormal region R_high and the boundary disturbance region S, filters out the region that presents high dynamic characteristics in color and shape, and takes it as the candidate fire source region F.

[0147] Specifically, traverse each red high-frequency abnormal region , and perform a spatial intersection calculation with the mask region corresponding to the boundary sequence S k to determine the degree of overlap; if the overlapping area exceeds a certain proportion (such as intersection over union IOU ≥ 0.3), it is considered as a strong candidate fire source region; merge all overlapping regions that meet the conditions, and output the final fire source marking region F; optionally, assign a confidence score to each F region as one of the input features for subsequent multi-modal discrimination.

[0148] This embodiment introduces a multi-modal fusion discrimination mechanism in step S700, comprehensively analyzes the texture stability, disturbance directionality, and historical smoke evolution behavior of the F region, and outputs the final fire judgment result L. Specifically, the following steps are included:

[0149] S701: This step is mainly used to determine whether the performance of the F region under different lighting conditions is stable, in order to eliminate the high-brightness interference caused by non-fire source phenomena such as light spots and mirror reflections.

[0150] Specifically, for the image frame where the F region is located, three kinds of image versions are constructed: the original RGB image; the grayscale image (using weighted grayscale transformation Y = 0.299R + 0.587G + 0.114B); and the gamma correction image (the gamma value γ is preferably 2.2, and the conversion formula is ); in each image version, texture features are extracted for the F region, preferably using a local binary pattern (LBP) method for texture encoding; the LBP histogram in the F region is calculated for each image channel, and the entropy value and standard deviation thereof are solved to reflect the texture stability; if the texture fluctuation difference in the three channels is significant (such as the LBP distribution cosine similarity between any two channels being less than a set threshold, such as 0.85), it is considered that the region has unstable texture, which may be a false fire source caused by light interference. This step can provide a texture stability score feature value S_T for the subsequent model.

[0151] S702: In order to distinguish flame disturbance from mechanical disturbance or boundary movement caused by hot air flow, this step models the directionality of the disturbance trajectory of the F region.

[0152] Specifically, in the continuous N frames of images, the outer boundary contour or the center of mass position of the F region is tracked to form a time sequence trajectory ; principal component analysis (PCA) is performed on the two-dimensional trajectory data to extract a first principal direction vector , representing the main disturbance direction; the cosine of the angle between each frame of disturbance vector and the main direction is statistically analyzed to obtain a direction disturbance deviation sequence A_seq, and the standard deviation σ_dir thereof is further calculated; if σ_dir is less than a set threshold (such as 10 degrees corresponding to a cosine of 0.984), it is judged that the disturbance has strong direction consistency, which meets the flame disturbance characteristics; if the deviation is large and irregular, it may be an environmental factor disturbance. This step outputs a disturbance direction consistency score S_D.

[0153] S703: This step analyzes whether there is a smoke evolution process in the space around the F region through time backtracking, which is used to assist in judging whether the current fire source region is accompanied by a normal fire development path.

[0154] Specifically, M frames (preferably 5 to 10 frames) before the frame where the F region is located are backtracked to obtain a sequence of image blocks of the spatially adjacent region; a smoke detection algorithm is performed on each image block, preferably using the following method: S channel elevation and V channel reduction in HSV space; local contrast of the image decreases (local variance is used for calculation); texture blurring increases (high-pass filter response decreases); a time sequence feature vector is constructed, and whether the above smoke indicators are continuously enhanced is analyzed; if there is a stable smoke change trend from weak to strong (for example, the detection indicators are monotonically increasing or the average change rate is greater than a set threshold), it is considered that there is a smoke evolution trend, and a score S_S (such as between 0 and 1) is output.

[0155] S704: After completing the analysis of the texture stability, disturbance directionality, and smoke evolution trend of the F region in three aspects, this step uses a multi-modal fusion model for final fire discrimination.

[0156] Specifically, a three-dimensional feature vector X = [S_T, S_D, S_S] is constructed, representing texture stability score, disturbance direction consistency score and smoke evolution matching score respectively; the feature vector is input into a pre-trained fire multi-modal fusion discrimination model, and the model type is preferably selected from: a multi-layer perception (MLP) with an input layer-hidden layer-output layer structure and a ReLU activation function; or a random forest model based on a decision tree, which is used for small sample high robustness scenarios; the model output is a fire judgment result L, which can be: a binary result (L = 1 indicates that the fire is judged, and L = 0 indicates that it is not a fire); or a continuous confidence score L ∈ [0, 1], which is used for setting a dynamic threshold by a subsequent alarm system.

[0157] In a preferred embodiment of the present application, when the fire judgment result L output by the multi-modal fusion discrimination model indicates that the fire is true, in order to realize the rapid response and subsequent tracing of the fire, the step S800 includes the generation and sending of an alarm signal, and the reconstruction and visual labeling of the fire source trajectory and the fire development path based on the candidate fire source region F, which specifically includes the following processing steps:

[0158] S801: When the fire judgment result L is determined as "fire state" (i.e. L = 1), a structured alarm data packet containing key information is immediately generated, and an alarm signal is sent through a network interface or a local I / O port.

[0159] The processing flow is as follows: a structured alarm data is constructed, which at least includes the following fields: alarm timestamp (T_alert), using system clock or video frame timestamp; fire area position (F_pos), represented by the bounding box coordinates or the centroid coordinates of the F region; alarm level (Level), which can be automatically classified based on the F region area, confidence value L, etc. (such as 1st level: small suspected range; 2nd level: obvious trend of spread); the above data is encapsulated into a structured data packet (such as JSON or binary protocol format); the alarm signal is sent to: a remote video monitoring platform; a local intelligent terminal (such as a voice alarm, a fire linkage module); a cloud disaster dispatch system (such as a city fire cloud platform) through a communication module (such as RS485, Ethernet, MQTT, 4G module, etc.).

[0160] S802: To assist subsequent tracing and fire point positioning, this step traces the historical evolution trajectory of the candidate fire source region F from the current fire determination frame to the previous N_back frames of images, tracking the historical evolution trajectory of the candidate fire source region F.

[0161] Specifically, the backtracking frame number N_back is set to 10 to 30 frames (according to the scene frame rate); for each frame (k = 1 to N_back) of the fire source candidate region , extract its geometric center coordinates ;

[0162] construct a time series trajectory set of the fire source region F, , representing the fire source path from the pre-fire evolution to the current;

[0163] Optionally, if the area of the F region is extremely small or the confidence is extremely low in the early frames, it can be inferred that the fire point started at time , further marking the "first appearance time point" of the fire.

[0164] S803: In the continuous frames after the fire determination, the system continues to track the spatial expansion trend of the fire source region F, and constructs the fire dynamic expansion trajectory P_expand, which is used to evaluate the spread speed and direction. The processing method is as follows:

[0165] For each frame image after the determination of the fire , extract the F region's: geometric center coordinates ; bounding box area ; contour shape (which can be used for irregular flame boundary judgment); calculate the area expansion rate ; construct the trajectory path , while recording the expansion speed sequence and direction vector sequence; if the expansion rate continues to increase and is accompanied by centroid shift, the fire direction and spread intensity can be dynamically determined.

[0166] S804: To assist video dispatch and on-site disposal, this step splices the starting path P_start and the expansion trajectory P_expand to generate the complete fire development trend path P_total, and visualizes it on the video frame.

[0167] The specific processing method is: merge P_start and P_expand into a continuous path line, and generate a curve structure with arrows according to the trajectory direction; set a color coding rule, for example: the starting point is marked as a green dot; the fire determination point is a red flashing point; the expansion trajectory is a red-yellow gradient line, and the deeper the color, the faster the expansion; in the alarm video frame image, P_total is superimposed as a layer drawing, and is synchronously embedded in the alarm video or used for real-time preview of the dispatch terminal; optionally, P_total and the original image are stored synchronously, which is used for accident playback or AI re-learning training set construction.

[0168] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A fire intelligent detection method based on artificial intelligence video analysis, characterized in that: include: S100: Acquires video surveillance stream of the target area and extracts the spatial color feature vector C and morphological feature vector M of each frame image; S200: Perform temporal difference analysis on consecutive frame images to obtain the change magnitude vectors ΔC and ΔM based on C and M, and establish an initial dynamic feature set D; S300: Construct a multi-scale temporal feature map group T, wherein T is composed of a three-dimensional tensor formed by the D of N consecutive frames of images under different time windows; S400: Use a dual-channel neural network model based on attention mechanism to extract temporal features from T and output a probability map P of suspected fire areas; S500: Perform boundary stability analysis on the suspected region in P to determine whether there is a boundary sequence S with flame-shaped perturbation characteristics, and calculate its perturbation consistency coefficient R. S600: If R exceeds the set threshold, further combine the high-frequency fluctuation characteristics of the red channel in C and ΔC to generate candidate fire source marking regions F; S700: Input F into the fire multimodal fusion discrimination model, fuse the texture stability, perturbation directionality and historical smoke evolution model of region F under different lighting channels, and output the final fire judgment result L; S800: If L determines that a fire has occurred, an alarm signal is sent and the fire's starting position and development path are marked in the video frame.

2. The intelligent fire detection method based on artificial intelligence video analysis according to claim 1, characterized in that: S200 includes: S201: Perform inter-frame registration processing on multiple consecutive frames of images within a preset time window; S202: Based on the registered image frame sequence, perform frame-by-frame difference operation on the spatial color feature vector C and morphological feature vector M of each frame image to calculate the color change amplitude ΔC and morphological change amplitude ΔM between adjacent frames. The difference method includes Euclidean distance or structural similarity calculation. S203: Perform filtering and smoothing on ΔC and ΔM; S204: Combine the smoothed ΔC and ΔM to construct a dynamic feature set D, which is used to describe the evolution characteristics of suspected flame or smoke regions in a video frame sequence.

3. The intelligent fire detection method based on artificial intelligence video analysis according to claim 1, characterized in that: The S300 includes: S301: Set multiple time window scale groups n is the total number of windows, and each time window corresponds to a frame interval, which is used to extract image sequence features of different time lengths from the dynamic feature set D. S302: For each time window, extract the feature map sequence of consecutive frames from the dynamic feature set D, and stack the sequence in time order to form a primary three-dimensional tensor Ti with size Wi×H×W, where H and W are the image height and width; S303: Perform temporal normalization on each primary 3D tensor Ti to generate a temporally normalized tensor Ti′; S304: Combine the time-normalized tensors Ti′ at all scales to form a multi-scale time-series feature map group T, which is used to characterize the dynamic evolution characteristics of fire at different time granularities.

4. The intelligent fire detection method based on artificial intelligence video analysis according to claim 1, characterized in that: The S400 includes: S401: Construct a dual-channel neural network model with a parallel structure, which receives color dynamic information and morphological dynamic information from the multi-scale temporal feature map group T respectively, and independently extracts the corresponding temporal spatial feature tensors; S402: Within each channel, calculate the keyframe weights in the time dimension of the input 3D tensor. S403: Perform channel fusion processing on the feature tensors output from the two channels to generate a fused multidimensional fire feature map; S404: The fused feature map is input into the spatial saliency decoding network, and convolutional and deconvolutional layers are used to gradually restore the suspected fire area probability map P with the same size as the original image. The value of each pixel in P represents the confidence probability that the location is a fire area.

5. The intelligent fire detection method based on artificial intelligence video analysis according to claim 1, characterized in that: The S500 includes: S501: Binarize the probability map P of suspected fire areas in multiple consecutive frames, extract the boundaries of areas with confidence scores higher than the first threshold, and extract the boundary contour set B for each frame. t ; S502: On the time series, the boundary contour set B t Inter-frame matching is performed, and boundary correspondence is constructed based on the geometric centroid position of the contour, the similarity of the boundary shape, and the area change ratio to form a candidate boundary sequence S; S503: Quantify the perturbation behavior of each boundary in the boundary sequence S and construct a perturbation feature vector set, including boundary deformation rate, centroid drift vector and local edge irregularity index; S504: Calculate the perturbation consistency coefficient R based on the perturbation feature vector group. R is the normalized result of the standard deviation of the perturbation features during the boundary change over time. If R is less than the second threshold, the boundary sequence is determined to have flame perturbation consistency.

6. The intelligent fire detection method based on artificial intelligence video analysis according to claim 1, characterized in that: The S600 includes: S601: Provided that the perturbation consistency coefficient R exceeds a set threshold, extract the spatial color features C of each frame from the image region corresponding to the boundary sequence S, and focus on extracting the red channel values. Pixel distribution; S602: Perform frequency domain analysis on the time-series value sequence of the red channel in consecutive frame images, extract the high-frequency components of the red channel, and construct the red high-frequency energy spectrum E; S603: Within the region corresponding to the boundary sequence, analyze the spatial distribution of the high-frequency energy spectrum E, identify the high-frequency fluctuation clustering region, and screen out the red high-frequency anomaly region R_high; S604: Perform spatial overlap calculation on the boundary sequence S and R_high, and extract the intersection region as the candidate fire source marker region F.

7. The intelligent fire detection method based on artificial intelligence video analysis according to claim 1, characterized in that: The S700 includes: S701: Extract texture stability index for candidate fire source marking region F under multiple illumination channels, and calculate its local texture consistency score in the original image, grayscale image and gamma-corrected image respectively; S702: Based on the contour deformation trajectory of region F in consecutive frames, construct a perturbation directionality vector set, obtain the main perturbation direction through principal component analysis, and calculate the direction perturbation deviation value. S703: Call the historical smoke evolution model to perform time-reverse feature comparison on region F, assess whether there is a continuous smoke diffusion process in its surrounding areas, and output a matching score; S704: The texture stability, perturbation directionality and historical smoke matching score are taken as inputs and fed into the fire multimodal fusion discrimination model for joint decision-making, and the final fire judgment result L is output, where L is a binary result or confidence score.

8. The intelligent fire detection method based on artificial intelligence video analysis according to claim 1, characterized in that: The S800 includes: S801: When the fire judgment result L is in the established state, generate the alarm time, target area location and alarm level, and send an alarm signal; S802: Starting from the time point when the fire was determined, trace back through consecutive video frames, extract the geometric center position of the candidate fire source marking region F in each frame image, and construct the fire starting path trajectory P_start; S803: In consecutive frames after the fire occurs, dynamically track the area expansion, boundary movement trend and center drift direction of region F, and calculate the fire spread path trajectory P_expand; S804: P_start and P_expand are concatenated to generate the fire development trend path P_total, and the fire source origin, movement direction and development trend are marked with trajectory lines in the alarm video frame.

Images