A flame detection method, system, medium and product fusing multi-modal features

Abstract

This application provides a flame detection method, system, medium, and product that integrates multimodal features, relating to the field of fire detection. Implementing this method, the detection system simultaneously acquires visible light and thermal images using a binocular pan-tilt unit. Image quality is optimized by combining Retinex enhancement and spatiotemporal denoising, and the two modal features are fused through spatial alignment. The detection system establishes a dynamic spatiotemporal map, using suspected flame primitives as nodes, and simultaneously considers spatial edge weights (Euclidean distance and temperature gradient) and temporal edge weights (temperature change rate and area expansion rate), fully utilizing the evolutionary characteristics of flames in the spatiotemporal dimension. The detection system uses a graph neural network model to analyze and judge the dynamic spatiotemporal map, which can effectively reduce the false alarm rate of fires. Compared with a single sensor, this multimodal feature fusion method can better overcome the influence of interference factors such as changes in illumination and smoke obstruction, improving the reliability and accuracy of flame detection.

Description

Technical Field

[0001] This application relates to the field of fire detection, and in particular to a flame detection method, system, medium, and product that integrates multimodal features. Background Technology

[0002] In recent years, photovoltaic power generation, as an important form of clean energy, has experienced rapid development and widespread application globally. Due to the dense equipment and heavy electrical load within photovoltaic power plants, fires can cause severe economic losses. Therefore, establishing an efficient and reliable fire detection system is crucial for ensuring the safe operation of photovoltaic power plants.

[0003] Currently, fire detection methods primarily rely on data acquisition from smoke and temperature sensors. This involves deploying numerous sensor nodes within the monitoring area to collect real-time smoke and temperature data. An alarm is triggered when these data become abnormal. Fire detection methods can also be based on image analysis using the visual channel, extracting features such as color and texture from visible light images to determine the presence of flames.

[0004] However, in practical applications, the above-mentioned fire detection methods are easily affected by factors such as changes in lighting, smoke obstruction, extreme temperatures, and dust coverage, making it difficult to guarantee the reliability of fire detection results. Summary of the Invention

[0005] This application provides a flame detection method, system, medium, and product that integrates multimodal features to improve the accuracy and timeliness of fire detection.

[0006] In a first aspect, this application provides a flame detection method that fuses multimodal features, applied to a detection system. The method includes: acquiring visible light images and thermal imaging images collected by a binocular pan-tilt unit within a preset time period; performing a Retinex enhancement operation on the visible light images to obtain an enhanced visible light image; and performing a spatiotemporal denoising operation on the thermal imaging images to obtain a denoised thermal imaging image. The binocular pan-tilt unit includes a visible light channel for acquiring changes in ambient light intensity and a thermal imaging channel for acquiring ambient temperature distribution. The enhanced visible light image and the denoised thermal imaging image are spatially aligned to obtain a multi-feature fused image sequence. The multi-feature fused image sequence includes multiple consecutive image frames, each image frame including ambient light intensity features and ambient temperature. Features: Based on multi-feature fusion image sequences, preset ambient light intensity conditions, and preset ambient temperature conditions, feature vectors corresponding to each suspected flame primitive are determined. The feature vectors represent the geometric morphology, temperature distribution, and edge structure features of the suspected flame primitive. A dynamic spatiotemporal graph is established with suspected flame primitives as nodes, Euclidean distance and temperature gradient between suspected flame primitives as spatial edge weights, and temperature change rate and area expansion rate of suspected flame primitives within a preset time period as temporal edge weights. The dynamic spatiotemporal graph is analyzed using a graph neural network model to obtain the confidence level of the suspected flame primitives. Suspicious flame primitives with confidence levels exceeding a preset confidence threshold are identified as real flame primitives.

[0007] By employing the above technical solution, the detection system simultaneously acquires visible light and thermal images using a binocular pan-tilt unit. Image quality is optimized through Retinex enhancement and spatiotemporal noise reduction, and the two modal features are fused through spatial alignment. The detection system establishes a dynamic spatiotemporal map, using suspected flame primitives as nodes, and simultaneously considers spatial edge weights (Euclidean distance and temperature gradient) and temporal edge weights (temperature change rate and area expansion rate), fully utilizing the evolutionary characteristics of flames in the spatiotemporal dimension. The detection system uses a graph neural network model to analyze and judge the dynamic spatiotemporal map, which can effectively reduce the false alarm rate of fires. Compared with a single sensor, this multimodal feature fusion method can better overcome the influence of interference factors such as changes in illumination and smoke obstruction, improving the reliability and accuracy of flame detection.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, based on a multi-feature fusion image sequence, a preset ambient light intensity condition, and a preset ambient temperature condition, the feature vectors corresponding to the suspected flame primitives are determined. Specifically, this includes: performing image segmentation on each image frame in the multi-feature fusion image sequence to obtain multiple candidate regions; calculating the average light intensity and average temperature of each candidate region; if the average light intensity is greater than a preset light intensity threshold and the average temperature is greater than a preset temperature threshold, the candidate region is determined as a suspected flame primitive; and extracting the geometric morphology features, temperature distribution features, and edge structure features of the suspected flame primitive to obtain the feature vector corresponding to the suspected flame primitive.

[0009] By employing the above technical solution, the detection system performs image segmentation on multi-feature fusion image sequences to obtain multiple candidate regions. These regions are then filtered using preset light intensity and temperature thresholds, effectively extracting potential flame areas, i.e., suspected flame primitives. This method considers threshold constraints in both light intensity and temperature dimensions, eliminating false alarms caused solely by abnormal lighting or temperature. This helps improve the accuracy of fire detection and reduces the false alarm rate.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, a dynamic spatiotemporal graph is established using suspected flame primitives as nodes, the Euclidean distance and temperature gradient between suspected flame primitives as spatial edge weights, and the temperature change rate and area expansion rate of suspected flame primitives within a preset time period as temporal edge weights. Specifically, this includes: within each image frame, treating all suspected flame primitives as nodes; within the same image frame, calculating the Euclidean distance and temperature gradient between the first and second suspected flame primitives; if the Euclidean distance is less than a preset distance threshold, establishing spatial edges between the nodes corresponding to the first and second suspected flame primitives respectively. Spatial edge weights are calculated based on Euclidean distance and temperature gradient. The first and second suspected flame primitives are any two suspected flame primitives. Between two consecutive frames, the suspected flame primitives in the previous frame are associated and matched with the suspected flame primitives in the next frame. If the association and matching are successful, a temporal edge is established between the nodes corresponding to the suspected flame primitives in the previous frame and the suspected flame primitives in the next frame. The temperature change rate and area expansion rate of the suspected flame primitives between the two consecutive frames are calculated. The temporal edge weights are calculated based on the temperature change rate and area expansion rate. The suspected flame primitive is any one of the suspected flame primitives.

[0011] By adopting the above technical solution, when constructing a dynamic spatiotemporal graph, the detection system sets a preset distance threshold to constrain the establishment conditions of spatial edges, which can avoid establishing invalid connections between suspicious flame primitives that are far apart. The detection system calculates the correlation matching of suspicious flame primitives between consecutive frames and constructs temporal edge weights based on the rate of temperature change and the rate of area expansion, which can effectively describe the dynamic evolution characteristics of suspicious flame primitives. This graph structure design, which considers both spatial topological relationships and temporal evolution characteristics, can comprehensively capture the spatiotemporal characteristics of flames, providing structured input data for subsequent graph neural network analysis, and helping to improve the accuracy and real-time performance of fire detection.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, the confidence level of the suspected flame primitive is obtained by analyzing the dynamic spatiotemporal graph through a graph neural network model. Specifically, this includes: determining the spatial domain features and temporal domain features of each suspected flame primitive based on the spatial edge weights, temporal edge weights, and the feature vectors corresponding to each suspected flame primitive; and inputting the feature vectors, spatial domain features, and temporal domain features corresponding to the suspected flame primitive into the graph neural network model to obtain the confidence level of the suspected flame primitive.

[0013] By adopting the above technical solution, the detection system extracts the spatial and temporal features of each suspicious flame primitive based on a dynamic spatiotemporal graph, constructing a multi-dimensional feature representation system. This system is then input into a graph neural network model for analysis, fully leveraging the advantages of deep learning to model and learn flame features. This method, combining traditional feature engineering and deep learning, utilizes the powerful feature learning capabilities of neural networks. By outputting the confidence level of suspicious flame primitives, it provides a quantifiable judgment criterion, allowing the detection system to flexibly adjust the confidence threshold according to actual application scenarios, balancing detection sensitivity and reliability.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, based on spatial edge weights, temporal edge weights, and the feature vectors corresponding to each suspicious flame primitive, the spatial domain features and temporal domain features of each suspicious flame primitive are determined. Specifically, this includes: identifying the adjacent primitives of the suspicious flame primitive in each image frame, calculating the product of the feature vector of each adjacent primitive and the corresponding spatial edge weight, summing the products to obtain the spatial domain features of the suspicious flame primitive; identifying the matching primitive of the suspicious flame primitive in the previous image frame, calculating the product of the feature vector of the matching primitive and the corresponding temporal edge weight to obtain the temporal domain features of the suspicious flame primitive.

[0015] By employing the above technical solution, the detection system identifies the adjacent primitives of suspicious flame primitives in each image frame and calculates the product of the feature vector of each adjacent primitive and the corresponding spatial edge weight, effectively capturing local spatial structure information. The system also identifies the matching primitives of suspicious flame primitives in the previous image frame and calculates the product of the feature vector of the matching primitive and the corresponding temporal edge weight, accurately describing the dynamic changes of the flame. This method of extracting spatial and temporal features considers both static spatial structure features and dynamic temporal evolution features, forming a comprehensive feature representation system. Through the product operation of feature vectors and corresponding weights, adaptive weighting of features is achieved, highlighting the contribution of important features, suppressing the influence of noisy features, and improving the effectiveness of feature representation.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, after analyzing the dynamic spatiotemporal graph using a graph neural network model to obtain the confidence level of suspicious flame primitives, and identifying suspicious flame primitives with confidence levels exceeding a preset confidence threshold as real flame primitives, the method further includes: calculating the physical coordinates of real flame primitives in three-dimensional space based on the pixel coordinates of real flame primitives in the multi-feature fusion image sequence and the intrinsic and extrinsic parameters of the binocular pan-tilt unit; calculating the fire spread direction and fire spread speed based on the changes in the physical coordinates of real flame primitives within a preset time period; integrating the fire spread direction, fire spread speed, and the total number of real flame primitives into fire alarm information and sending it to the fire emergency response terminal.

[0017] By employing the aforementioned technical solution, the detection system maps the pixel coordinates of real flame elements in a multi-feature fusion image sequence to three-dimensional space using the internal and external parameters of a binocular pan-tilt unit, achieving precise flame location. By tracking the changes in the physical coordinates of the real flame elements, the system can accurately calculate the direction and speed of fire spread, providing a quantitative basis for fire early warning. The system integrates this crucial information into fire alarm messages and sends them to the emergency response terminal, achieving a complete closed loop from detection to warning. This fire analysis method based on physical coordinates not only provides the spatial situation of fire development but also offers firefighters specific tactical deployment references, effectively improving the targeting and timeliness of emergency response.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, after analyzing the dynamic spatiotemporal graph using a graph neural network model to obtain the confidence level of the suspected flame primitives, and identifying the suspected flame primitives whose confidence level exceeds a preset confidence threshold as real flame primitives, the method further includes: acquiring temperature time-series data of the real flame primitives, the temperature time-series data including the highest temperature, average temperature, and temperature variance of the real flame primitives within a preset time period; based on the temperature time-series data, using a time series prediction model to predict the temperature evolution trend of the real flame primitives to obtain a predicted temperature curve; determining the combustion stage of the real flame primitives according to the slope and curvature of the predicted temperature curve, the combustion stage including the initial ignition stage, the rapid development stage, and the stable combustion stage; and determining the data acquisition frequency of the binocular pan-tilt unit based on the combustion stage.

[0019] By adopting the above technical solution, the detection system predicts the temperature evolution trend of real flame elements based on temperature time-series data and a time-series prediction model, thereby determining the different combustion stages of the real flame elements. The detection system dynamically adjusts the data acquisition frequency of the binocular pan-tilt unit according to the combustion stage, ensuring monitoring density during critical periods while avoiding data redundancy, thus optimizing the allocation of monitoring resources. This predictive and adaptive data acquisition strategy not only improves system operating efficiency but also provides reliable data support for fire development trend analysis.

[0020] In a second aspect, embodiments of this application provide a detection system comprising: one or more processors and a memory; the memory is coupled to the one or more processors and is used to store computer program code, the computer program code including computer instructions, wherein the one or more processors invoke the computer instructions to cause the detection system to perform the method described in the first aspect and any possible implementation thereof.

[0021] Thirdly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a detection system, cause the detection system to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a detection system, cause the detection system to perform the method described in the first aspect and any possible implementation thereof.

[0023] Understandably, the detection system provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0024] This application provides a flame detection method that integrates multimodal features. It simultaneously acquires visible light and thermal images using a binocular pan-tilt unit, and optimizes image quality through Retinex enhancement and spatiotemporal denoising. Furthermore, it fuses the two modal features through spatial alignment, effectively overcoming the susceptibility of single sensors to interference from factors such as lighting changes and smoke obstruction, thus improving the reliability and accuracy of flame detection. Further, this application establishes a dynamic spatiotemporal map, using suspected flame primitives as nodes, and simultaneously considering spatial edge weights (Euclidean distance and temperature gradient) and temporal edge weights (temperature change rate and area expansion rate). This fully utilizes the evolutionary characteristics of flames in the spatiotemporal dimension, enabling comprehensive capture of the flame's spatiotemporal features. Finally, a graph neural network model is used to analyze and judge the dynamic spatiotemporal map, effectively reducing the false alarm rate and providing a high-precision, high-reliability technical solution for fire detection.

[0025] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. By adopting the above technical solution, the detection system simultaneously acquires visible light and thermal images using a binocular pan-tilt unit. Image quality is optimized by combining Retinex enhancement and spatiotemporal denoising, and the two modal features are fused through spatial alignment. The detection system establishes a dynamic spatiotemporal map, using suspected flame primitives as nodes, and simultaneously considers spatial edge weights (Euclidean distance and temperature gradient) and temporal edge weights (temperature change rate and area expansion rate), fully utilizing the evolution characteristics of flames in the spatiotemporal dimension. The detection system uses a graph neural network model to analyze and judge the dynamic spatiotemporal map, which can effectively reduce the false alarm rate of fires. Compared with a single sensor, this multimodal feature fusion method can better overcome the influence of interference factors such as changes in illumination and smoke obstruction, improving the reliability and accuracy of flame detection.

[0026] 2. By adopting the above technical solution, when constructing the dynamic spatiotemporal graph, the detection system sets a preset distance threshold to constrain the establishment conditions of spatial edges, which can avoid establishing invalid connections between suspicious flame primitives that are far apart. The detection system calculates the correlation matching of suspicious flame primitives between consecutive frames and constructs temporal edge weights based on the rate of temperature change and the rate of area expansion, which can effectively describe the dynamic evolution characteristics of suspicious flame primitives. This graph structure design, which considers both spatial topological relationships and temporal evolution characteristics, can comprehensively capture the spatiotemporal characteristics of flames, providing structured input data for subsequent graph neural network analysis, and helping to improve the accuracy and real-time performance of fire detection.

[0027] 3. By adopting the above technical solution, the detection system identifies the adjacent primitives of the suspicious flame primitives in each image frame, calculates the product of the feature vector of each adjacent primitive and the corresponding spatial edge weight, and can effectively capture local spatial structure information. The detection system also identifies the matching primitives of the suspicious flame primitives in the previous image frame, calculates the product of the feature vector of the matching primitive and the corresponding temporal edge weight, and can accurately describe the dynamic change characteristics of the flame. This method of extracting spatial and temporal features considers both static spatial structure features and dynamic temporal evolution features, forming a comprehensive feature representation system. Through the product operation of feature vectors and corresponding weights, adaptive weighting of features is achieved, which can highlight the contribution of important features, suppress the influence of noisy features, and improve the effectiveness of feature representation. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating a flame detection method that integrates multimodal features in an embodiment of this application. Figure 2 This is another flowchart illustrating the flame detection method that integrates multimodal features in the embodiments of this application; Figure 3 This is a schematic diagram of the physical device structure of the detection system in the embodiments of this application. Detailed Implementation

[0029] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0030] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0031] The following describes the process of the method provided in this implementation. Please refer to [link / reference]. Figure 1 This is a flowchart illustrating a flame detection method that integrates multimodal features in an embodiment of this application.

[0032] S101. Acquire visible light images and thermal imaging images collected by the binocular gimbal within a preset time period. Perform Retinex enhancement operation on the visible light images to obtain a visible light enhanced image. Perform spatiotemporal domain noise reduction operation on the thermal imaging images to obtain a thermal imaging noise reduced image. The binocular gimbal includes a visible light channel for acquiring changes in ambient light intensity and a thermal imaging channel for acquiring ambient temperature distribution. Among them, a binocular pan-tilt unit refers to a pan-tilt device containing two image acquisition channels, used to simultaneously acquire different types of image data; a visible light channel refers to an imaging channel used to acquire changes in light intensity in the environment, usually composed of a visible light camera; a thermal imaging channel refers to an infrared imaging channel used to acquire temperature distribution in the environment, usually composed of an infrared thermal imager; a preset duration refers to a pre-set time period for continuous data acquisition, such as 1 minute or 5 minutes; a Retinex enhancement operation represents an image enhancement algorithm used to improve image contrast and detail; and a spatiotemporal domain noise reduction operation represents noise suppression processing performed simultaneously in the spatial and temporal dimensions.

[0033] Specifically, firstly, the detection system continuously acquires images within a preset time (e.g., 1 minute) through the visible light channel and thermal imaging channel of the binocular pan-tilt unit: For the visible light images acquired through the visible light channel, the detection system applies the multi-scale Retinex algorithm (which enhances the local contrast of the visible light images through Gaussian filtering and logarithmic domain operations, making details in dark areas more clearly visible) for enhancement processing; for the thermal imaging images acquired through the thermal imaging channel, the detection system combines temporal Kalman filtering and spatial bilateral filtering for noise reduction processing, effectively suppressing thermal noise interference while maintaining temperature edge information.

[0034] S102. Spatial alignment is performed on the visible light enhanced image and the thermal imaging denoised image to obtain a multi-feature fused image sequence. The multi-feature fused image sequence includes multiple consecutive image frames, and each image frame includes ambient light intensity features and ambient temperature features. Spatial alignment refers to the process of mapping corresponding physical locations in visible light enhanced images and thermal imaging denoised images to the same image coordinate positions; multi-feature fusion image sequence refers to a sequence of multiple temporally consecutive image frames that have completed feature fusion; ambient light intensity features refer to feature data describing the distribution of light intensity in a scene; ambient temperature features refer to feature data describing the distribution of temperature in a scene.

[0035] Specifically, firstly, the detection system establishes a spatial mapping relationship between the visible light image and the thermal imaging image based on the geometric calibration parameters of the binocular pan-tilt unit. Then, the system transforms the thermal imaging image to the same viewing angle as the visible light image through projection transformation, ensuring that pixels at the same physical location in both images have the same image coordinates. The system then fuses the registered enhanced visible light image and the denoised thermal imaging image at the pixel level, generating a fused image that includes both ambient light intensity and ambient temperature features. Finally, the system organizes all fused images within a preset time period into an image sequence, i.e., a multi-feature fused image sequence.

[0036] S103. Based on the multi-feature fusion image sequence, preset ambient light intensity conditions and preset ambient temperature conditions, determine the feature vectors corresponding to the suspected flame primitives and the suspected flame primitives respectively. The feature vectors represent the geometric morphology features, temperature distribution features and edge structure features of the suspected flame primitives. Among them, the suspected flame primitive refers to the basic unit that is suspected to be a flame in the multi-feature fusion image sequence; the feature vector refers to the feature set represented in numerical form; the geometric morphology feature refers to the feature that describes the shape and size of the suspected flame primitive; the temperature distribution feature refers to the feature that describes the spatial distribution of temperature inside the suspected flame primitive; the edge structure feature refers to the feature that describes the primitive outline and boundary characteristics of the suspected flame primitive; the preset ambient light intensity condition refers to the predefined light intensity threshold constraint; and the preset ambient temperature condition refers to the predefined temperature threshold constraint.

[0037] Specifically, firstly, the detection system performs adaptive threshold segmentation on each frame of the multi-feature fused image sequence to obtain multiple candidate regions. For each candidate region, the detection system calculates the average light intensity and average temperature. If both a preset light intensity threshold (e.g., greater than 150) and a preset temperature threshold (e.g., greater than 60℃) are met, it is marked as a suspicious flame primitive. Then, the detection system extracts features from each suspicious flame primitive: geometric features include area, perimeter, and roundness; temperature distribution features include maximum temperature, average temperature, and temperature gradient; edge structure features include edge sharpness and edge complexity. All features are combined into a multi-dimensional feature vector for subsequent analysis.

[0038] Optionally, under normal circumstances, based on the multi-feature fusion image sequence, preset ambient light intensity conditions, and preset ambient temperature conditions, the determination of the suspected flame primitive and its corresponding feature vector can be achieved in the following way, without limitation: perform image segmentation on each image frame in the multi-feature fusion image sequence to obtain multiple candidate regions, and calculate the average light intensity and average temperature of each candidate region; if the average light intensity is greater than a preset light intensity threshold and the average temperature is greater than a preset temperature threshold, the candidate region is determined as a suspected flame primitive; extract the geometric morphology features, temperature distribution features, and edge structure features of the suspected flame primitive to obtain the feature vector corresponding to the suspected flame primitive.

[0039] S104. Using suspected flame primitives as nodes, Euclidean distance and temperature gradient between suspected flame primitives as spatial edge weights, and temperature change rate and area expansion rate of suspected flame primitives within a preset time period as temporal edge weights, a dynamic spatiotemporal graph is established. Among them, the dynamic spatiotemporal graph refers to the graph structure model that describes the relationship between suspicious flame primitives in the temporal and spatial dimensions; a node refers to the basic unit in the graph structure model that represents a single suspicious flame primitive; Euclidean distance represents the straight-line distance between two suspicious flame primitives in the image space; temperature gradient represents the rate of temperature change between two suspicious flame primitives; spatial edge weight represents the numerical value describing the strength of the spatial relationship between suspicious flame primitives in the same frame; temperature change rate refers to the speed at which the temperature of a suspicious flame primitive changes over time; area expansion rate refers to the speed at which the area of ​​a suspicious flame primitive grows over time; and temporal edge weight represents the numerical value describing the strength of the temporal relationship between suspicious flame primitives in adjacent frames.

[0040] Specifically, the detection system treats each suspected flame primitive as a node in a dynamic spatiotemporal graph. Within the same image frame, the system calculates the Euclidean distance between any two suspected flame primitives. If the Euclidean distance is less than a preset distance threshold (e.g., 50 pixels), a spatial edge is established between the corresponding nodes. The spatial edge weight consists of two parts: spatial proximity based on Euclidean distance (closer distances result in higher weights) and thermal conductivity based on temperature gradient (higher gradients result in higher weights). Between adjacent image frames, the system establishes correspondences between suspected flame primitives using a matching algorithm (e.g., IoU-based matching) and establishes temporal edges between successfully matched nodes. The temporal edge weight also consists of two parts: the rate of temperature change (reflecting the intensity of combustion) and the area expansion rate (reflecting the speed of fire spread).

[0041] Optionally, in general, using suspected flame primitives as nodes, the Euclidean distance and temperature gradient between suspected flame primitives as spatial edge weights, and the temperature change rate and area expansion rate of suspected flame primitives within a preset time period as temporal edge weights, a dynamic spatiotemporal graph can be established in the following way, without limitation: Within each image frame, all suspected flame primitives are treated as nodes; within the same image frame, the Euclidean distance and temperature gradient between the first and second suspected flame primitives are calculated; if the Euclidean distance is less than a preset distance threshold, spatial edges are established between the nodes corresponding to the first and second suspected flame primitives. Spatial edge weights are calculated based on Euclidean distance and temperature gradient. The first and second suspected flame primitives are any two suspected flame primitives. Between two consecutive frames, the suspected flame primitives in the previous frame are associated and matched with the suspected flame primitives in the next frame. If the association and matching are successful, a temporal edge is established between the nodes corresponding to the suspected flame primitives in the previous frame and the suspected flame primitives in the next frame. The temperature change rate and area expansion rate of the suspected flame primitives between the two consecutive frames are calculated. The temporal edge weights are calculated based on the temperature change rate and area expansion rate. The suspected flame primitive is any one of the suspected flame primitives.

[0042] The following is a specific example of a fire detection scenario. Suppose that in an indoor scene, suspicious flame primitives are detected in two consecutive frames (times t and t+1): (1) Three suspicious flame primitives were detected in frame t: Flame Element A: Position (100, 150) pixels, temperature 300℃, area 100 square pixels; Flame primitive B: Location (130, 160) pixels, temperature 350℃, area 80 square pixels; Flame primitive C: Position (200, 200) pixels, temperature 250℃, area 60 square pixels; (2) Three suspicious flame primitives were detected in frame t+1: Flame element A': Location (105, 155) pixels, temperature 320℃, area 120 square pixels; Flame primitive B': Position (135, 165) pixels, temperature 380℃, area 100 square pixels; Flame primitive C': Position (205, 205) pixels, temperature 260℃, area 65 square pixels; (3) The process of constructing a dynamic spatiotemporal graph: Spatial edge construction (within the same frame): In frame t: The Euclidean distance between A and B is approximately 35 pixels, which is less than the 50-pixel threshold. Spatial edges are then established. Since the Euclidean distances between A and C, and between B and C are all greater than the 5050 pixel threshold, no spatial edges are established. AB space edge weight calculation: Distance weight: 1 - (35 / 50) = 0.3; Temperature gradient: (350-300) / 35 = 1.43℃ / pixel; The overall spatial edge weight is: 0.3 × 0.5 + (1.43 / 100) × 0.5 = 0.15 + 0.007 = 0.157; The calculation is similar for frame t+1; Temporal construction (between adjacent frames): Location-based matching: A corresponds to A' (maximum IoU); B corresponds to B' (maximum IoU); C corresponds to C' (maximum IoU); A-A' time edge weight calculation: Temperature change rate: (320-300) / 1s = 20℃ / s; Area expansion rate: (120-100) / 100=20%; The overall time edge weights are: (20 / 100) × 0.5 + 0.2 × 0.5 = 0.1 + 0.1 = 0.2; The time edge weights of B-B' and C-C' are calculated similarly. The final dynamic spacetime graph includes: Six nodes (A, B, C, A', B', C'); Two spatial edges (AB, A'-B'); There are 3 time edges (A-A', B-B', C-C').

[0043] S105. Analyze the dynamic spatiotemporal graph using a graph neural network model to obtain the confidence level of the suspected flame primitives. Identify the suspected flame primitives whose confidence level exceeds the preset confidence threshold as real flame primitives.

[0044] Here, the graph neural network model represents a deep learning model used to process graph structure data; confidence level refers to the degree of certainty of the graph neural network model that a suspicious flame primitive is a real flame primitive, with a value ranging from 0 to 1; the pre-set confidence threshold represents the probability threshold used to distinguish between real flames and fake flames; a real flame primitive refers to a suspicious flame primitive that has been determined to be a real flame by the graph neural network model.

[0045] Specifically, the detection system employs a graph neural network (GNN) model to analyze the dynamic spatiotemporal graph. This GNN model contains multiple layers of graph convolutional layers, each aggregating neighborhood information of nodes: spatial edges enable the GNN model to learn the spatial correlation of flame regions within the same frame, while temporal edges enable it to capture the temporal evolution patterns of flames. The input to the GNN model includes node features (feature vectors corresponding to suspicious flame primitives), spatial edge features (spatial edge weights), and temporal edge features (temporal edge weights). Through multi-layer feature extraction and nonlinear transformation, the GNN model outputs a confidence score between 0 and 1 for each node. The detection system marks suspicious flame primitives corresponding to nodes with confidence scores higher than a preset confidence threshold (e.g., 0.8) as real flame primitives, thus completing flame detection. During the training phase, the GNN model uses a large amount of labeled data, including real flame samples under various lighting and weather conditions, as well as easily confused non-flame samples (such as strong light reflections and heat sources), to improve the model's robustness and generalization ability.

[0046] Optionally, in general, the confidence level of a suspected flame primitive can be obtained by analyzing the dynamic spatiotemporal graph using a graph neural network model in the following way, without limitation: Based on the spatial edge weights, temporal edge weights, and the feature vectors corresponding to each suspected flame primitive, determine the spatial domain features and temporal domain features of each suspected flame primitive; input the feature vectors, spatial domain features, and temporal domain features corresponding to the suspected flame primitive into the graph neural network model to obtain the confidence level of the suspected flame primitive.

[0047] Optionally, in general, based on spatial edge weights, temporal edge weights, and the feature vectors corresponding to each suspicious flame primitive, the spatial domain features and temporal domain features of each suspicious flame primitive can be determined in the following ways, without limitation: Identify the adjacent primitives of the suspicious flame primitive in each image frame, calculate the product of the feature vector of each adjacent primitive and the corresponding spatial edge weight, and sum the products to obtain the spatial domain features of the suspicious flame primitive; Identify the matching primitives of the suspicious flame primitive in the previous image frame, calculate the product of the feature vector of the matching primitive and the corresponding temporal edge weight to obtain the temporal domain features of the suspicious flame primitive.

[0048] By employing the above technical solution, the detection system simultaneously acquires visible light and thermal images using a binocular pan-tilt unit. Image quality is optimized through Retinex enhancement and spatiotemporal noise reduction, and the two modal features are fused through spatial alignment. The detection system establishes a dynamic spatiotemporal map, using suspected flame primitives as nodes, and simultaneously considers spatial edge weights (Euclidean distance and temperature gradient) and temporal edge weights (temperature change rate and area expansion rate), fully utilizing the evolutionary characteristics of flames in the spatiotemporal dimension. The detection system uses a graph neural network model to analyze and judge the dynamic spatiotemporal map, which can effectively reduce the false alarm rate of fires. Compared with a single sensor, this multimodal feature fusion method can better overcome the influence of interference factors such as changes in illumination and smoke obstruction, improving the reliability and accuracy of flame detection.

[0049] The method provided in this implementation will now be described in more detail. Please refer to [link / reference]. Figure 2 This is another flowchart illustrating the flame detection method that integrates multimodal features in this application embodiment.

[0050] The following steps may or may not be performed after step S105; this is not limited here: S201. Based on the pixel coordinates of the real flame primitive in the multi-feature fusion image sequence and the intrinsic and extrinsic parameters of the binocular gimbal, calculate the physical coordinates of the real flame primitive in three-dimensional space.

[0051] In this context, pixel coordinates refer to the two-dimensional position information of the real flame primitive on the image plane, usually represented by (x, y); intrinsic parameters refer to the parameter matrix describing the optical characteristics of the camera, including focal length, principal point coordinates, etc.; extrinsic parameters refer to the parameter matrix describing the position and attitude of the camera in the world coordinate system, including rotation matrix and translation vector; physical coordinates refer to the position of the real flame primitive in real three-dimensional space, usually represented by (X, Y, Z).

[0052] Specifically, the detection system acquires the calibration parameters of the two cameras on the binocular gimbal, including intrinsic parameters (such as focal length, principal point offset, and radial distortion coefficient) and extrinsic parameters (the relative positional relationship between the cameras). Then, based on binocular vision principles, the system locates the corresponding points of the actual flame primitives in the left and right images and calculates the disparity using triangulation. Finally, through the reprojection equation, the system converts the two-dimensional pixel coordinates and disparity into three-dimensional physical coordinates. This process considers lens distortion correction, ensuring the accuracy of the coordinate transformation. The detection system performs this coordinate transformation on the centroid position of each actual flame primitive.

[0053] S202. Calculate the direction and speed of fire spread based on the physical coordinate changes of the actual flame element within a preset time.

[0054] Among them, the fire spread direction refers to the spatial direction of the flame spread, usually represented by a direction vector; the fire spread speed refers to the spatial rate of the flame spread, usually represented by the displacement distance per unit time; and the change in physical coordinates represents the trajectory of the actual flame element's position in three-dimensional space over time.

[0055] Specifically, the detection system arranges the physical coordinates of all real flame primitives within a preset time period (e.g., 1 minute) in chronological order, forming a spatiotemporal trajectory sequence. For each time point, the detection system calculates the position vectors between adjacent real flame primitives and determines the main spread direction, i.e., the fire spread direction, through principal component analysis (PCA). The fire spread rate can be obtained by calculating the rate of change of displacement of the flame region boundary along the main spread direction. Simultaneously, the detection system also calculates the acceleration component of the flame spread rate to assess the fire development trend.

[0056] S203. The direction of fire spread, the speed of fire spread, and the total number of real flame elements are integrated into fire alarm information and sent to the fire emergency response terminal.

[0057] Among them, fire alarm information refers to fire-related early warning data packets generated by the detection system; fire emergency response terminal refers to the equipment that receives and displays fire alarm information, such as computers in the monitoring center and mobile devices of firefighters; the total number of real flame primitives represents the number of all currently detected flame areas.

[0058] Specifically, the detection system converts the direction of fire spread into an easily understandable directional representation (e.g., 45 degrees northeast) and the speed of fire spread into standard units (e.g., meters per second). The system integrates this information with data such as the total number of actual flame elements, the specific location and size of each element, into a structured fire alarm information package. The system then sends the fire alarm information to the fire emergency response terminal. Based on the severity of the fire, the system can set different alarm levels and ensure that the fire alarm information is delivered to relevant personnel in a timely manner through various notification methods (e.g., audible and visual alarms, push notifications).

[0059] S204. Obtain the temperature time series data of the real flame element. The temperature time series data includes the highest temperature, average temperature and temperature variance of the real flame element within a preset time period.

[0060] Among them, temperature time series data refers to the data sequence describing the change of flame temperature over time; maximum temperature refers to the maximum temperature value of the real flame element at a certain moment within a preset time period; average temperature refers to the average temperature of the real flame element within a preset time period; temperature variance refers to the statistical quantity describing the dispersion of the temperature distribution of the real flame element; preset time period indicates the pre-set data collection time period, such as 1 minute or 5 minutes.

[0061] Specifically, the detection system continuously collects temperature data for each real flame element through a thermal imaging channel within a preset time period. Based on the temperature data and sampling time (the sampling interval is typically on the order of milliseconds), the detection system organizes the data into time-series temperature data and calculates the highest temperature, average temperature, and temperature variance of the real flame element within the preset time period.

[0062] S205. Based on temperature time series data, a time series prediction model is used to predict the temperature evolution trend of the real flame element, and the predicted temperature curve is obtained.

[0063] Among them, the time series prediction model refers to the mathematical model used to predict the future trend of temperature time series data, such as LSTM, ARIMA, etc.; the temperature evolution trend refers to the development direction and law of temperature change over time; the predicted temperature curve represents the function curve of temperature change over a future period of time predicted by the time series prediction model.

[0064] Specifically, the detection system performs feature engineering on the temperature time-series data, extracting temporal features (such as timestamps and time periods) and statistical features (such as moving averages and trend indicators). Then, the system employs a Long Short-Term Memory (LSTM) network as the core of its prediction model. This model, with multiple LSTM layers and fully connected layers, effectively captures long-term dependencies in temperature changes. The prediction model takes past temperature time-series data as input and outputs predicted temperature values ​​for several future time points. The system trains the model by minimizing the prediction error and uses a validation set to fine-tune the model parameters. The final generated predicted temperature curve includes both the predicted temperature value and the predicted temperature range.

[0065] S206. Based on the slope and curvature of the predicted temperature curve, determine the combustion stages of the actual flame element. The combustion stages include the initial ignition stage, the rapid development stage, and the stable combustion stage.

[0066] Among them, the slope represents the instantaneous rate of change of the predicted temperature curve at a certain point; the curvature represents the degree of curvature of the predicted temperature curve at a certain point; the combustion stages represent different stages of flame development; the initial stage of ignition refers to the stage when the flame has just formed; the rapid development stage refers to the stage when the fire grows rapidly; and the stable combustion stage refers to the stage when the fire reaches a relatively stable state.

[0067] Specifically, the detection system calculates the first derivative (slope) and second derivative (curvature) of the predicted temperature curve at various time points. Based on the changing patterns of these characteristic values, the system determines the combustion stage: a small slope and near-zero curvature indicate the initial ignition stage; a significantly positive slope and noticeable curvature change indicate the rapid development stage; and a slope approaching zero and stable curvature indicate the stable combustion stage. The system also incorporates auxiliary features such as absolute temperature and variance to improve the accuracy of combustion stage determination.

[0068] S207. Based on combustion stages, determine the data acquisition frequency of the binocular gimbal.

[0069] The data acquisition frequency refers to the number of times image data is acquired per unit time, usually expressed in frame rate (fps).

[0070] Specifically, the detection system dynamically adjusts the data acquisition frequency based on the characteristics of different combustion stages: in the early stages of a fire, when the fire changes relatively slowly, a lower acquisition frequency (e.g., 10 fps) is used to conserve system resources; in the rapid development stage, when the fire changes drastically, the acquisition frequency is increased (e.g., 30 fps) to capture more detailed information; and in the stable combustion stage, the acquisition frequency is appropriately reduced (e.g., 15 fps) depending on the stability of the fire. The detection system also controls the upper limit of the data acquisition frequency based on hardware constraints such as storage capacity and network bandwidth.

[0071] The detection system in the embodiments of this application is described below from the perspective of hardware processing. Please refer to [link / reference]. Figure 3 This is a schematic diagram of the physical device structure of the detection system in this application embodiment.

[0072] It should be noted that, Figure 3 The structure of the detection system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0073] like Figure 3 As shown, the detection system includes a CPU 301, which can perform various appropriate actions and processes according to a program stored in the read-only memory ROM 302 or a program loaded from the storage section 308 into the random access memory RAM 303, such as executing the flame detection method fusion of multimodal features described in the above embodiments. The RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An I / O interface 305 is also connected to the bus 304.

[0074] The following components are connected to I / O interface 305: input section 306 including audio input devices, push-button switches, etc.; output section 307 including a liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 308 including a hard disk, etc.; and communication section 309 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 309 performs communication processing via a network such as the Internet. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0075] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by CPU 301, it performs the various functions defined in this application.

[0076] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0077] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0078] Specifically, the detection system of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the flame detection method that integrates multimodal features provided in the above embodiment.

[0079] In another aspect, this application also provides a computer-readable storage medium, which may be included in the detection system described in the above embodiments; or it may exist independently and not assembled into the detection system. The storage medium carries one or more computer programs, which, when executed by a processor of the detection system, cause the detection system to implement the flame detection method integrating multimodal features provided in the above embodiments.

[0080] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0081] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0082] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A flame detection method integrating multimodal features, characterized in that, Applied to a detection system, the method includes: The binocular gimbal acquires visible light and thermal images within a preset time period. Retinex enhancement is performed on the visible light image to obtain an enhanced visible light image. Spatiotemporal denoising is performed on the thermal image to obtain a denoised thermal image. The binocular gimbal includes a visible light channel for acquiring changes in ambient light intensity and a thermal imaging channel for acquiring ambient temperature distribution. Spatially align the visible light enhanced image and the thermal imaging denoised image to obtain a multi-feature fused image sequence. The multi-feature fused image sequence includes multiple consecutive image frames, and each image frame includes ambient light intensity features and ambient temperature features. Based on the multi-feature fusion image sequence, preset ambient light intensity conditions, and preset ambient temperature conditions, a suspicious flame primitive and its corresponding feature vector are determined. The feature vector represents the geometric morphology features, temperature distribution features, and edge structure features of the suspicious flame primitive. A dynamic spatiotemporal graph is established using the suspected flame primitives as nodes, the Euclidean distance and temperature gradient between the suspected flame primitives as spatial edge weights, and the temperature change rate and area expansion rate of the suspected flame primitives within the preset time period as temporal edge weights. The dynamic spatiotemporal graph is analyzed using a graph neural network model to obtain the confidence level of the suspected flame primitives. Suspicious flame primitives whose confidence levels exceed a preset confidence threshold are identified as real flame primitives.

2. The flame detection method integrating multimodal features according to claim 1, characterized in that, The process of determining the suspicious flame primitive and its corresponding feature vector based on the multi-feature fusion image sequence, preset ambient light intensity conditions, and preset ambient temperature conditions specifically includes: Each image frame in the multi-feature fused image sequence is segmented to obtain multiple candidate regions, and the average light intensity and average temperature of each candidate region are calculated. If the average light intensity is greater than a preset light intensity threshold and the average temperature is greater than a preset temperature threshold, the candidate region is determined as the suspected flame element. Extract the geometric morphology features, temperature distribution features, and edge structure features of the suspected flame primitive to obtain the feature vector corresponding to the suspected flame primitive.

3. The flame detection method integrating multimodal features according to claim 2, characterized in that, The process of establishing a dynamic spatiotemporal graph, using the suspected flame primitives as nodes, the Euclidean distance and temperature gradient between the suspected flame primitives as spatial edge weights, and the temperature change rate and area expansion rate of the suspected flame primitives within the preset time period as temporal edge weights, specifically includes: Within each image frame, all suspected flame primitives are treated as nodes; Within the same image frame, calculate the Euclidean distance and temperature gradient between the first suspected flame primitive and the second suspected flame primitive; if the Euclidean distance is less than a preset distance threshold, establish a spatial edge between the nodes corresponding to the first suspected flame primitive and the second suspected flame primitive, and calculate the spatial edge weight based on the Euclidean distance and the temperature gradient, wherein the first suspected flame primitive and the second suspected flame primitive are any two suspected flame primitives; Between two consecutive frames, the suspected flame primitives in the previous frame are associated and matched with the suspected flame primitives in the next frame. If the association and matching are successful, a time edge is established between the nodes corresponding to the suspected flame primitives in the previous frame and the suspected flame primitives in the next frame. The temperature change rate and area expansion rate of the suspected flame primitives between the two consecutive frames are calculated. The time edge weight is calculated based on the temperature change rate and the area expansion rate. The suspected flame primitive is any one of the suspected flame primitives.

4. The flame detection method integrating multimodal features according to claim 3, characterized in that, The step of analyzing the dynamic spatiotemporal graph using a graph neural network model to obtain the confidence level of the suspected flame primitive specifically includes: Based on the spatial edge weights, the temporal edge weights, and the feature vectors corresponding to each of the suspected flame primitives, the spatial domain features and temporal domain features of each suspected flame primitive are determined. The feature vector, spatial domain feature, and temporal domain feature corresponding to the suspected flame primitive are input into the graph neural network model to obtain the confidence level of the suspected flame primitive.

5. The flame detection method integrating multimodal features according to claim 4, characterized in that, The determination of the spatial domain features and temporal domain features of each suspected flame primitive based on the spatial edge weights, the temporal edge weights, and the feature vectors corresponding to each suspected flame primitive specifically includes: Identify the adjacent primitives of the suspected flame primitive in each image frame, calculate the product of the feature vector of each adjacent primitive and the corresponding spatial edge weight, and sum the products to obtain the spatial domain features of the suspected flame primitive; Identify the matching primitives of the suspicious flame primitives in the previous image frame, calculate the product of the feature vector of the matching primitives and the corresponding temporal edge weights, and obtain the temporal domain features of the suspicious flame primitives.

6. The flame detection method based on the fusion of multimodal features according to claim 1, characterized in that, After the step of analyzing the dynamic spatiotemporal graph using a graph neural network model to obtain the confidence level of the suspected flame primitives, and identifying suspected flame primitives with confidence levels exceeding a preset confidence threshold as real flame primitives, the method further includes: Based on the pixel coordinates of the real flame primitive in the multi-feature fused image sequence and the intrinsic and extrinsic parameters of the binocular gimbal, the physical coordinates of the real flame primitive in three-dimensional space are calculated. Based on the physical coordinate changes of the real flame element within a preset time period, calculate the direction and speed of fire spread. The fire spread direction, the fire spread speed, and the total number of real flame elements are integrated into a fire alarm message and sent to the fire emergency response terminal.

7. The flame detection method based on multimodal features according to claim 1, characterized in that, After the step of analyzing the dynamic spatiotemporal graph using a graph neural network model to obtain the confidence level of the suspected flame primitives, and identifying suspected flame primitives with confidence levels exceeding a preset confidence threshold as real flame primitives, the method further includes: Acquire the temperature time-series data of the real flame element, which includes the highest temperature, average temperature and temperature variance of the real flame element within a preset time period. Based on the temperature time series data, a time series prediction model is used to predict the temperature evolution trend of the real flame element, and a predicted temperature curve is obtained. Based on the slope and curvature of the predicted temperature curve, the combustion stages of the actual flame element are determined, including the initial ignition stage, the rapid development stage, and the stable combustion stage. Based on the combustion phases, the data acquisition frequency of the binocular gimbal is determined.

8. A detection system, characterized in that, The detection system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the detection system to perform the flame detection method of fusing multimodal features as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instruction is executed on the detection system, the detection system performs the flame detection method that integrates multimodal features as described in any one of claims 1-7.

10. A computer program product, characterized in that, When the computer program product is run on the detection system, the detection system performs the flame detection method that integrates multimodal features as described in any one of claims 1-7.

Images