Flame recognition and fire extinguishing system for indoor fire fighting

Abstract

The invention relates to the technical field of computer vision, and discloses a flame recognition and fire extinguishing system for indoor fire fighting, comprising: a binocular infrared imaging module for acquiring a thermal imaging image; the monocular visible light imaging module is used for acquiring a visible light image; the infrared stereoscopic vision module is used for carrying out processing and feature extraction on the thermal imaging images, carrying out feature aggregation to obtain an aggregated feature map, carrying out parallax optimization on the aggregated feature map, and carrying out calculation according to a parallax optimization result to obtain depth information of the thermal imaging images at different visual angles; the multi-mode stereoscopic vision fusion module is used for fusing the thermal imaging image with the depth information and the visible light image; the flame identification and fire behavior prediction module is used for performing flame detection according to the fused image, predicting a bounding box, confidence and a category label of the flame in the fused image, and predicting a future diffusion position of the flame, a diffusion scale of the flame and a diffusion trend of the flame; and the self-adaptive water gun spraying control module controls the water gun to execute spraying operation.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a flame recognition and extinguishing system for indoor fire protection. Background Technology

[0002] Existing indoor flame detection and extinguishing systems have numerous problems and shortcomings when performing tasks such as fire warning and fire suppression, as follows:

[0003] (1) Insufficient detection accuracy in indoor environments

[0004] Most current systems identify flame targets using visible light or infrared images, and measure flame distance using infrared or laser sensors. However, in complex indoor environments, situations where flames are obscured or interfered with by light sources can lead to numerous false positives and false negatives when relying solely on two-dimensional images for flame detection. Furthermore, during a fire, smoke and light can severely interfere with infrared and laser measurements of flame distance and height, making it difficult to accurately measure the flame's true location and distance.

[0005] (2) Insufficient precision control of water gun spray

[0006] Current firefighting equipment typically uses preset water nozzle spray parameters, lacking intelligent adjustment capabilities. This leads to water waste and inaccurate spraying, as it cannot adaptively adjust based on flame size, location, or temperature. Many systems, when controlling water nozzles to perform firefighting operations, cannot dynamically adjust their spray strategy according to the flame's location, temperature, and area, easily resulting in inaccurate firefighting or wasted water resources. This lack of intelligent firefighting strategy limits its responsiveness and efficiency in complex fire scenarios (such as fire spread or multi-point fires). Summary of the Invention

[0007] This invention provides a flame identification and extinguishing system for indoor fire protection, which can solve the problems in the prior art.

[0008] This invention provides a flame detection and extinguishing system for indoor fire protection, wherein the system includes:

[0009] The binocular infrared imaging module is used to acquire thermal imaging images from different perspectives. The information of each pixel in the thermal imaging image includes the temperature value.

[0010] A monocular visible light imaging module is used to acquire visible light images;

[0011] The infrared stereo vision module is used to process thermal imaging images from different perspectives, extract features from the processed images, aggregate the extracted features to obtain aggregated feature maps from different perspectives, optimize the disparity of the aggregated feature maps from different perspectives, and calculate the depth information of the thermal imaging images from different perspectives based on the disparity optimization results.

[0012] The multimodal stereo vision fusion module is used to fuse thermal imaging images with depth information and visible light images to obtain a fused image.

[0013] The flame recognition and fire prediction module is used to detect flames based on the fused image, predict the bounding box, confidence level and category label of the flame in the fused image, and predict the future spread location, spread scale and spread trend of the flame based on the bounding box.

[0014] The adaptive water gun spray control module is used to control the water gun to perform spraying operations based on the future spread position of the flame, the spread size of the flame, and the spread trend of the flame.

[0015] Preferably, the system further includes a gimbal for mounting the infrared imaging module and the visible light imaging module, the gimbal having at least two degrees of freedom.

[0016] Preferably, the infrared stereo vision module includes a processing module, a feature extraction module, a guided aggregation module, and an optimization module, wherein,

[0017] The processing module is used to filter thermal imaging images from different perspectives.

[0018] The feature extraction module is used to perform multi-scale feature extraction on the processed image using a fully convolutional neural network.

[0019] The guided aggregation module is used to aggregate the features of adjacent pixels through adaptive convolution operations to obtain aggregated feature maps from different perspectives.

[0020] The optimization module is used to generate a cost volume from the aggregated feature maps under different viewpoints through the disparity assumption, and to optimize the disparity estimation result by using convolution and cost aggregation operations. The depth information of the thermal imaging image under different viewpoints is calculated based on the disparity optimization result, where the cost volume represents the pixel matching cost under different disparities.

[0021] Preferably, the depth information of thermal imaging images at different viewpoints calculated based on the parallax optimization results includes:

[0022] Output a disparity map based on the disparity optimization results;

[0023] The final disparity value is calculated using the soft disparity regression method;

[0024] Depth information of thermal imaging images from different viewpoints is calculated based on the final disparity value.

[0025] Preferably, the depth information of thermal imaging images from different viewpoints is calculated using the following formula:

[0026]

[0027] Where Z(x,y) is the depth value of each pixel in the thermal image under different viewpoints, f is the focal length of the binocular infrared imaging module, B is the baseline length of the binocular infrared imaging module, and d(x,y) is the final disparity value.

[0028] Preferably, the thermal imaging image with depth information and the visible light image are fused to obtain the fused image, which includes:

[0029] Bicubic interpolation is used to perform interpolation on thermal imaging images with depth information so that the resolution of thermal imaging images with depth information matches the resolution of visible light images.

[0030] The visible light image is aligned with the differenced thermal imaging image. Using the visible light image as a guide image, a fusion algorithm with guided filtering is used to supplement visual detail information into the depth information of the differenced thermal imaging image, resulting in a fused image.

[0031] Preferably, flame detection is performed based on the fused image, predicting the bounding box, confidence level, and class label of the flame in the fused image, and predicting the future spread location, spread scale, and spread trend of the flame based on the bounding box, including:

[0032] The Yolov10 deep learning algorithm was used to detect flames in the fused image.

[0033] Using the Yolov10 deep learning algorithm, through feature extraction and regression layers, the position and bounding box B(x1,y1,x2,y2) of the flame in the fused image are predicted, and the bounding box, confidence score, and class label of the flame in the fused image are output.

[0034] After detecting a flame, the temperature T(x,y) of the image region containing the bounding box is obtained from the fused image. If the temperature T(x,y) is greater than the flame temperature threshold T, then... fire If the detected flame is identified as a high-risk ignition source, it is determined to be a general-risk ignition source; otherwise, it is determined to be a general-risk ignition source.

[0035] Flame contour information B is determined within the bounding box using temperature and 3D information. t ;

[0036] Flame profile information B is collected every predetermined time step t. t Time series data [B t-k , T t+n B t , T t The input is fed into a recurrent neural network (RNN), T t+n B is the predicted average flame temperature at a future time step t+n. t-k For the flame outline information at a past time step tk, T t The average temperature of the flame is given by time step t, k is the window length, and the flame profile information includes the flame position, flame area, and flame volume.

[0037] The hidden layers of an RNN continuously process time-series data through a recursive structure, memorizing the flame spread pattern at past time steps tk and extracting the dynamic features of flame spread.

[0038] The output layer of the RNN predicts the flame position, flame area, flame volume, and average temperature at a future time step t+n based on the extracted dynamic features. The flame position, flame area, flame volume, and average temperature at a future time step t+n reflect the future spread position, spread scale, and spread trend of the flame.

[0039] Preferably, controlling the water gun to perform the spraying operation based on the future spread location of the flame, the spread size of the flame, and the spread trend of the flame includes:

[0040] The Deep Deterministic Strategy Gradient (DDPG) algorithm is used to pre-adjust the water gun's spray parameters based on the future spread location, scale, and trend of the flame. The spray parameters include the spray angle θ, spray force F, and spray direction φ.

[0041] The water gun is controlled to perform the spraying operation according to the pre-adjusted spray parameters.

[0042] The above technical solution provides a flame identification and extinguishing system that integrates binocular ranging and monocular recognition. It detects flames using a visible light module while using a binocular infrared thermal imaging stereo vision system for ranging, improving the flame positioning accuracy in indoor environments with penetrating smoke and low visibility. This significantly enhances the environmental adaptability, flame detection and positioning accuracy, and extinguishing stability of the flame identification and extinguishing system during indoor fire fighting. Attached Figure Description

[0043] The accompanying drawings, which form part of this specification, are provided to further illustrate embodiments of the invention and, together with the textual description, explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0044] Figure 1 A schematic diagram of the overall process of a flame detection and extinguishing system for indoor fire protection according to an embodiment of the present invention is shown. Detailed Implementation

[0045] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0047] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0048] Figure 1 A schematic diagram of the overall process of a flame detection and extinguishing system for indoor fire protection according to an embodiment of the present invention is shown.

[0049] like Figure 1 As shown, this embodiment of the invention provides a flame detection and extinguishing system for indoor fire protection, wherein the system includes:

[0050] The binocular infrared imaging module is used to acquire thermal imaging images from different perspectives. The information of each pixel in the thermal imaging image includes the temperature value.

[0051] A monocular visible light imaging module is used to acquire visible light images;

[0052] The infrared stereo vision module is used to process thermal imaging images from different perspectives, extract features from the processed images, aggregate the extracted features to obtain aggregated feature maps from different perspectives, optimize the disparity of the aggregated feature maps from different perspectives, and calculate the depth information of the thermal imaging images from different perspectives based on the disparity optimization results.

[0053] The multimodal stereo vision fusion module is used to fuse thermal imaging images with depth information and visible light images to obtain a fused image.

[0054] The flame recognition and fire prediction module is used to detect flames based on the fused image, predict the bounding box, confidence level and category label of the flame in the fused image, and predict the future spread location, spread scale and spread trend of the flame based on the bounding box.

[0055] The adaptive water gun spray control module is used to control the water gun to perform spraying operations based on the future spread position of the flame, the spread size of the flame, and the spread trend of the flame.

[0056] The above technical solution provides a flame identification and extinguishing system that integrates binocular ranging and monocular recognition. It detects flames using a visible light module while using a binocular infrared thermal imaging stereo vision system for ranging, improving the flame positioning accuracy in indoor environments with penetrating smoke and low visibility. This significantly enhances the environmental adaptability, flame detection and positioning accuracy, and extinguishing stability of the flame identification and extinguishing system during indoor fire fighting.

[0057] The binocular infrared imaging module is a binocular thermal imaging camera, and the monocular visible light imaging module is a monocular visible light camera.

[0058] According to one embodiment of the present invention, the system further includes a gimbal for setting the infrared imaging module and the visible light imaging module, the gimbal having at least two degrees of freedom.

[0059] In other words, a gimbal can be a gimbal with at least two degrees of freedom (horizontal and pitch).

[0060] According to one embodiment of the present invention, the infrared stereo vision module includes a processing module, a feature extraction module, a guided aggregation module, and an optimization module, wherein,

[0061] The processing module is used to filter thermal imaging images from different perspectives.

[0062] The feature extraction module is used to perform multi-scale feature extraction on the processed image using a fully convolutional neural network.

[0063] The guided aggregation module is used to aggregate the features of adjacent pixels through adaptive convolution operations to obtain aggregated feature maps from different perspectives.

[0064] The optimization module is used to generate a cost volume from the aggregated feature maps under different viewpoints through the disparity assumption, and to optimize the disparity estimation result by using convolution and cost aggregation operations. The depth information of the thermal imaging image under different viewpoints is calculated based on the disparity optimization result, where the cost volume represents the pixel matching cost under different disparities.

[0065] According to one embodiment of the present invention, calculating the depth information of thermal imaging images from different viewpoints based on parallax optimization results includes:

[0066] Output a disparity map based on the disparity optimization results;

[0067] The final disparity value is calculated using the soft disparity regression method;

[0068] Depth information of thermal imaging images from different viewpoints is calculated based on the final disparity value.

[0069] According to one embodiment of the present invention, the depth information of thermal imaging images from different viewpoints is calculated using the following formula:

[0070]

[0071] Where Z(x,y) is the depth value of each pixel in the thermal image under different viewpoints, f is the focal length of the binocular infrared imaging module (each camera), B is the baseline length of the binocular infrared imaging module, and d(x,y) is the final parallax value.

[0072] According to one embodiment of the present invention, a fusion operation is performed on a thermal imaging image with depth information and a visible light image to obtain a fused image, including:

[0073] Bicubic interpolation is used to perform interpolation on thermal imaging images with depth information so that the resolution of thermal imaging images with depth information matches the resolution of visible light images.

[0074] The visible light image is aligned with the differenced thermal imaging image. Using the visible light image as a guide image, a fusion algorithm with guided filtering is used to supplement visual detail information into the depth information of the differenced thermal imaging image, resulting in a fused image.

[0075] According to one embodiment of the present invention, flame detection is performed based on the fused image, and the bounding box, confidence level, and class label of the flame in the fused image are predicted. Furthermore, the future spread location, spread scale, and spread trend of the flame are predicted based on the bounding box.

[0076] The Yolov10 deep learning algorithm was used to detect flames in the fused image.

[0077] Using the Yolov10 deep learning algorithm, through feature extraction and regression layers, the position and bounding box B(x1,y1,x2,y2) of the flame in the fused image are predicted, and the bounding box, confidence score, and class label of the flame in the fused image are output.

[0078] After detecting a flame, the temperature T(x,y) of the image region containing the bounding box is obtained from the fused image. If the temperature T(x,y) is greater than the flame temperature threshold T, then... fire If the detected flame is identified as a high-risk ignition source, it is determined to be a general-risk ignition source; otherwise, it is determined to be a general-risk ignition source.

[0079] Flame contour information B is determined within the bounding box using temperature and 3D information. t ;

[0080] Flame profile information B is collected every predetermined time step t. t Time series data [B t-k , T t+n B t , T t The input is fed into a recurrent neural network (RNN), T t+n B is the predicted average flame temperature at a future time step t+n. t-k For the flame outline information at a past time step tk, T t The average temperature of the flame is given by time step t, k is the window length, and the flame profile information includes the flame position, flame area, and flame volume.

[0081] The hidden layers of an RNN continuously process time-series data through a recursive structure, memorizing the flame spread pattern at past time steps tk and extracting the dynamic features of flame spread.

[0082] The output layer of the RNN predicts the flame position, flame area, flame volume, and average temperature at a future time step t+n based on the extracted dynamic features. The flame position, flame area, flame volume, and average temperature at a future time step t+n reflect the future spread position, spread scale, and spread trend of the flame.

[0083] According to one embodiment of the present invention, controlling a water gun to perform a spraying operation based on the future spread position of the flame, the spread size of the flame, and the spread trend of the flame includes:

[0084] The Deep Deterministic Strategy Gradient (DDPG) algorithm is used to pre-adjust the water gun's spray parameters based on the future spread location, scale, and trend of the flame. The spray parameters include the spray angle θ, spray force F, and spray direction φ.

[0085] The water gun is controlled to perform the spraying operation according to the pre-adjusted spray parameters.

[0086] This invention utilizes a fusion technology combining binocular infrared thermal imaging cameras and monocular visible light cameras to achieve real-time three-dimensional flame localization and temperature monitoring. It can accurately detect the shape, location, volume, and temperature of flames in complex indoor environments, significantly improving the accuracy and efficiency of fire detection. By combining the YOLOv10 target detection algorithm with multimodal image data, it accurately identifies flame areas and monitors the flame's temperature distribution in real time. Simultaneously, a recurrent neural network (RNN) predicts the flame's spread trend and temperature changes, providing early warning of fire development and intelligent decision support for firefighting operations. Furthermore, this invention can adaptively adjust the water jet's spray angle, spray force, and spray direction based on the real-time three-dimensional location, temperature, and area changes of the flame, ensuring precise water coverage of the flame area.

[0087] The flame identification and extinguishing system for indoor fire protection described in this invention is described below with reference to examples.

[0088] In this example, the flame recognition and extinguishing system can be deployed on hardware platforms such as embedded development boards and industrial control computers with image processing capabilities, equipped with binocular thermal imaging cameras (two infrared thermal imaging cameras), a monocular visible light camera, and a gimbal with at least two degrees of freedom (horizontal and pitch). The system establishes an infrared stereoscopic vision model (module) using the binocular thermal imaging cameras.

[0089] After the flame recognition and extinguishing system is activated, the infrared stereo vision model uses two infrared thermal imaging cameras to acquire thermal images (left and right infrared images) from different perspectives, and calculates the depth information in the scene using the parallax principle. To address the issue of high noise and low thermal imaging resolution in the images returned by the infrared thermal imaging cameras (referred to as heatmaps), the infrared stereo vision model introduces a guided aggregation module and a deep learning stereo matching network with a multi-scale feature extraction mechanism for parallax optimization. This improves the accuracy of parallax estimation during depth information calculation. The parallax optimization steps are as follows:

[0090] (1) Filter the input left and right infrared images to remove noise interference;

[0091] (2) Use a fully convolutional neural network to extract multi-scale features from the left and right images, mainly including texture, edge and region features with high robustness;

[0092] (3) Use the guided aggregation module to process features and aggregate the features of adjacent pixels through adaptive convolution operation, thereby improving the disparity estimation accuracy for sparse regions and noisy regions;

[0093] (4) Generate a cost volume from the feature maps of the left and right images using the disparity assumption, representing the pixel matching cost under different disparities. Employ convolution and cost aggregation operations to progressively optimize the disparity estimation results;

[0094] (5) Through the cost volume optimization process, a disparity map is output, and the final disparity value is accurately calculated using a soft disparity regression method. Specifically, the disparity map can be converted into depth information using the following formula:

[0095]

[0096] Where Z(x, y) is the depth value of each pixel in the image, f is the focal length of the camera in the stereo camera, B is the baseline length of the stereo camera, and d(x, y) is the optimized disparity value (the final disparity value).

[0097] After parallax optimization, the model calculates the depth information of the heatmap. The information of each pixel in the heatmap includes not only the temperature value, but also the corresponding spatial coordinates (X,Y,Z).

[0098] The multimodal stereo vision fusion module fuses thermal maps with depth information with visible light images to output images with depth information, temperature information, and high-resolution visual detail information.

[0099] During the fusion process, in order to compensate for the lack of flame outline and detail caused by the low resolution of the infrared thermal imaging camera, a fusion strategy with accuracy compensation and depth interpolation is adopted. The specific steps are as follows:

[0100] (1) Since infrared images with depth information are relatively coarse and have low resolution, bicubic interpolation is used to interpolate them to match the resolution of visible light images. The specific interpolation formula is as follows:

[0101]

[0102] Where w(i,j) is the weight, and Z′(x,y) is the thermal imaging image after interpolation (i.e., the high-resolution infrared image generated after interpolation).

[0103] (2) Align the visible light image with the infrared image, use the high-resolution visible light image as the guide image, and supplement the visual detail information (such as edges and textures) into the depth information of the infrared image through the fusion algorithm of the guide filter.

[0104] (3) The high-temperature areas in the fused image are dynamically presented through color-coded visualization.

[0105] The next step is to input the fused image containing multi-dimensional data such as 3D depth information, flame outline, and temperature distribution into the flame recognition and fire prediction module to detect and locate the flame and predict the development trend of the fire.

[0106] The flame recognition and fire prediction module can first use the YOLOv10 deep learning algorithm to detect flames in the fused image. YOLOv10 predicts the position and bounding box B(x1,y1,x2,y2) of the flame in the image through feature extraction layer and regression layer, and outputs the bounding box, confidence score and class label (flame) of the target in the image.

[0107] After the YOLOv10 algorithm identifies the target flame, the module can perform a preliminary analysis of the flame's intensity and hazard: obtain the temperature distribution T(x, y) within the bounding box region from the fused image, and set a temperature threshold T for the flame. fire If the temperature T(x, y) of the flame area is higher than the threshold, the identified target flame is determined to be a high-danger fire source; otherwise, it is determined to be a general-danger fire source.

[0108] By combining the YOLOv10 target detection algorithm with an AI-based flame spread prediction model, the detected flame spread trend can be monitored in real time during the fire extinguishing process.

[0109] The flame recognition and fire prediction module inputs the flame, after preliminary intensity and hazard assessment, into an AI-based flame spread prediction model to predict the flame spread trend and future temperature changes. The specific steps are as follows:

[0110] (1) Determine the flame contour information B within the bounding box output by the target detection algorithm using temperature and 3D information. t (Including flame location, area, volume, etc.);

[0111] (2) The system collects flame profile information every predetermined time step. Let the predetermined time step be t, and the average temperature of the flame be T. t The predicted range of the flame at future times (predicted flame profile information and predicted average temperature) is B. t+n and T t+n The time series data [B] t-k , T t+n B t , T t The input is fed into a recurrent neural network (RNN), where k is the window length;

[0112] (3) The hidden layer of the RNN continuously processes time-series data through its recursive structure, memorizes the flame diffusion pattern of the previous few steps (e.g., tk steps), and extracts the dynamic features of flame diffusion. The output layer of the RNN predicts the flame position, area, volume, and average temperature value at the future time step t+n based on the features, which can reflect: the future diffusion position (direction of movement) of the flame, the diffusion scale of the flame, and the diffusion trend of the flame (tending to be more intense or tending to be extinguished). The system sends these three evaluation results to the adaptive water gun spray control module to pre-adjust the water gun spray parameters, including: spray angle θ, spray force F, and spray direction φ.

[0113] The adaptive water cannon spray control module uses deep reinforcement learning technology to dynamically adjust the water cannon's spray parameters based on the flame's spread location, scale, and trend, ensuring precise and efficient fire extinguishing. The system continuously monitors the flame's state changes and optimizes the fire extinguishing strategy in real time based on feedback information until the flame is completely extinguished. To meet the continuous action requirements of water cannon spray control, a Deep Deterministic Policy Gradient (DDPG) algorithm is employed.

[0114] By introducing deep reinforcement learning algorithms and combining the three-dimensional position, temperature, and dynamic changes of the flame, the system can intelligently adjust the angle, force, and direction of the water gun during spraying.

[0115] Furthermore, the intelligent water gun spray control problem can be modeled as a Markov decision process (MDP), which includes elements such as state, action, reward, and policy, specifically:

[0116] 1. State Space S: The system constructs the state space based on the data provided by the flame target detection and temperature analysis module, including: flame position coordinates, area, volume, temperature, and the current spray parameters of the water gun;

[0117] 2. Action Space A: The system's actions are the spraying strategies adopted by the water gun at each time step, including adjusting the water gun's spray angle θ1, changing the spray force F1, and adjusting the spray direction φ1;

[0118] 3. Reward Function R: The core of the reward function is to measure the fire extinguishing effect. The goal is to extinguish the flames quickly and efficiently. The reward can be defined according to the following criteria:

[0119] ① If the flame temperature drops significantly, a positive reward is obtained.

[0120] ②If the flame area shrinks or goes out, a positive reward is obtained.

[0121] ③ If the fire extinguishing process fails to effectively reduce the temperature or shrink the flame area, a negative reward will be given.

[0122] 4. Strategy (α|s): The system selects the optimal action α based on the current state s through the policy network to maximize the cumulative reward.

[0123] During water jet spraying, the system synchronously monitors the temperature, area, and position changes of the flame in real time, and adjusts the water jet spray parameters based on feedback to ensure precise flame coverage. The feedback and adjustment process is as follows:

[0124] (1) Flame temperature decreases and area shrinks: The system continues to maintain the current injection parameters until the flame is completely extinguished;

[0125] (2) The flame temperature does not decrease significantly or the area expands: The system dynamically adjusts the spray angle θ1 and force F1 based on the feedback data to optimize the spray coverage of the water flow;

[0126] (3) Flame temperature and area fluctuation: The system determines whether the flame is reigniting. If an abnormality is detected, the system will further increase the spray force to ensure that the flame is completely extinguished.

[0127] As can be seen from the above embodiments, the flame detection and extinguishing system for indoor fire protection described in this invention has at least the following advantages compared to the prior art:

[0128] (1) The system described in this invention achieves accurate three-dimensional positioning and real-time temperature monitoring of indoor flames by fusing multimodal stereo vision technology and combining a binocular infrared thermal imaging camera with a monocular visible light camera. Even in environments with abundant smoke or poor lighting conditions, the system can still effectively identify flames and provide accurate spatial information. This multimodal fusion technology improves the accuracy and reliability of indoor fire identification, significantly reduces false alarms, and is particularly suitable for high-frequency monitoring scenarios in indoor fire protection.

[0129] (2) The system described in this invention introduces dynamic monitoring of flame temperature and analysis of diffusion trends, adapting to the complexity of indoor fires. Unlike traditional fixed-threshold detection methods, this system can dynamically determine whether the fire is spreading or weakening based on changes in flame temperature and space. This function ensures that the system can issue timely warnings and initiate emergency response before the fire spreads to critical areas, making it particularly suitable for indoor fire protection scenarios in enclosed or multi-story buildings.

[0130] (3) The system described in this invention automatically adjusts the water jet spray strategy through an intelligent fire extinguishing control algorithm based on deep reinforcement learning, thereby improving the fire extinguishing efficiency of indoor fire fighting. When a flame is detected, the system optimizes the spray trajectory, angle, and intensity of the water jet based on parameters such as the flame's location, size, and temperature through deep reinforcement learning, ensuring that the water flow accurately covers the flame area. During the spraying process, the system continuously monitors the temperature changes of the flame and dynamically adjusts the fire extinguishing intensity based on feedback until the flame is completely extinguished.

[0131] In summary, this invention directly enhances the performance of flame recognition and extinguishing systems for indoor fire protection, particularly in terms of recognition accuracy, scene stability, and extinguishing strategy flexibility, providing significant improvements.

[0132] In the description of this invention, it should be understood that the orientation or positional relationship indicated by directional terms such as "front, back, up, down, left, right", "horizontal, vertical, horizontal" and "top, bottom" is generally based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing this invention and simplifying the description. Unless otherwise stated, these directional terms do not indicate or imply that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the scope of protection of this invention; the directional terms "inner" and "outer" refer to the inner and outer contours relative to the outline of each component itself.

[0133] For ease of description, spatial relative terms such as "above," "on top of," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation beyond the orientation of the device as described in the figures. For example, if the device in the figures were inverted, a device described as "above" or "on top of" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0134] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, the above terms have no special meaning and therefore should not be construed as limiting the scope of protection of this invention.

[0135] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A flame detection and extinguishing system for indoor fire protection, characterized in that, The system includes: The binocular infrared imaging module is used to acquire thermal imaging images from different perspectives. The information of each pixel in the thermal imaging image includes the temperature value. A monocular visible light imaging module is used to acquire visible light images; The infrared stereo vision module is used to process thermal imaging images from different perspectives, extract features from the processed images, aggregate the extracted features to obtain aggregated feature maps from different perspectives, optimize the disparity of the aggregated feature maps from different perspectives, and calculate the depth information of the thermal imaging images from different perspectives based on the disparity optimization results. The multimodal stereo vision fusion module is used to fuse thermal imaging images with depth information and visible light images to obtain a fused image. The flame recognition and fire prediction module is used to detect flames based on the fused image, predict the bounding box, confidence level and category label of the flame in the fused image, and predict the future spread location, spread scale and spread trend of the flame based on the bounding box. The adaptive water gun spray control module is used to control the water gun to perform spraying operations based on the future spread position of the flame, the spread size of the flame, and the spread trend of the flame.

2. The system according to claim 1, characterized in that, The system also includes a gimbal for mounting the infrared imaging module and the visible light imaging module, the gimbal having at least two degrees of freedom.

3. The system according to claim 2, characterized in that, The infrared stereo vision module includes a processing module, a feature extraction module, a guided aggregation module, and an optimization module, wherein... The processing module is used to filter thermal imaging images from different perspectives. The feature extraction module is used to perform multi-scale feature extraction on the processed image using a fully convolutional neural network. The guided aggregation module is used to aggregate the features of adjacent pixels through adaptive convolution operations to obtain aggregated feature maps from different perspectives. The optimization module is used to generate a cost volume from the aggregated feature maps under different viewpoints through the disparity assumption, and to optimize the disparity estimation result by using convolution and cost aggregation operations. The depth information of the thermal imaging image under different viewpoints is calculated based on the disparity optimization result, where the cost volume represents the pixel matching cost under different disparities.

4. The system according to claim 3, characterized in that, The depth information of thermal imaging images at different viewpoints calculated based on the parallax optimization results includes: Output a disparity map based on the disparity optimization results; The final disparity value is calculated using the soft disparity regression method; Depth information of thermal imaging images from different viewpoints is calculated based on the final disparity value.

5. The system according to claim 4, characterized in that, The depth information of thermal imaging images from different viewpoints is calculated using the following formula: Where Z(x,y) is the depth value of each pixel in the thermal image under different viewpoints, f is the focal length of the binocular infrared imaging module, B is the baseline length of the binocular infrared imaging module, and d(x,y) is the final disparity value.

6. The system according to claim 5, characterized in that, The fusion operation between a thermal imaging image with depth information and a visible light image yields the following fused images: Bicubic interpolation is used to perform interpolation on thermal imaging images with depth information so that the resolution of thermal imaging images with depth information matches the resolution of visible light images. The visible light image is aligned with the differenced thermal imaging image. Using the visible light image as a guide image, a fusion algorithm with guided filtering is used to supplement visual detail information into the depth information of the differenced thermal imaging image, resulting in a fused image.

7. The system according to claim 6, characterized in that, Flame detection is performed based on the fused image. The bounding box, confidence score, and class label of the flame in the fused image are predicted. Furthermore, based on the bounding box, the future spread location, scale, and trend of the flame are predicted, including: The Yolov10 deep learning algorithm was used to detect flames in the fused image. Using the Yolov10 deep learning algorithm, through feature extraction and regression layers, the position and bounding box B(x1,y1,x2,y2) of the flame in the fused image are predicted, and the bounding box, confidence score, and class label of the flame in the fused image are output. After detecting a flame, the temperature T(x,y) of the image region containing the bounding box is obtained from the fused image. If the temperature T(x,y) is greater than the flame temperature threshold T, then... fire If the detected flame is identified as a high-risk ignition source, it is determined to be a general-risk ignition source; otherwise, it is determined to be a general-risk ignition source. Flame contour information B is determined within the bounding box using temperature and 3D information. t ; Flame profile information B is collected every predetermined time step t. t Time series data [B t-k T t+n B t T t The input is fed into a recurrent neural network (RNN), T t+n B is the predicted average flame temperature for a future time step t+n. t-k For the flame outline information at a past time step tk, T t The flame temperature is the average temperature of the flame at time step t, k is the window length, and the flame profile information includes the flame position, flame area, and flame volume. The hidden layers of an RNN continuously process time-series data through a recursive structure, memorizing the flame spread pattern at past time steps tk and extracting the dynamic features of flame spread. The output layer of the RNN predicts the flame position, flame area, flame volume, and average temperature at a future time step t+n based on the extracted dynamic features. The flame position, flame area, flame volume, and average temperature at a future time step t+n reflect the future spread position, spread scale, and spread trend of the flame.

8. The system according to claim 7, characterized in that, Controlling the water jet spraying operation based on the future location, scale, and trend of the flame spread includes: The Deep Deterministic Strategy Gradient (DDPG) algorithm is used to pre-adjust the water gun's spray parameters based on the future spread location, scale, and trend of the flame. The spray parameters include the spray angle θ, spray force F, and spray direction φ. The water gun is controlled to perform the spraying operation according to the pre-adjusted spray parameters.

Images