Fire source detection, identification and alarm system and method of intelligent fire-fighting robot

Abstract

The invention relates to the technical field of fire source detection, and discloses a fire source detection identification and alarm system and method for an intelligent fire-fighting robot, and the method comprises the steps: employing a deep learning target detection model, carrying out the high-precision identification of a fire source based on the data of a multi-modal sensor, extracting the initial three-dimensional position coordinates of the fire source, and providing basic data for subsequent tracking; besides, the fire source is dynamically tracked based on a Kalman filtering algorithm, the fire source spreading trend is predicted in combination with the motion state of the fire source, a fire behavior development prediction map is further constructed, and accurate pre-judgment of the fire behavior spreading range is achieved. And according to a fire source identification result and a fire behavior development prediction map, a fire alarm strategy is optimized, an alarm level, a personnel evacuation route and fire-fighting resource allocation are dynamically adjusted, and the intelligent degree of fire emergency response is improved. According to the technical scheme, the accuracy and real-time performance of fire monitoring can be effectively improved, the intelligent level of fire prevention and control and emergency disposal is improved, and casualties and property losses are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of fire source detection, and in particular to a fire source detection, identification and alarm system and method for an intelligent fire-fighting robot. Background Art

[0002] Fire detection and early warning systems play a vital role in fire safety. Traditional fire detection equipment primarily consists of smoke detectors, temperature sensors, and infrared detectors. These devices are typically installed in fixed locations and detect fires by measuring smoke concentration, temperature changes, or infrared radiation in the air. Consequently, traditional fire detection systems suffer from limited coverage, high false alarm rates, limited response speed, and the inability to accurately locate fire sources.

[0003] To address the limitations of traditional fire detection methods, intelligent firefighting robots have emerged as an emerging technology in fire prevention, firefighting, and rescue efforts in recent years. These robots can navigate complex environments, acquire fire source information through multimodal sensors, and incorporate artificial intelligence technology to identify and track fire sources. However, existing intelligent firefighting robots still suffer from low fire source identification accuracy and delayed fire tracking. This results in incomplete alarm information and hinders the timeliness of firefighting decisions. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a fire source detection, identification and alarm system and method for an intelligent fire-fighting robot, so as to solve the problems of low fire source identification accuracy and relatively lagging fire tracking information in fire source detection of existing intelligent fire-fighting robots, resulting in incomplete alarm information and affecting the timeliness of fire-fighting decisions.

[0005] The first aspect of the present invention discloses a fire source detection, identification and alarm system for an intelligent fire-fighting robot, the system comprising a fire source data management module, a fire source identification module, a fire source tracking module and an alarm management module;

[0006] The fire source data management module is used to collect and process multimodal sensor data; the multimodal sensor data is used for fire source detection;

[0007] The fire source identification module includes a deep learning target detection model, which performs fire source identification operations based on multimodal sensor data through the deep learning target detection model to obtain a fire source identification result. When the fire source identification result indicates that a fire source exists, the initial three-dimensional position coordinates of the fire source are extracted, the fire source identification result is sent to the alarm management module, and the initial three-dimensional position coordinates of the fire source are sent to the fire source tracking module.

[0008] The fire source tracking module is used to dynamically track the fire source based on its initial three-dimensional position coordinates using the Kalman filter algorithm and predict the fire spread trend. Based on the predicted fire spread trend, a fire development prediction map is constructed and sent to the alarm management module.

[0009] The alarm management module is used to issue fire alarms based on the fire source identification results and the fire development prediction map.

[0010] Furthermore, the multimodal sensor data includes depth camera data, thermal imaging sensor data, and gas sensor data;

[0011] The fire source identification operation includes a flame feature extraction operation and a flame target detection operation; the flame feature extraction operation includes the following steps:

[0012] S11. Convert the depth camera data from RGB channels to HSV color space, and extract flame color features based on the converted HSV color space;

[0013] S12. Extracting temperature features based on the thermal imaging sensor data, and screening a high-temperature region above a set threshold based on the extracted temperature features as a first high-temperature region;

[0014] S13. Extract combustion product characteristics based on gas sensor data, and eliminate non-fire source high-temperature objects in the first high-temperature area according to the combustion product characteristic information to obtain a second high-temperature area.

[0015] Furthermore, the flame target detection operation includes:

[0016] The flame color features, the second high-temperature area, and the combustion product features are input into the deep learning target detection model, and the flame target bounding box is output by the deep learning target detection model.

[0017] Furthermore, the operation of extracting the initial three-dimensional position coordinates of the fire source includes calculating the initial three-dimensional position coordinates of the fire source according to the flame target bounding box and depth information of the depth camera.

[0018] Furthermore, the process of dynamically tracking the fire source based on the initial three-dimensional position coordinates of the fire source using the Kalman filter algorithm includes the following steps:

[0019] S21. Set the initial fire source state; the initial fire source state includes the initial three-dimensional position coordinates, velocity and acceleration of the fire source;

[0020] S22. Predicting the future fire source state based on the initial fire source state through the state transition matrix, and optimizing the predicted future fire source state through the process noise covariance matrix to obtain the first fire source state;

[0021] S23. Calculate the prediction error based on the real-time multimodal sensor data through the observation matrix, and perform a Kalman gain update operation on the first fire source state according to the prediction error to obtain the second fire source state;

[0022] S24. Output the three-dimensional position coordinates of the current fire source based on the second fire source state, and use the three-dimensional position coordinates of the current fire source as the new initial three-dimensional position coordinates to repeat steps S21 to S24;

[0023] S25. Use the three-dimensional position coordinates of the current fire source output each time as the basic data of the fire source trajectory.

[0024] Furthermore, the fire source spread trend prediction operation includes predicting the fire source spread trend based on the fire source trajectory basic data by using an extended Kalman filter algorithm.

[0025] Furthermore, the multimodal sensor data also includes environmental monitoring data; and the method of predicting the fire source spread trend based on the fire source trajectory basic data using the extended Kalman filter algorithm includes the following steps:

[0026] S31. Establish the fire spread state equation based on the fire source trajectory basic data;

[0027] S32. Calculate the direction of fire spread based on environmental monitoring data and thermal imaging sensor data;

[0028] S33. Predict the spread area of the fire source at different time steps based on the diffusion direction of the fire source through the fire source spread state equation, and generate fire source spread trend prediction information based on the predicted spread area; the fire source spread trend prediction information includes the future coverage range of the fire source, center position change, combustion intensity trend and fire threat area.

[0029] Furthermore, constructing a fire source development map based on the fire source spread trend includes the following steps:

[0030] S41. Record and store historical movement information of fire sources, including historical fire source locations and combustion intensity change information;

[0031] S42. Construct a fire development prediction map based on historical movement information and fire source spread trend prediction information, and send the fire development prediction map to the alarm management module.

[0032] Furthermore, the fire source development prediction path and the corresponding fire source spread trend prediction information are marked in the fire development prediction map.

[0033] The second aspect of the present invention discloses a fire source detection, identification and alarm method for an intelligent firefighting robot, which is applied to the system disclosed in the first aspect and includes:

[0034] Collecting and processing multimodal sensor data; the multimodal sensor data is used for fire source detection;

[0035] A fire source identification operation is performed based on multimodal sensor data using a deep learning target detection model to obtain a fire source identification result; when the fire source identification result indicates that a fire source exists, the initial three-dimensional position coordinates of the fire source are extracted;

[0036] The Kalman filter algorithm is used to dynamically track the fire source based on its initial three-dimensional position coordinates and predict the fire spread trend. A fire development prediction map is constructed based on the predicted fire spread trend.

[0037] Fire alarms are issued based on fire source identification results and fire development prediction maps.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention realizes efficient and accurate fire source detection, identification, tracking and alarming through the coordinated work of the fire source data management module, the fire source identification module, the fire source tracking module and the alarm management module. Among them, the fire source identification module adopts a deep learning target detection model to perform high-precision identification of the fire source based on multimodal sensor data, and extracts the initial three-dimensional position coordinates of the fire source to provide basic data for subsequent tracking; the fire source tracking module dynamically tracks the fire source based on the Kalman filter algorithm, predicts the fire source spread trend based on the fire source movement state, and further constructs a fire development prediction map to achieve accurate prediction of the fire spread range; the alarm management module optimizes the fire alarm strategy based on the fire source identification results and the fire development prediction map, dynamically adjusts the alarm level, personnel evacuation route, and fire resource allocation, and improves the intelligence level of fire emergency response. Through the technical solution of the present invention, the accuracy and real-time performance of fire monitoring can be effectively improved, the intelligence level of fire prevention and control and emergency response can be improved, and casualties and property losses can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of the application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0041] Figure 1 This is a structural diagram of a fire source detection, identification and alarm system of an intelligent firefighting robot disclosed in Embodiment 1 of the present invention;

[0042] Figure 2 The present invention further discloses a flow chart of a method for detecting, identifying, and alarming a fire source by an intelligent firefighting robot. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0044] Example 1

[0045] The first aspect of the present invention discloses a fire source detection, identification and alarm system for an intelligent fire fighting robot. Figure 1 , Figure 1 This is a schematic diagram of the structure of a fire source detection, identification and alarm system for an intelligent firefighting robot disclosed in an embodiment of the present invention, which includes a fire source data management module, a fire source identification module, a fire source tracking module and an alarm management module;

[0046] The fire source data management module is used to collect and process multimodal sensor data; the multimodal sensor data is used for fire source detection;

[0047] The fire source identification module includes a deep learning target detection model, which performs fire source identification operations based on multimodal sensor data through the deep learning target detection model to obtain a fire source identification result. When the fire source identification result indicates that a fire source exists, the initial three-dimensional position coordinates of the fire source are extracted, the fire source identification result is sent to the alarm management module, and the initial three-dimensional position coordinates of the fire source are sent to the fire source tracking module.

[0048] The fire source tracking module is used to dynamically track the fire source based on its initial three-dimensional position coordinates using the Kalman filter algorithm and predict the fire spread trend. Based on the predicted fire spread trend, a fire development prediction map is constructed and sent to the alarm management module.

[0049] The alarm management module is used to issue fire alarms based on the fire source identification results and the fire development prediction map.

[0050] Furthermore, multimodal sensor data includes but is not limited to depth camera data, thermal imaging sensor data, gas sensor data, and environmental monitoring data.

[0051] Specifically, the multimodal sensor module in the embodiment of the present invention is integrated into the main body of the intelligent fire-fighting robot, and performs real-time monitoring of the surrounding fire-fighting situation while the robot is moving.

[0052] The depth camera can preferably be an RGB-D depth camera. The depth camera data is used to extract flame characteristics such as flame color, shape, and texture, as well as obtain information about the depth of the fire source. Thermal imaging sensor data is used to detect high-temperature areas, while gas sensor data is used to detect combustion product characteristics such as carbon monoxide, smoke particle concentration, and combustible gas composition in the air. In this embodiment, environmental monitoring data refers to environmental data such as air humidity, temperature, and wind speed in the area monitored by the firefighting robot.

[0053] The fire source identification operation includes the flame feature extraction operation and the flame target detection operation. The flame feature extraction operation includes the following steps:

[0054] S11. Convert the depth camera data from RGB channels to HSV color space, and extract flame color features based on the converted HSV color space;

[0055] S12. Extracting temperature features based on the thermal imaging sensor data, and screening a high-temperature region above a set threshold based on the extracted temperature features as a first high-temperature region;

[0056] S13. Extract combustion product characteristics based on gas sensor data, and eliminate non-fire source high-temperature objects in the first high-temperature area according to the combustion product characteristic information to obtain a second high-temperature area.

[0057] Understandably, during the fire source identification process, some objects (such as electric heating equipment, sunlit areas, and the surfaces of industrial equipment) will display higher temperatures, but they are not actual fire sources. Therefore, relying solely on thermal imaging sensor data may lead to false alarms, while gas sensor data (such as smoke concentration, CO2 / CO, methane and other combustible gas concentrations) can help eliminate these false alarms. Real flames are often accompanied by combustion products (such as smoke and CO2 / CO), while ordinary high-temperature objects do not release these gases. If a high-temperature area is detected but no combustion products are detected, it is likely a non-fire source high-temperature object, such as a heating device, and should be eliminated.

[0058] The present invention can reduce the false alarm rate and improve the accuracy of fire source identification by using gas sensor data to assist thermal imaging sensor data in screening high-temperature areas.

[0059] Furthermore, the flame target detection operation includes:

[0060] The flame color features, the second high-temperature area, and the combustion product features are input into the deep learning target detection model, and the flame target bounding box is output by the deep learning target detection model.

[0061] Specifically, flame color features, high-temperature area and combustion product features are extracted based on the collected historical depth camera images, thermal imaging sensor data and gas sensor data, and the flame area is annotated based on the collected historical RGB images, thermal imaging sensor data and gas sensor data. The real bounding box of the flame target is generated and used as training data for the deep learning target detection model to train the model based on the Faster R-CNN deep learning network.

[0062] Preferably, the shape and texture features of the flame are extracted based on the depth camera image data, and the fire source category is annotated in combination with the gas sensor data. The fire source category is then identified using a deep learning target detection model. In this embodiment, the fire source category represents the combustion category of the fire, including but not limited to open flames, hidden fire sources (such as furniture afterburning), flash fire (such as flammable materials igniting instantly in a high temperature environment), smoldering (such as a slow-burning cable fire), and deflagration (such as the accumulation and rapid combustion of combustible gases).

[0063] After identifying the specific fire source category, different alarm strategies can be set based on the fire source category, such as timely alarms for open flames, hidden fire sources, flash fires, and smoldering fires and providing corresponding flame levels, while special alarms are set for deflagrations and emergency evacuation measures are immediately implemented.

[0064] Furthermore, the operation of extracting the initial three-dimensional position coordinates of the fire source includes calculating the initial three-dimensional position coordinates of the fire source according to the flame target bounding box and depth information of the depth camera.

[0065] Specifically, the flame target bounding box is a two-dimensional bounding box, i.e., a rectangular box (x min ,y min ,x manx ,y max ), (x min ,y min ) is the coordinate of the upper left corner of the rectangle, (x manx ,y max ) is the coordinate of the lower right corner of the rectangular box. Combined with the depth information D of the depth camera, the initial three-dimensional position coordinates (X, Y, Z) of the fire source are calculated:

[0066] X=(x i -c x )×D / f x

[0067] Y=(y i -c y )×D / f y

[0068] Z=D

[0069] Among them, x i 、yi is the pixel coordinate in the 2D bounding box, c x 、c y is the optical center of the camera, f x 、f y is the focal length of the camera.

[0070] Furthermore, the process of dynamically tracking the fire source based on the initial three-dimensional position coordinates of the fire source using the Kalman filter algorithm includes the following steps:

[0071] S21. Set the initial fire source state; the initial fire source state includes the initial three-dimensional position coordinates, velocity and acceleration of the fire source;

[0072] S22. Predicting the future fire source state based on the initial fire source state through the state transition matrix, and optimizing the predicted future fire source state through the process noise covariance matrix to obtain the first fire source state;

[0073] S23. Calculate the prediction error based on the real-time multimodal sensor data through the observation matrix, and perform a Kalman gain update operation on the first fire source state according to the prediction error to obtain the second fire source state;

[0074] S24. Output the three-dimensional position coordinates of the current fire source based on the second fire source state, and use the three-dimensional position coordinates of the current fire source as the new initial three-dimensional position coordinates to repeat steps S21 to S24;

[0075] S25. Use the three-dimensional position coordinates of the current fire source output each time as the basic data of the fire source trajectory.

[0076] Specifically, the fire source velocity is calculated based on the change in fire source position and the time interval between two adjacent image frames, and the acceleration is calculated based on the velocity change. In this embodiment, a process noise covariance matrix is set to simulate the uncertainty caused by sensor noise or flame jump, which is mainly used to affect the acceleration term.

[0077] In step S23 , the prediction error is calculated by comparing the data actually monitored by the sensor with the data predicted by the Kalman filter.

[0078] In a fire environment, flames are often unstable and rapidly moving, making them difficult to accurately track using traditional methods. Kalman filtering improves the accuracy of fire source tracking through an iterative prediction-update process, enabling firefighting robots to more accurately determine the spread of a fire.

[0079] Furthermore, the fire source spread trend prediction operation includes predicting the fire source spread trend based on the fire source trajectory basic data using an extended Kalman filter algorithm, specifically including the following steps:

[0080] S31. Establish the fire spread state equation based on the fire source trajectory basic data;

[0081] S32. Calculate the direction of fire spread based on environmental monitoring data and thermal imaging sensor data;

[0082] S33. Predict the spread area of the fire source at different time steps based on the diffusion direction of the fire source through the fire source spread state equation, and generate fire source spread trend prediction information based on the predicted spread area; the fire source spread trend prediction information includes the future coverage range of the fire source, center position change, combustion intensity trend and fire threat area.

[0083] Preferably, the fire spread state equation is established on the physical model, specifically:

[0084] x t+1 =f(x t ,u t )+w t

[0085] Among them, x t+1 is the predicted next fire spread state, including but not limited to the location of the fire source and the intensity of combustion; f(x t ,u t ) is the fire source spread state transfer equation, which depends on the current fire source spread state x t and the control variable u t , where the controlled variables include at least the direction of fire spread, distribution of combustion materials, etc.; t is the process noise, which indicates the uncertainty of the fire source.

[0086] In a real fire, the spread of fire is affected by factors such as air flow, temperature gradient, and the distribution of combustibles. Therefore, the direction of fire spread is calculated using environmental monitoring data and thermal imaging sensor data, specifically including:

[0087] Calculate the temperature change rate around the fire source based on the thermal imaging sensor data, that is, the temperature gradient:

[0088]

[0089] in, is the temperature gradient vector, They respectively represent the changing trends of temperature in different directions; they respectively represent the changing rates of temperature in the x, y, and z axis directions.

[0090] Obtain wind speed v based on environmental monitoring data w and wind direction θ w , calculate the effect of wind on the spread of fire source through wind speed and wind direction D w :

[0091] Dw =v w ·(cosθ w ,sinθ w )

[0092]

[0093] Among them, D f is the final fire source diffusion direction, α and β are weight coefficients, and the sum is 1. Preferably, α is 0.3 and β is 0.7.

[0094] The spread trend of a fire source is affected by complex nonlinear environmental factors and is subject to great uncertainty. This embodiment integrates multiple environmental factors through the extended Kalman filter (EKF) algorithm to improve the robustness of fire source spread prediction. It can also continuously adjust the prediction results as new data is input, adapting to environmental changes in real time.

[0095] Furthermore, constructing a fire source development map based on the fire source spread trend includes the following steps:

[0096] S41. Record and store historical movement information of fire sources, including historical fire source locations and combustion intensity change information;

[0097] S42. Construct a fire development prediction map based on historical movement information and fire source spread trend prediction information, and send the fire development prediction map to the alarm management module.

[0098] Furthermore, the fire source development prediction path and the corresponding fire source spread trend prediction information are marked on the fire development prediction map.

[0099] After receiving the fire spread prediction map, the alarm management module adjusts the alarm strategy based on the map information. Specifically, after identifying high-risk areas where the fire is about to spread (such as densely populated areas, areas where flammable and explosive materials are stored, and areas where key equipment is located), the alarm level of different areas is adjusted according to the speed of fire development. Customized alarm information is sent to different groups of people within the fire spread range (such as people in escape areas and building management personnel). Evacuation routes are dynamically adjusted based on the predicted path of the fire source development, thereby improving the response efficiency of the fire protection system and minimizing casualties and property losses.

[0100] Example 2

[0101] The second aspect of the present invention discloses a fire source detection, identification and alarm method of an intelligent fire fighting robot, please refer to Figure 2 , Figure 2 1 is a flow chart of a fire source detection, identification and alarm method of an intelligent firefighting robot disclosed in another embodiment of the present invention, the method comprising:

[0102] Collecting and processing multimodal sensor data; the multimodal sensor data is used for fire source detection;

[0103] A fire source identification operation is performed based on multimodal sensor data using a deep learning target detection model to obtain a fire source identification result; when the fire source identification result indicates that a fire source exists, the initial three-dimensional position coordinates of the fire source are extracted;

[0104] The Kalman filter algorithm is used to dynamically track the fire source based on its initial three-dimensional position coordinates and predict the fire spread trend. A fire development prediction map is constructed based on the predicted fire spread trend.

[0105] Fire alarms are issued based on fire source identification results and fire development prediction maps.

[0106] Furthermore, the multimodal sensor data includes depth camera data, thermal imaging sensor data, and gas sensor data.

[0107] The fire source identification operation includes the flame feature extraction operation and the flame target detection operation. The flame feature extraction operation includes the following steps:

[0108] S11. Convert the depth camera data from RGB channels to HSV color space, and extract flame color features based on the converted HSV color space;

[0109] S12. Extracting temperature features based on the thermal imaging sensor data, and screening a high-temperature region above a set threshold based on the extracted temperature features as a first high-temperature region;

[0110] S13. Extract combustion product characteristics based on gas sensor data, and eliminate non-fire source high-temperature objects in the first high-temperature area according to the combustion product characteristic information to obtain a second high-temperature area.

[0111] Furthermore, the flame target detection operation includes:

[0112] The flame color features, the second high-temperature area, and the combustion product features are input into the deep learning target detection model, and the flame target bounding box is output by the deep learning target detection model.

[0113] Furthermore, the operation of extracting the initial three-dimensional position coordinates of the fire source includes calculating the initial three-dimensional position coordinates of the fire source according to the flame target bounding box and depth information of the depth camera.

[0114] Furthermore, the process of dynamically tracking the fire source based on the initial three-dimensional position coordinates of the fire source using the Kalman filter algorithm includes the following steps:

[0115] S21. Set the initial fire source state; the initial fire source state includes the initial three-dimensional position coordinates, velocity and acceleration of the fire source;

[0116] S22. Predicting the future fire source state based on the initial fire source state through the state transition matrix, and optimizing the predicted future fire source state through the process noise covariance matrix to obtain the first fire source state;

[0117] S23. Calculate the prediction error based on the real-time multimodal sensor data through the observation matrix, and perform a Kalman gain update operation on the first fire source state according to the prediction error to obtain the second fire source state;

[0118] S24. Output the three-dimensional position coordinates of the current fire source based on the second fire source state, and use the three-dimensional position coordinates of the current fire source as the new initial three-dimensional position coordinates to repeat steps S21 to S24;

[0119] S25. Use the three-dimensional position coordinates of the current fire source output each time as the basic data of the fire source trajectory.

[0120] Furthermore, the multimodal sensor data also includes environmental monitoring data. The operation of predicting the fire source spread trend includes predicting the fire source spread trend based on the fire source trajectory basic data using an extended Kalman filter algorithm, specifically including the following steps:

[0121] S31. Establish the fire spread state equation based on the fire source trajectory basic data;

[0122] S32. Calculate the direction of fire spread based on environmental monitoring data and thermal imaging sensor data;

[0123] S33. Use the fire spread state equation to predict the fire spread area at different time steps based on the fire spread direction. Generate fire spread trend prediction information based on the predicted spread area. Fire spread trend prediction information includes the future coverage of the fire source, changes in the center location, combustion intensity trends, and fire threat areas.

[0124] Furthermore, constructing a fire source development map based on the fire source spread trend includes the following steps:

[0125] S41. Record and store historical movement information of fire sources, including historical fire source locations and combustion intensity change information;

[0126] S42. Construct a fire development prediction map based on historical movement information and fire source spread trend prediction information.

[0127] Furthermore, the fire source development prediction path and the corresponding fire source spread trend prediction information are marked on the fire development prediction map.

[0128] It should be noted that the specific implementation process of Example 2 is similar to that of Example 1 and will not be repeated in this embodiment.

[0129] Finally, it should be noted that the fire source detection, identification and alarm system and method of an intelligent fire-fighting robot disclosed in the embodiment of the present invention only discloses a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that it is still possible to modify the technical solutions recorded in the aforementioned embodiments, or to replace some of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A fire source detection, identification and alarm system for an intelligent fire-fighting robot, characterized in that: The system includes a fire source data management module, a fire source identification module, a fire source tracking module and an alarm management module; The fire source data management module is used to collect and process multimodal sensor data; the multimodal sensor data is used for fire source detection; The fire source identification module includes a deep learning target detection model, which performs fire source identification operations based on multimodal sensor data through the deep learning target detection model to obtain a fire source identification result. When the fire source identification result indicates that a fire source exists, the initial three-dimensional position coordinates of the fire source are extracted, the fire source identification result is sent to the alarm management module, and the initial three-dimensional position coordinates of the fire source are sent to the fire source tracking module. The fire source tracking module is used to dynamically track the fire source based on its initial three-dimensional position coordinates using the Kalman filter algorithm and predict the fire spread trend. Based on the predicted fire spread trend, a fire development prediction map is constructed and sent to the alarm management module. The alarm management module is used to issue fire alarms based on the fire source identification results and the fire development prediction map.

2. The fire source detection, identification and alarm system of the intelligent fire-fighting robot according to claim 1 is characterized in that: The multimodal sensor data includes depth camera data, thermal imaging sensor data, and gas sensor data; The fire source identification operation includes a flame feature extraction operation and a flame target detection operation; The flame feature extraction operation includes the following steps: S11. Convert the depth camera data from RGB channels to HSV color space, and extract flame color features based on the converted HSV color space; S12. Extracting temperature features based on the thermal imaging sensor data, and screening a high-temperature region above a set threshold based on the extracted temperature features as a first high-temperature region; S13. Extract combustion product characteristics based on gas sensor data, and eliminate non-fire source high-temperature objects in the first high-temperature area according to the combustion product characteristic information to obtain a second high-temperature area.

3. The fire source detection, identification and alarm system of the intelligent fire-fighting robot according to claim 2 is characterized in that: The flame target detection operation includes: The flame color features, the second high-temperature area, and the combustion product features are input into the deep learning target detection model, and the flame target bounding box is output by the deep learning target detection model.

4. The fire source detection, identification and alarm system of the intelligent fire-fighting robot according to claim 3 is characterized in that: The operation of extracting the initial three-dimensional position coordinates of the fire source includes calculating the initial three-dimensional position coordinates of the fire source according to the flame target bounding box and depth information of the depth camera.

5. The fire source detection, identification and alarm system of the intelligent fire-fighting robot according to claim 1 is characterized in that: The process of dynamically tracking the fire source based on the initial three-dimensional position coordinates of the fire source using the Kalman filter algorithm includes the following steps: S21. Set the initial fire source state; the initial fire source state includes the initial three-dimensional position coordinates, velocity and acceleration of the fire source; S22. Predicting the future fire source state based on the initial fire source state through the state transition matrix, and optimizing the predicted future fire source state through the process noise covariance matrix to obtain the first fire source state; S23. Calculate the prediction error based on the real-time multimodal sensor data through the observation matrix, and perform a Kalman gain update operation on the first fire source state according to the prediction error to obtain the second fire source state; S24. Output the three-dimensional position coordinates of the current fire source based on the second fire source state, and use the three-dimensional position coordinates of the current fire source as the new initial three-dimensional position coordinates to repeat steps S21 to S24; S25. Use the three-dimensional position coordinates of the current fire source output each time as the basic data of the fire source trajectory.

6. The fire source detection, identification and alarm system of the intelligent fire-fighting robot according to claim 5 is characterized in that: The fire source spread trend prediction operation includes predicting the fire source spread trend based on the fire source trajectory basic data by using an extended Kalman filter algorithm.

7. The fire source detection, identification and alarm system of the intelligent fire-fighting robot according to claim 6 is characterized in that: The multimodal sensor data also includes environmental monitoring data; and the method of predicting the fire source spread trend based on the fire source trajectory basic data using the extended Kalman filter algorithm includes the following steps: S31. Establish the fire spread state equation based on the fire source trajectory basic data; S32. Calculate the direction of fire spread based on environmental monitoring data and thermal imaging sensor data; S33. Predict the spread area of the fire source at different time steps based on the diffusion direction of the fire source through the fire source spread state equation, and generate fire source spread trend prediction information based on the predicted spread area; the fire source spread trend prediction information includes the future coverage range of the fire source, center position change, combustion intensity trend and fire threat area.

8. The fire source detection, identification and alarm system of the intelligent fire-fighting robot according to claim 7 is characterized in that: The construction of a fire source development map based on the fire source spread trend comprises the following steps: S41. Record and store historical movement information of fire sources, including historical fire source locations and combustion intensity change information; S42. Construct a fire development prediction map based on historical movement information and fire source spread trend prediction information, and send the fire development prediction map to the alarm management module.

9. The fire source detection, identification and alarm system of the intelligent fire-fighting robot according to claim 8, characterized in that: The fire source development prediction path and the corresponding fire source spread trend prediction information are marked in the fire development prediction map.

10. A fire source detection, identification and alarm method for an intelligent fire-fighting robot, characterized in that: The method applies the system according to any one of claims 1 to 9, and the method includes: Collecting and processing multimodal sensor data; the multimodal sensor data is used for fire source detection; A fire source identification operation is performed based on multimodal sensor data using a deep learning target detection model to obtain a fire source identification result; when the fire source identification result indicates that a fire source exists, the initial three-dimensional position coordinates of the fire source are extracted; The Kalman filter algorithm is used to dynamically track the fire source based on its initial three-dimensional position coordinates and predict the fire spread trend. A fire development prediction map is constructed based on the predicted fire spread trend. Fire alarms are issued based on fire source identification results and fire development prediction maps.

Images