Fire-fighting robot and fire-fighting method based on AI vision and multi-modal perception

Abstract

This invention relates to the field of firefighting robot technology, and discloses a fully autonomous intelligent firefighting robot based on AI vision and multimodal perception. Its features include a multimodal perception module, an AI edge computing decision-making module, a fully autonomous execution module, a hierarchical linkage communication module, a cloud scheduling module, and a power management module. These modules achieve data interoperability and command coordination through an industrial-grade CAN bus protocol. The invention also discloses a firefighting method. This invention has the following main beneficial technical effects: high perception accuracy and comprehensive coverage; accurate AI recognition and strong anti-interference capability; precise fire extinguishing and high efficiency; fully autonomous closed-loop operation and unattended operation; strong environmental adaptability and wide scenario adaptation; applicable to firefighting in new energy charging pile scenarios, multi-machine collaborative firefighting in underground parking lot oil fire disposal, and cross-platform linkage firefighting in energy storage power station level 3 fires, etc. It features hierarchical linkage and strong collaboration; data support and iterative optimization.

Description

Technical Field

[0001] This invention relates to the field of firefighting robot technology, and in particular to a fully autonomous intelligent firefighting robot and firefighting method based on AI vision and multimodal perception. It is applicable to fire monitoring, early warning and autonomous handling in scenarios such as new energy infrastructure (charging piles, energy storage power stations), underground / multi-level parking lots, logistics warehouses, tunnels and pipe corridors, and belongs to the cross-technical fields of intelligent firefighting equipment, computer vision, Internet of Things and automatic control. Background Technology

[0002] While the new energy industry is developing rapidly, the fire risk of infrastructure such as charging piles and energy storage power stations, as well as related supporting scenarios, has increased significantly. Traditional fire protection solutions have many technical shortcomings: First, they rely on manual detection and response, resulting in multiple and time-consuming response steps, making it impossible to achieve rapid intervention in the early stages of a fire, especially for sudden fires such as battery thermal runaway, which can easily lead to missing the best opportunity for fire extinguishing; Second, the sensing methods are limited, mostly using a single smoke / temperature sensor based on a threshold-triggered monitoring mechanism, which is prone to missed or false alarms in complex environments such as dense smoke, darkness, and obstruction, and cannot identify the type and intensity of the fire; Third, the fire extinguishing methods are fixed, employing... Single extinguishing agents, based on experience-based fire suppression strategies, cannot be accurately adapted to different fire types such as electrical fires, oil fires, and solid fires, easily leading to low fire suppression efficiency and secondary damage to equipment; fourth, poor environmental adaptability, traditional fire-fighting equipment lacks multi-sensor fusion navigation technology, making it difficult to work in dangerous environments with high temperature, toxicity, and explosion-proof requirements, and navigation accuracy is low in scenarios without GPS; fifth, lack of full-process closed-loop capability, with monitoring, response, and linkage links disconnected, operations are based on manual commands, cannot achieve unmanned operation, have high human operating costs, and the response data is not systematically stored, which is not conducive to subsequent hazard investigation and strategy optimization.

[0003] CN120123829A discloses a method and apparatus for identifying fire hazards using AI visual analysis technology. It innovatively constructs a multi-source data acquisition and fusion mechanism, integrating multimodal data such as image and video streams, 3D space, thermal imaging, and environmental perception. A feature extraction model based on transfer learning is designed, combined with a hierarchical classification structure and attention mechanism, achieving high-precision hazard feature identification through ensemble learning. Temporal analysis and spatial positioning technologies are introduced to construct a hazard feature association network and evolution model, enabling prediction of hazard development trends and discovery of common hazards. However, there is still room for improvement in its intelligence and early warning capabilities.

[0004] While existing firefighting robots possess certain mobility and firefighting capabilities, they generally suffer from low levels of multimodal perception fusion, weak generalization ability of AI recognition models, lack of intelligent algorithm support for extinguishing agent adaptation, and lack of distributed scheduling technology for multi-robot collaboration. These shortcomings prevent them from meeting the high-precision, fully autonomous, and intelligent firefighting requirements of new energy scenarios and complex, enclosed spaces. Therefore, developing a fully autonomous firefighting robot that integrates computer vision fusion technology, multi-sensor information fusion technology, deep learning recognition technology, automatic control and precision spraying technology, and distributed cluster scheduling technology is crucial to solving these technical problems. Summary of the Invention

[0005] This invention addresses the shortcomings of existing firefighting technologies by providing a firefighting robot and method based on AI vision and multimodal perception. By integrating core technologies such as multi-sensor information fusion, deep learning, dynamic path planning, and closed-loop flow control, it achieves a fully unattended firefighting closed-loop process, enabling multi-dimensional accurate perception, AI-powered intelligent fire source identification, adaptive fire suppression based on fire type, high-precision navigation in GPS-free environments, and tiered, coordinated emergency response. This solves the technical problems of traditional solutions, such as slow response, low perception accuracy, inaccurate fire suppression, poor environmental adaptability, and weak coordination, thereby improving the efficiency and safety of fire response in new energy and complex scenarios. These are achieved through the following technical solutions.

[0006] A firefighting robot based on AI vision and multimodal perception includes a multimodal perception module, an AI edge computing decision-making module, a fully autonomous execution module, a hierarchical linkage communication module, a cloud scheduling module, and a power management module. Each module achieves data interoperability and command coordination through an industrial-grade CAN bus protocol, constructing a full-process intelligent firefighting system from anomaly detection to coordinated response. The specific module technical solutions are as follows: Multimodal sensing module: Integrates LiDAR, visible light industrial camera, uncooled infrared thermal imager, multi-channel gas sensor array, ultrasonic sensor, and high-precision temperature, humidity / smoke sensor. Based on multi-sensor time synchronization and spatial registration technology, it realizes synchronous acquisition and preprocessing of multi-source data, solves the problem of inconsistent data dimensions and acquisition frequencies of different sensors, and provides high-dimensional and high-precision raw data for subsequent decision-making.

[0007] Among them, the lidar is based on the TOF (Time of Flight) ranging principle, with a detection range of 100m and an accuracy of ±2cm. It supports 360° SLAM mapping and reconstructs the three-dimensional spatial information of the scene through point cloud data. The visible light industrial camera is based on the CMOS image sensing principle, with a minimum illumination of 0.001 Lux and supports HDR wide dynamic range. It improves the image acquisition quality in complex lighting environments through multi-frame exposure fusion technology. Uncooled infrared thermal imagers are based on the infrared focal plane array (FPA) detection principle, with a temperature measurement range of -20℃ to 550℃ and a thermal sensitivity of <50mK. They achieve visualization of heat sources and quantitative detection of temperature through infrared radiation energy conversion. The multi-channel gas sensor array is based on the electrochemical / catalytic combustion detection principle. CO detection uses the electrochemical principle with an accuracy of ±5ppm and a response time of <30s, while combustible gas detection uses the catalytic combustion principle to achieve accurate detection of toxic and combustible gases. The ultrasonic sensor is based on the principle of echo ranging, with a detection range of 0.2-5m and an accuracy of ±1cm, enabling spatial perception in near-field blind zones.

[0008] The AI ​​edge computing decision-making module is equipped with an industrial-grade edge computing chip (computing power ≥16 TOPS), and incorporates a multimodal data fusion algorithm, a fire source identification neural network model, a fire level determination model, and a dynamic path planning algorithm. Based on edge computing technology, it enables real-time local decision-making without cloud intervention, ensuring the timeliness of emergency response. The core technology principles are as follows: Multimodal data fusion algorithm: Based on DS evidence theory, feature layer and decision layer fusion is performed on multi-source data such as point cloud, image, infrared, gas, temperature and humidity from the perception module to improve the credibility and integrity of the data. The confidence level of the fusion result is ≥95%. Fire source identification neural network model: Based on the YOLOv8 deep learning framework, a convolutional neural network (CNN) with dual-modal fusion of visible light and infrared thermal imaging is constructed. It is trained with massive fire scene data (including complex scenes such as darkness, dense smoke, and obstruction) to achieve accurate identification of fire source location and fire type (battery thermal runaway / electric fire, oil fire, and ordinary solid fire). The false alarm rate is <0.5%, the false negative rate is close to 0, and the identification response time is <5s. Fire severity determination model: Based on the fuzzy comprehensive evaluation method, it integrates four core characteristic parameters: fire source area, temperature change rate, gas concentration, and spread trend. Through fuzzification, fuzzy reasoning, and defuzzification processes, it achieves quantitative determination of fire severity. Dynamic path planning algorithm: Based on the fusion strategy of A* algorithm and dynamic window method (DWA), it combines static environmental data from LiDAR SLAM mapping and dynamic environmental data from ultrasonic sensors to plan the optimal fire extinguishing path. The positioning accuracy reaches the centimeter level in the absence of GPS and the obstacle avoidance response time is <2s.

[0009] The fully autonomous execution module includes a high-precision autonomous navigation mobile chassis, a multi-extinguishing agent intelligent switching spray system, and an electric actuator. It achieves precise command execution based on closed-loop automatic control technology. The core technical principle is as follows: High-precision autonomous navigation mobile chassis: It adopts a tracked design and is based on the differential steering control principle. It is suitable for complex terrains such as slopes and narrow passages. It achieves uniform and precise movement through PID speed closed-loop control. The response time from anomaly identification to arrival at the fire source is ≤30s. Intelligent switching and spraying system for multiple extinguishing agents: It integrates storage units for four types of extinguishing agents: perfluorohexanone, ultrafine dry powder, foam extinguishing agent, and high-pressure fine water mist. Based on the on / off control of electromagnetic reversing valve and the PID flow closed-loop control principle, it realizes millisecond-level switching and quantitative spraying of extinguishing agents. It constructs an intelligent matching model of fire type-extinguishing agent-spraying parameters, and automatically adapts the extinguishing pressure, flow rate, and angle according to the fire type, with a spraying accuracy of ±0.5L / min. Electric actuator: Based on the servo motor drive principle, it realizes auxiliary operations such as power outage and fire isolation in the corresponding area through torque closed-loop control, with an execution accuracy of ±0.5°, and can be linked with scene facilities through industrial communication protocols.

[0010] The hierarchical linkage communication module includes local edge communication nodes and cross-platform linkage interfaces, supports multiple communication modes such as 5G / 4G / LoRa, and achieves seamless integration of multiple platforms and facilities based on heterogeneous network converged communication technology. The core technology principle is as follows: Based on a custom hierarchical linkage communication protocol, a data frame encryption transmission method (AES-256 encryption) is adopted to ensure data transmission security. Differentiated reporting strategies are implemented according to the fire level: Level 1 fires are handled locally and the data is archived; Level 2 fires push the data to the property management office's fire protection system; and Level 3 fires are simultaneously uploaded to the emergency management department and the 119 fire command center. The cross-platform linkage interface is based on the OPC UA industrial IoT communication standard, enabling bidirectional data interaction with facilities such as charging pile control systems, vehicle moving robots, ventilation systems, battery isolation systems, and fireproof roller shutters within the scenario, with a linkage response time of <3s.

[0011] The cloud-based dispatch module consists of a cloud-based intelligent fire protection brain, an AI training platform, and a global dispatch system. It leverages cloud computing and distributed cluster dispatching technologies to achieve multi-robot collaboration and algorithm iteration optimization. The core technical principles are as follows: Global scheduling system: Based on the distributed consensus algorithm (Raft), it realizes the cluster networking and division of labor scheduling of multiple fire-fighting robots, and automatically allocates tasks such as positioning, fire extinguishing, isolation and data collection according to the scene area and fire source distribution; AI training platform: Based on federated learning technology, it integrates fire protection data from multiple scenarios to train models while protecting the privacy of on-site data. The trained models are then distributed to the robot locally through incremental updates. Data Management: Based on cloud storage technology, it stores, analyzes and visualizes fire protection data (monitoring data, response records, video data, etc.) for all scenarios. The data storage period is ≥3 years and supports dual backup on local and cloud.

[0012] The power management module uses a high-temperature resistant, explosion-proof lithium battery (nominal voltage 48V, capacity 100Ah), and is based on a battery management system (BMS) to achieve charge / discharge protection and range monitoring. The core technology principle is as follows: Accurate estimation of remaining battery capacity (SOC) is achieved based on the Kalman filter algorithm, with an estimation error of <5%. It features four protection functions: over-temperature, over-voltage, over-current, and short-circuit protection. Based on the PID temperature control principle, it achieves battery thermal management, ensuring the robot's continuous working ability in environments ranging from -20℃ to 60℃, with a single charge providing ≥8 hours of battery life.

[0013] The fire-fighting robot of this invention adopts a lightweight modular design. The sensing module, fire extinguishing module, navigation module, and communication module are all independent modular units, which are connected through standardized quick-connect interfaces (aviation plugs + guide rail slots). Based on the modular design theory, the functional modules can be quickly replaced. The replacement time for functional modules such as gas detection, windproof fire extinguishing, and high-level spraying is less than 10 minutes, reducing scene adaptation and maintenance costs.

[0014] It has the ability to protect against extreme environments. The main body is made of high temperature resistant and explosion-proof alloy material (316L stainless steel + Kevlar protective layer). Based on the IP protection level design standard, it achieves a protection level of IP67. It has the characteristics of high temperature resistance, smoke resistance, water and dust resistance, and explosion protection. The explosion protection level reaches Ex d IIB T4 Gb. It can actively intervene and deal with dangerous environments such as dense smoke, toxic gases, and high temperature.

[0015] The fire-fighting robot of this invention has a built-in scene adaptive adjustment algorithm. Based on scene feature extraction and machine learning classification technology, it automatically identifies the application scene (new energy battery swapping station / energy storage station, logistics warehouse, tunnel corridor, etc.) by collecting feature data such as scene three-dimensional space, facility type, and environmental parameters through the perception module, and adaptively adjusts core parameters such as monitoring sensitivity, inspection range, spray angle, and operation mode to achieve precise adaptation to different scenes.

[0016] A firefighting method, based on the aforementioned AI vision and multimodal perception firefighting robot, is characterized by comprising the following steps: Step 1, Daily Inspection: The multimodal perception module collects spatial, temperature, gas, and visual data in the scene 24 / 7 based on multi-sensor time synchronization and spatial registration technology. The lidar completes scene SLAM mapping based on the TOF ranging principle. The data is uploaded to the cloud scheduling module in real time via CAN bus. Step 2, Anomaly Identification: After the multimodal perception module detects a fire source or fire precursor, it transmits the preprocessed multi-source data to the AI ​​edge computing decision module. Step 3, AI decision-making: Data is fused using a multimodal data fusion algorithm based on DS evidence theory, fire source location and fire type are identified using a YOLOv8 dual-modal fusion convolutional neural network model, fire level is determined using a fire intensity determination model based on fuzzy comprehensive evaluation method, and optimal fire extinguishing path is planned using a dynamic path planning algorithm based on A*+DWA fusion strategy. Step 4, Autonomous Handling: The fully autonomous execution module moves to the fire source along the planned path, the electric actuator completes auxiliary operations based on the servo motor drive principle, and the multi-extinguishing agent intelligent switching spray system matches the extinguishing agent and sprays it quantitatively based on the PID flow closed-loop control principle; Step 5, Hierarchical linkage: Execute the reporting strategy based on the custom hierarchical linkage communication protocol according to the fire level, link facilities within the scenario or external platforms based on the OPC UA standard, and realize cluster division of labor scheduling based on the distributed consensus algorithm in the multi-machine collaborative scenario. Step 6, Data Retention and Optimization: All process data is encrypted and stored locally and in the cloud using AES-256. The cloud scheduling module analyzes the data and optimizes the algorithm model based on federated learning technology, and distributes it to the fire robot through incremental updates to achieve iterative upgrades.

[0017] The fire-fighting method described above is characterized in that, in step 4, for battery thermal runaway / electrical fires, perfluorohexanone or ultrafine dry powder extinguishing agents based on the principle of physical asphyxiation and heat absorption are used; for oil fires, foam extinguishing agents based on the principle of oxygen isolation are used; and for ordinary solid fires, high-pressure fine water mist extinguishing agents based on the principle of cooling and asphyxiation are used. After extinguishing the fire, continuous monitoring is performed for more than 30 minutes to prevent reignition, and the spraying accuracy is ±0.5L / min.

[0018] Compared with the prior art, the present invention has the following significant beneficial technical effects: 1. High perception accuracy and comprehensive coverage: Based on multi-sensor time synchronization and spatial registration technology and DS evidence theory data fusion algorithm, it realizes multi-dimensional accurate perception of space, temperature, gas and vision, solves the monitoring blind spot problem in complex environments such as darkness, dense smoke and obstruction, perception response time <10 seconds, obstacle avoidance accuracy ±2cm, and provides reliable data support for early fire identification.

[0019] 2. AI recognition is accurate and anti-interference capability is strong: Based on YOLOv8 dual-modal fusion convolutional neural network, and trained with massive complex scene data, it can accurately distinguish between flames and interference sources such as light and sunlight, with a false alarm rate of <0.5%. It can also identify the early signs of battery thermal runaway (abnormal temperature rise + trace gas leakage), and realize early detection and early intervention based on the characteristics of the early stage of fire.

[0020] 3. Precise and efficient fire extinguishing: Based on PID flow closed-loop control technology, a multi-extinguishing agent intelligent adaptation system is constructed. It automatically matches the optimal extinguishing agent and spray parameters for different fire types, improving the fire extinguishing efficiency by more than 80% compared with traditional single extinguishing agent solutions. Moreover, extinguishing agents such as perfluorohexanone and ultrafine dry powder are based on physical fire extinguishing principles, leaving no residue after extinguishing the fire and not damaging new energy precision equipment.

[0021] 4. Fully autonomous closed-loop and unattended operation: Integrating edge computing, dynamic path planning, closed-loop automatic control and other technologies, it realizes unattended operation of the entire process from daily inspection to anomaly identification, fire source determination, path planning, autonomous fire extinguishing, linkage reporting and data retention. Emergency response is completed within 30 seconds. Based on unmanned operation, it reduces labor costs by more than 80% and solves the problem of manual dependence in traditional solutions.

[0022] 5. Strong environmental adaptability and wide scene adaptation: Based on TOF ranging and SLAM mapping technology, it realizes centimeter-level navigation in the absence of GPS, adapts to terrains such as narrow passages and slopes, and the body is based on explosion-proof and IP protection design standards, and can work in extreme environments such as -20℃~60℃, explosion-proof, and toxic gas. Through scene adaptive adjustment and modular design, it can be adapted to various scenarios such as new energy, warehouses, and tunnels.

[0023] 6. Hierarchical linkage and strong collaboration: A three-level intelligent response mechanism is constructed based on the fuzzy comprehensive evaluation method. Combined with the OPC UA standard and heterogeneous network converged communication technology, a cross-platform and cross-facility linkage system is realized to achieve precise handling of different fire situations and multi-party collaboration, reducing secondary losses. At the same time, multi-machine collaborative networking is realized based on the distributed consensus algorithm to improve the handling efficiency of large-area fires.

[0024] 7. Data-driven support and optimizable iteration: The entire process of data processing is systematically stored based on cloud storage technology, supporting dual backups on both local and cloud platforms, providing data support for fire hazard investigation and response strategy optimization; and based on federated learning technology, the algorithm can be iterated both locally and on the cloud, continuously optimizing the model while protecting data privacy and adapting to changes in scenario requirements. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the robot in this invention.

[0026] Figure 2This is a schematic diagram of the fire-fighting method in this invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand and implement this patent, the invention will be further described in detail below with reference to specific embodiments. This embodiment uses the application of a fire-fighting robot in a new energy charging pile scenario as an example, and is not intended to limit the scope of protection of this invention. Please see... Figure 1-2 And refer to the invention content.

[0028] Example 1: Application of firefighting robots in new energy charging pile scenarios Equipment deployment: One main fire-fighting robot and one auxiliary fire-fighting robot are deployed in the new energy charging pile cluster area. Local edge computing nodes and cloud-based smart fire-fighting brain are built in conjunction with them. The fire-fighting robots communicate with the charging pile control system and property fire protection system through the OPC UA interface to complete scene SLAM 3D mapping and parameter initialization.

[0029] Routine Inspection: The fire-fighting robot achieves 24 / 7 autonomous patrol through a multimodal perception module, moves along a planned path based on differential steering control, collects point cloud data through TOF ranging, and updates the scene's 3D information in real time, and a visible light + infrared thermal imager simultaneously collects images and infrared data based on CMOS and FPA detection principles, and a gas sensor array monitors the surrounding CO and combustible gas concentrations based on electrochemical principles, while simultaneously checking for hidden dangers such as blocked fire lanes and illegal parking. Inspection data is uploaded to the cloud in real time via a CAN bus.

[0030] Anomaly monitoring: When a charging pile shows signs of battery thermal runaway, the infrared thermal imager detects an abnormal increase in the charging pile's temperature within a short period of time (from room temperature to 80°C) through infrared radiation energy conversion. The gas sensor array detects trace amounts of CO gas (concentration 8ppm). The multimodal perception module preprocesses the multi-source data based on time synchronization technology and transmits it to the AI ​​edge computing decision module.

[0031] AI Decision Making: The AI ​​edge computing decision module integrates multi-source data through DS evidence theory, determining the data confidence level to be 98%; based on the YOLOv8 dual-modal fusion CNN model, it identifies the battery as a precursor to thermal runaway, with no open flame; through the fuzzy comprehensive evaluation method, it integrates parameters such as temperature change rate and gas concentration, determining it to be a level one fire (without open flame); at the same time, it plans the optimal arrival path based on the A*+DWA fusion algorithm, with a positioning accuracy of ±2cm.

[0032] Autonomous response: The mobile chassis of the fully autonomous execution module moves autonomously to the charging pile along the planned path based on PID speed closed-loop control. The electric actuator automatically cuts off the power supply to the area through the charging pile control system based on the servo motor drive principle. The multi-extinguishing agent intelligent switching spray system matches perfluorohexanone extinguishing agent based on PID flow closed-loop control, and sprays it precisely at a pressure of 0.5MPa and a flow rate of 5L / min. It quickly suppresses the trend of thermal runaway through the principles of physical asphyxiation and heat absorption.

[0033] Data retention and feedback: Temperature, gas concentration, fire extinguishing parameters, and video data throughout the entire incident are automatically stored locally and in the cloud using AES-256 encryption. The data is encrypted and traceable. This incident was classified as a Level 1 fire, so it is only archived locally and not reported to external platforms.

[0034] Post-processing optimization: The cloud-based intelligent fire protection brain analyzes the data from this thermal runaway incident and optimizes the infrared temperature measurement sensitivity and gas detection frequency parameters of the charging pile area based on machine learning. This optimization is then distributed to the AI ​​model of the fire robot via incremental updates, improving the efficiency of identifying and handling similar hazards in the future.

[0035] Example 2: Multi-machine collaborative application for oil fire suppression in underground parking lots Anomaly Detection: A vehicle's fuel tank caught fire in an underground parking lot, forming an oil fire. The visible light + infrared thermal imager of the fire-fighting robot detected an open flame and a high-temperature heat source (temperature 280℃). The gas sensor array detected combustible gas (concentration 1.2% LEL). The AI ​​edge computing decision module determined it to be a level 2 fire (controlled fire, not spreading) using the fuzzy comprehensive evaluation method, and triggered multi-machine collaborative instructions based on the distributed consensus algorithm (Raft).

[0036] Multi-robot linkage: The main fire-fighting robot plans its path based on the A*+DWA fusion algorithm and arrives at the fire source. The intelligent switching spray system of multiple extinguishing agents matches the foam extinguishing agent. Based on PID flow closed-loop control, it achieves fire suppression by covering open flames and extinguishing fires by isolating oxygen through foam coverage. The auxiliary fire-fighting robot arrives simultaneously and is responsible for monitoring the surrounding temperature and fire isolation. Based on the OPC UA interface, it links the parking lot fireproof roller shutter to close to prevent the fire from spreading.

[0037] Tiered reporting: During the handling process, local edge computing nodes push data such as fire location, type, and handling progress to the property management office's fire protection system in real time through a customized tiered linkage protocol. Property staff can view on-site video in real time through the cloud platform.

[0038] Subsequent handling: After the open flame is extinguished, the main fire-fighting robot continues to spray a small amount of foam extinguishing agent based on quantitative spray control to prevent reignition. The auxiliary fire-fighting robot continuously monitors the surrounding temperature for 30 minutes. After confirming that there is no risk of reignition, the handling is completed and the entire process data is archived.

[0039] Example 3: Cross-platform linkage application for level 3 fire in energy storage power stations Fire assessment: The battery cabinet of the energy storage power station experienced thermal runaway and ignited an open flame. The fire showed a tendency to spread. The multimodal perception module of the fire-fighting robot detected that the fire source area was expanding (≥5㎡), the temperature was rising to 300℃, and the CO concentration was rising significantly (≥50ppm). The AI ​​edge computing decision module determined it to be a level three fire through fuzzy comprehensive evaluation method.

[0040] Emergency Response: The fire-fighting robot immediately initiated the spraying of perfluorohexanone + ultrafine dry powder composite extinguishing agent. Based on dual-path PID flow closed-loop control, it achieved quantitative spraying of the two extinguishing agents to suppress the fire. At the same time, the electric actuator cut off the main power supply to the battery cabinet area and linked the energy storage power station battery isolation system through the OPC UA interface to prevent a chain reaction in other battery cabinets.

[0041] Cross-platform reporting: The hierarchical linkage communication module uses 5G communication to simultaneously upload data such as fire location, type, fire intensity, and real-time video to the emergency management department's command platform and the 119 fire command center, and simultaneously issue emergency rescue requests. Data transmission is encrypted using AES-256 to ensure transmission security.

[0042] Multi-party collaboration: Firefighting robots continuously monitor the fire based on real-time data collection and provide feedback data, providing accurate on-site information to firefighters until they arrive. This achieves seamless integration between robot-led initial response and manual rescue, and significantly reduces fire losses due to the rapid response and data feedback from robots.

[0043] The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The scope of protection of the present invention should be limited to the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A fire-fighting robot based on AI vision and multi-modal perception, characterized in that, It includes a multimodal perception module, an AI edge computing decision-making module, a fully autonomous execution module, a hierarchical linkage communication module, a cloud scheduling module, and a power management module. Each module achieves data communication and command coordination through an industrial-grade CAN bus protocol. The multimodal sensing module integrates a lidar, a visible light industrial camera, an uncooled infrared thermal imager, a multi-channel gas sensor array, an ultrasonic sensor, and a high-precision temperature, humidity, and smoke sensor. It achieves synchronous acquisition of multi-source data based on multi-sensor time synchronization and spatial registration technology. The lidar is based on the TOF ranging principle, with a detection distance of no less than 100m and an accuracy of ±2cm. The visible light industrial camera is based on the CMOS image sensing principle, with a minimum illumination of 0.001Lux. The uncooled infrared thermal imager is based on the infrared focal plane array detection principle, with a temperature range of -20℃ to 550℃ and a thermal sensitivity of <50mK. In the multi-channel gas sensor array, CO detection is based on an electrochemical principle, with an accuracy of ±5ppm and a response time of <30s. The ultrasonic sensor is based on the echo ranging principle, with a detection range of 0.2 to 5m and an accuracy of ±1cm. The AI ​​edge computing decision module is equipped with an industrial-grade edge computing chip with a computing power of ≥16 TOPS. It incorporates a multimodal data fusion algorithm based on DS evidence theory, a convolutional neural network model for fire source identification based on visible light + infrared thermal imaging dual-modal fusion using the YOLOv8 framework, a fire level determination model based on fuzzy comprehensive evaluation method, and a dynamic path planning algorithm based on A*+DWA fusion strategy. The false alarm rate for fire source identification is <0.5%, the positioning accuracy reaches the centimeter level in the absence of GPS, and the identification response time is <5s. The fully autonomous execution module includes a high-precision autonomous navigation mobile chassis, a multi-extinguishing agent intelligent switching spray system, and an electric actuator. The mobile chassis is based on differential steering and PID speed closed-loop control principles, with a response time of ≤30s from anomaly detection to arrival at the fire source. The multi-extinguishing agent intelligent switching spray system integrates storage units for four types of extinguishing agents: perfluorohexanone, ultrafine dry powder, foam extinguishing agent, and high-pressure fine water mist. Based on electromagnetic reversing valve on / off control and PID flow closed-loop control principles, it achieves millisecond-level switching and quantitative spraying of extinguishing agents, with a spraying accuracy of ±0.5L / min. The electric actuator is based on servo motor drive and torque closed-loop control principles, with an execution accuracy of ±0.5°. The hierarchical linkage communication module includes a local edge communication node and a cross-platform linkage interface, supports 5G / 4G / LoRa multi-communication modes, and is based on heterogeneous network converged communication technology and OPC UA industrial IoT communication standard. It constructs a custom hierarchical linkage communication protocol with AES-256 encryption, which can link the charging pile control system, vehicle moving robot, ventilation system and battery isolation system in the scene, with a linkage response time of <3s. The cloud-based scheduling module consists of a cloud-based intelligent fire protection brain, an AI training platform, and a global scheduling system. It realizes cluster networking and division of labor scheduling of multiple fire robots based on a distributed consensus algorithm, optimizes and incrementally updates the AI ​​model based on federated learning technology, and stores and manages fire data based on cloud storage technology. The data storage period is ≥3 years, and it supports local autonomous iteration and cloud optimization of the algorithm. The power management module uses a high-temperature resistant and explosion-proof lithium battery. It achieves accurate SOC estimation based on the Kalman filter algorithm with an estimation error of <5%. It has four protection functions: over-temperature, over-voltage, over-current, and short circuit. It achieves battery thermal management based on the PID temperature control principle, ensuring that the robot can work in an environment of -20℃ to 60℃, and has a single charge life of ≥8 hours.

2. The fire-fighting robot based on AI vision and multi-modal perception according to claim 1, wherein, The multi-extinguishing agent intelligent switching spray system has a built-in intelligent matching model of fire type-extinguishing agent-spraying parameters. It can automatically match the optimal extinguishing agent and corresponding spray pressure, flow rate and angle based on the PID flow closed-loop control principle according to three fire types: battery thermal runaway / electrical fire, oil fire and ordinary solid fire.

3. The firefighting robot based on AI vision and multimodal perception according to claim 1, characterized in that, The fire intensity determination model is based on the fuzzy comprehensive evaluation method, which integrates four characteristic parameters: fire source area, temperature change rate, gas concentration, and spread trend. It automatically determines the fire intensity level through fuzzification, fuzzy inference, and defuzzification processes. Level 1 fires are small fires, which are handled locally and the data is archived. Level 2 fires are controlled fires, and the data is pushed to the property management office's fire protection system while the fire is being handled. Level 3 fires are open flames or have a spread trend, and the data is uploaded to the emergency management department and the 119 fire command center while the fire is being handled.

4. The firefighting robot based on AI vision and multimodal perception according to claim 1, characterized in that, The fire-fighting robot adopts a lightweight modular design. The sensing module, fire extinguishing module, navigation module, and communication module are independent modular units, which are connected through standardized quick-connect interfaces. Based on the modular design theory, the functional modules can be quickly replaced. The replacement time for gas detection, windproof fire extinguishing, and high-level spray functional modules is less than 10 minutes.

5. The firefighting robot based on AI vision and multimodal perception according to claim 1, characterized in that, The firefighting robot body is made of high-temperature resistant and explosion-proof alloy material with stainless steel 316L + Kevlar protective layer. Based on the IP protection level design standard, it achieves a protection level of IP67 and an explosion-proof level of Ex d IIB T4 Gb. It has the characteristics of high temperature resistance, smoke resistance, waterproof and dustproof, and explosion-proof, and can work in environments with dense smoke and toxic gases.

6. The firefighting robot based on AI vision and multimodal perception according to claim 1, characterized in that, The AI ​​edge computing decision module has a built-in scene adaptive adjustment algorithm based on scene feature extraction and machine learning classification technology. It can automatically identify the application scene through the scene three-dimensional space, facility type and environmental parameter feature data collected by the perception module, and adaptively adjust the core parameters of monitoring sensitivity, inspection range, spray angle and operation mode.

7. The firefighting robot based on AI vision and multimodal perception according to claim 1, characterized in that, The fully autonomous execution module's electric actuator can achieve power outage and fireproof isolation auxiliary operation in the corresponding area. The mobile chassis is a tracked design, and it is adapted to complex terrain without GPS, such as slopes and narrow passages, based on the differential steering control principle. The obstacle avoidance response time is <2s.

8. The firefighting robot based on AI vision and multimodal perception according to claim 1, characterized in that, The confidence level of the multi-source data fusion result of the multimodal perception module is ≥95%, the cross-platform linkage interface of the hierarchical linkage communication module supports bidirectional data interaction, and the cloud scheduling module can perform dual backup of monitoring data, handling records, and video data both locally and in the cloud.

9. A firefighting method, based on the AI ​​vision and multimodal perception firefighting robot according to any one of claims 1 to 8, characterized in that, Includes the following steps: Step 1, Daily Inspection: The multimodal perception module collects spatial, temperature, gas, and visual data in the scene 24 / 7 based on multi-sensor time synchronization and spatial registration technology. The lidar completes scene SLAM mapping based on the TOF ranging principle. The data is uploaded to the cloud scheduling module in real time via CAN bus. Step 2, Anomaly Identification: After the multimodal perception module detects a fire source or fire precursor, it transmits the preprocessed multi-source data to the AI ​​edge computing decision module. Step 3, AI decision-making: Data is fused using a multimodal data fusion algorithm based on DS evidence theory, fire source location and fire type are identified using a YOLOv8 dual-modal fusion convolutional neural network model, fire level is determined using a fire intensity determination model based on fuzzy comprehensive evaluation method, and optimal fire extinguishing path is planned using a dynamic path planning algorithm based on A*+DWA fusion strategy. Step 4, Autonomous Handling: The fully autonomous execution module moves to the fire source along the planned path, the electric actuator completes auxiliary operations based on the servo motor drive principle, and the multi-extinguishing agent intelligent switching spray system matches the extinguishing agent and sprays it quantitatively based on the PID flow closed-loop control principle; Step 5, Hierarchical linkage: Execute the reporting strategy based on the custom hierarchical linkage communication protocol according to the fire level, link facilities within the scenario or external platforms based on the OPC UA standard, and realize cluster division of labor scheduling based on the distributed consensus algorithm in the multi-machine collaborative scenario. Step 6, Data Retention and Optimization: All process data is encrypted and stored locally and in the cloud using AES-256. The cloud scheduling module analyzes the data and optimizes the algorithm model based on federated learning technology, and distributes it to the fire robot through incremental updates to achieve iterative upgrades.

10. The fire-fighting method according to claim 9, characterized in that, In step 4, for battery thermal runaway / electrical fires, use perfluorohexanone or ultrafine dry powder extinguishing agents based on the principle of physical asphyxiation and heat absorption; for oil fires, use foam extinguishing agents based on the principle of oxygen isolation; for ordinary solid fires, use high-pressure fine water mist extinguishing agents based on the principle of cooling and asphyxiation. After extinguishing the fire, monitor continuously for more than 30 minutes to prevent reignition, with a spray accuracy of ±0.5L / min.

Images