Fire detection and disposal system and method based on multi-robot cooperation

Abstract

The invention provides a fire detection and disposal system and method based on multi-robot cooperation, and the system comprises a quadruped robot platform, a cloud platform, and a wheel-foot robot platform. Wherein the quadruped robot platform is used for detecting a fire behavior and sending information to the cloud platform in real time; the cloud platform receives information of the quadruped robot platform and dispatches and controls the quadruped robot platform and the wheel-foot robot platform; the wheel-foot robot platform is used for receiving an instruction of the cloud platform to execute fire extinguishing treatment; according to the method, the system is used for fire detection and disposal.

Description

Technical Field

[0001] This invention relates to a fire detection and response system and method, and more particularly to a fire detection and response system and method based on multi-robot collaboration. Background Technology

[0002] In many unmanned environments, such as warehouses, field base stations, and unmanned work areas of large factories, fires often cannot be dealt with promptly in their early stages due to a lack of monitoring methods. Traditional fixed fire detection equipment is limited by its installation location, has blind spots, and cannot be flexibly moved to the fire source for extinguishing operations, often only being detected when the fire has clearly spread. This delay in fire detection and the lack of mobility of firefighting equipment not only cause significant property damage but also pose a serious threat to the surrounding environment and personnel safety. Current technologies lack fully automated fire detection and response systems, failing to address the problem of timely fire detection and control in unmanned environments. Summary of the Invention

[0003] Purpose of the invention: The technical problem to be solved by the present invention is to provide a fire detection and handling system and method based on multi-robot collaboration, which addresses the shortcomings of the existing technology.

[0004] To address the aforementioned technical problems, this invention discloses a fire detection and response system and method based on multi-robot collaboration, wherein the system comprises: Quadruped robot platforms, cloud platforms, and wheeled robot platforms; among them... The quadruped robot platform is used to detect fires and send information to the cloud platform in real time; the cloud platform receives information from the quadruped robot platform and schedules and controls the quadruped robot platform and the wheeled robot platform; the wheeled robot platform is used to receive instructions from the cloud platform to perform fire extinguishing operations.

[0005] Furthermore, the quadruped robot platform includes: A quadruped robot, and a lidar, inertial odometry, vision sensor, fire and smoke detection module, alarm module, and main control module installed on the quadruped robot; wherein, The quadruped robot is a vehicle with a lidar that scans upwards in a hemispherical shape mounted on its top, a vision sensor mounted on its head, and an inertial odometer for measuring motion status. The fire and smoke detection module performs multi-scale feature extraction based on the images acquired by the visual sensor to obtain the bounding box coordinates of the flame and smoke and the corresponding confidence level. The alarm module triggers an alarm based on the confidence level. The main control module generates a local map based on information obtained from lidar, inertial odometry, and visual sensors, interacts with the cloud platform, and controls the overall operation of the quadruped robot platform.

[0006] Furthermore, the cloud platform includes: Based on the local maps generated by all quadruped robot platforms, a global map is generated, and inspection path planning is performed for collaborative control of the quadruped robot platforms. Based on the fire information obtained from all quadruped robot platforms, a disposal path is planned to control the wheeled robot platforms to handle the fire.

[0007] Furthermore, the wheeled robot platform includes: A wheeled robot and a fire extinguishing device mounted on the wheeled robot are used to execute the scheduling commands of the cloud platform to handle fire situations.

[0008] This invention also proposes a fire detection and handling method based on multi-robot collaboration, using the aforementioned system for fire detection and handling, including the following steps: Step 1: Create a local elevation map centered on the quadruped robot and merge and update it into a global elevation map; Step 2: Construct a quantitative model of the quadruped robot's passability and perform inspection path planning; Step 3: Use a quadruped robot to conduct inspections along the planned inspection path and detect fire and smoke targets. Step 4: Obtain the target detection results of fire and smoke through the cloud platform and plan the fire response path; Step 5: Use the wheeled robot to handle the fire according to the planned fire handling path.

[0009] Furthermore, the step 1, which involves establishing a local elevation map centered on the quadruped robot and merging and updating it into a global elevation map, includes: Step 1-1: Perform pose estimation for the quadruped robot based on visual inertial odometry; Steps 1-2: Create a local elevation map centered on the quadruped robot and update it in real time; Steps 1-3 involve fusing the local elevation maps created by all quadruped robots to construct a global elevation map.

[0010] Furthermore, the process of establishing and updating a local elevation map centered on the quadruped robot as described in steps 1-2 includes: Step 1-2-1, Local Elevation Map Construction, as detailed below: The three-dimensional point cloud data of the environment is calculated from the depth image acquired by the quadruped robot, and the three-dimensional point cloud data is downsampled. Based on the quadruped robot pose obtained in step 1-1, the downsampled 3D point cloud data is converted to the global coordinate system; Project the 3D point cloud onto a 2D raster map plane of a preset resolution; for each raster cell... The position of each grid cell in the computational environment map in three-dimensional space. , means as follows: ; Among them, grid unit The coordinates on the environmental map plane are: and The corresponding terrain height value is ; Collect all point clouds that fall within the horizontal range of this grid. , means as follows: ; The height of the grid cell is calculated based on a preset statistical value of the height of all points. ; The final local map contains the horizontal position of each raster cell. and corresponding height value Local elevation map; Step 1-2-2, local elevation map update, details are as follows: For new areas observed by the quadruped robot's sensors, the grid height value is directly filled in; For recurring regions observed by the quadruped robot's sensors—that is, grids with existing height values—the height values ​​of these grids are updated using a pre-defined fusion strategy based on the new observation point cloud. .

[0011] Furthermore, the fusion of the local elevation maps established for all quadruped robots as described in steps 1-3 includes: Step 1-3-1: Coordinate alignment. Using the pose information of each quadruped robot, all local elevation maps are converted to a unified global coordinate system. Step 1-3-2, Overlapping Area Processing: For overlapping areas of different local elevation maps, a preset fusion strategy is used to calculate the final height value of each grid cell in the overlapping area. Step 1-3-3: Fill non-overlapping areas by directly filling with local elevation map data from a single source; Steps 1-3-4: Dynamic update. The global elevation map is updated in real time based on the local elevation maps uploaded by each quadruped robot.

[0012] Furthermore, step 2, which involves constructing a quadruped robot's passability quantification model and planning inspection paths, includes: Step 2-1: Calculate the passability values ​​for the raster cells in the global elevation map, as follows: ; Among them, grid unit Passability value , For the current grid cell height, For the current grid cell The slope, For the current grid cell roughness, , and It depends on the preset threshold of the quadruped robot structure. , and For control coefficients; Slope value The calculation method is as follows: ; in, and Representing grid cells Physical dimensions in the x and y directions; Roughness The calculation method is as follows: ; in, To fall into the grid cell The number of point clouds, For the first point cloud The height of each point Average height; Step 2-2: Plan the inspection path based on the passability value. When the passability value of a grid cell is less than the threshold, the area is determined to be impassable, and obstacle avoidance is performed in the inspection path planning, as follows: Step 2-2-1, Accessibility Map Construction, will The grid cells are marked as obstacles, forming a binary obstacle map, where This is the passability threshold; Step 2-2-2: The inspection path planning uses the improved A* method to search for the optimal path on the accessibility map. Let the cost function of the inspection path be... , means as follows: ; in, From the starting point to the current node The actual cost of movement, This is a heuristic estimate of the distance from the current node to the destination. Step 2-2-3, Terrain Adaptation Improvement, involves calculating the actual movement cost. At that time, a passability indicator was introduced. As a penalty, that is, from the previous node Move to the current node The cost is: ; in, It is the Euclidean distance between the two nodes. This is the terrain penalty coefficient; Step 2-2-4, Dynamic obstacle avoidance: When a new obstacle or terrain change is detected, path replanning is triggered.

[0013] Furthermore, step 4, which involves planning the fire response route, includes: Step 4-1, semantic map construction, as follows: A semantic segmentation network is used to parse the global elevation map and identify preset key objects; Define risk quantification rules, and assign each grid cell based on the identified key objects. Label risk value ; Step 4-2, cascading filtering path planning, as follows: Step 4-2-1: Terrain accessibility filtering, i.e., excluding all... The grid cells generate terrain that can be accessed through the region; Step 4-2-2, Fire Risk Screening, details are as follows: Exclude all areas where the terrain allows passage. The grid is used to generate a safe area, which is a set of safe grid cells. , means as follows: ; in, Risk threshold; Step 4-2-3, shortest path search, details are as follows: In the safe area The A* method is used to plan the shortest path, and the cost function is given by... , means as follows: ; in, For actual mobility costs, A heuristic function is represented as follows: ; in, Representing the The location of the step It is Step position, These are the coordinates of the target point.

[0014] Beneficial effects: 1. This invention constructs a heterogeneous collaborative operation system of quadruped robots and wheeled-legged robots. Legged robots are assigned inspection and response tasks based on their terrain adaptability. Quadruped robots are used for fire inspection, while wheeled-legged robots are used for fire response. The quadruped robots, leveraging their excellent terrain adaptability, autonomously construct environmental maps and perform environmental inspections, intelligent fire and smoke identification, precise positioning, and alarm tasks. Data is uploaded to the cloud in real time via a 5G network. The wheeled-legged robots, with their high mobility and large load capacity due to their wheeled movement, are equipped with fire extinguishing devices and respond rapidly to commands from the cloud platform, performing precise fire extinguishing tasks.

[0015] 2. This invention employs a cloud-based intelligent management platform that integrates multimodal data fusion and analysis functions. It supports real-time display of video streams transmitted by the legged robot, visual detection results of fire and smoke, and the robot's real-time positioning. Users can remotely control the robot through text input, voice, and a visual interface, flexibly adjusting operational strategies to form an efficient human-machine interaction loop.

[0016] 3. This invention employs elevation map modeling technology to establish a local elevation map more adapted to the movement characteristics of quadruped robots, dynamically generating local 3D environment models that fit complex terrain. Through map fusion and map updating technologies, a robot-centric local elevation map is established. Compared to traditional point cloud maps and voxel maps, elevation map maps only need to calculate the height value of the highest point within the grid cell, thus significantly reducing the computational resource consumption during mapping and completely preserving the elevation information of environmental obstacles. This provides legged robots with accurate environmental perception data, supports dynamic obstacle avoidance and path planning, and significantly improves the efficiency and reliability of emergency response in complex fire scenarios. Attached Figure Description

[0017] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0018] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0019] Figure 2 This is a three-dimensional structural diagram of a quadruped robot.

[0020] Figure 3 This is a schematic diagram of the pose calculation process for a quadruped robot.

[0021] Figure 4 This is a schematic diagram of the process of generating and updating local maps for a quadruped robot. Detailed Implementation

[0022] The overall concept of this system is as follows: a quadruped robot equipped with a high-precision vision sensor (RGB camera), LiDAR, navigation control unit, and wireless communication module; a wheeled robot equipped with a fire extinguishing device (dry powder / gas injection system), robotic arm, path planning module, and high-capacity battery; and a cloud platform integrating an AI analysis engine, real-time map database, and remote control interface. Specifically, this system mainly consists of a quadruped robot platform, a cloud platform, and a wheeled robot platform.

[0023] Part 1, Quadruped Robot Platform: As the core mobile platform of the entire system, the quadruped robot is equipped with LiDAR, vision sensors, inertial odometry, and fire and smoke detection modules. With its stable mobility, it can adapt to complex and varied terrain. Its main control module is responsible for equipment coordination, data processing, and communication, ensuring the system's efficient and stable operation.

[0024] The quadruped robot, such as Figure 2 As shown, a robot with high stability and strong load capacity was selected. Its leg joints feature a six-degree-of-freedom design, enabling a 30kg load capacity, making it suitable for complex terrains such as factories and warehouses. A high-precision LiDAR is mounted on the top of the robot body, enabling hemispherical ultra-wide-angle scanning and measuring 360° of three-dimensional space above the radar. The vision sensor uses a front-facing camera, mounted in an adjustable position on the robot's head for optimal field of view. Simultaneously, an inertial odometry system is incorporated to accurately measure the robot's motion state. All sensor data is transmitted in real-time to a cloud platform, where ROS (Robot Operating System) performs multi-source data fusion and real-time processing.

[0025] Visual sensor module: Located on the quadruped robot platform, it is used to acquire RGB images in the space in real time during the movement of the quadruped robot, and transmit clear and comprehensive image data to the fire and smoke detection module. It provides a key data source for accurately identifying smoke and fire sources and is the core front-end component of the data acquisition in the entire fire detection process.

[0026] Fire and Smoke Detection Module: Located on a quadruped robot platform, this module uses deep learning algorithms to perform in-depth analysis of images transmitted from visual sensors. Through extensive image learning, it accurately identifies smoke and fire sources and provides recognition confidence levels, offering crucial information for fire assessment and forming the core computational component of fire detection.

[0027] The fire and smoke visual detection module is based on an improved YOLO v8 model, with algorithmic optimizations for flame and smoke features. The model embeds a CBAM attention module into the CSPDarknet53 backbone network to enhance the detection capabilities of flame edge features (such as flickering textures) and semi-transparent smoke. The visual data processing unit incorporates a two-level AI recognition algorithm. First, it uses a YOLO v7 model to detect flames on real-time transmitted images. This model is trained on a dataset of 20,000 multi-scene flame images, covering complex scenes such as backlighting, occlusion, and dynamic blur, achieving an 85% recognition accuracy under complex lighting conditions. During detection, the model extracts features from the input image at multiple scales, outputting the bounding box coordinates of the flame and smoke along with their corresponding confidence scores. For small target detection, the model employs a feature fusion strategy to improve fine-grained recognition accuracy, ensuring that the detection rate of flames and smoke meets practical application requirements. If no open flame is detected, the U-Net semantic segmentation model is used to analyze the semi-transparent smoke region in the image, and cross-validation is performed using smoke sensor data to effectively eliminate interference factors such as dust and water mist.

[0028] Alarm module: Located on the quadruped robot platform, it immediately triggers an alarm when the confidence level of the fire and smoke detection module reaches a preset threshold. It wirelessly transmits fire coordinates, detection time, and other information to the cloud platform, while simultaneously triggering a local audible and visual alarm to warn the surrounding area and promptly disseminate fire information.

[0029] Part Two: Cloud Platform This system serves as the intelligent central hub, receiving and storing data from the quadruped robots in real time. Operators can view patrol routes, images, and alarm details through a visual interface. It can automatically generate control commands for the quadruped robots based on the fire situation, scheduling them to perform firefighting tasks and achieving coordinated system operation.

[0030] The cloud platform uses an environmental map generated by the quadruped robot platform and employs the Dijkstra algorithm to plan a global inspection path, highlighting high-risk areas and generating comprehensive movement trajectories. The map is divided according to regional functions, such as storage areas and passageways in a warehouse. Based on map features, the cloud platform automatically generates a global inspection path and sets the robot's inspection cycle and priority strategy. The path is distributed to the quadruped robot's navigation module in a gridded trajectory format, prioritizing coverage of high-risk areas such as flammable material storage areas and electrical equipment areas. The quadruped robot moves along the planned path, with its leg joints using hydraulic drives to adaptively adjust its gait. The navigation module combines real-time sensor data to achieve dynamic obstacle avoidance.

[0031] Part Three: Wheeled Robot Platform Used to execute firefighting operations after receiving instructions from the cloud platform. The wheel-legged robot uses a combination of wheels and legs for movement, employing its onboard robotic arm to grasp firefighting devices and precisely navigate to the fire location according to instructions; it is a key mobile device for firefighting operations.

[0032] The wheeled robot uses its robotic arm to adjust the direction of the fire extinguishing nozzles and spray the extinguishing medium at the core area of ​​the fire. The fire extinguishing process employs a tiered control strategy: the first stage involves spraying ABC dry powder to suppress open flames for 10 seconds; the second stage releases heptafluoropropane gas for asphyxiation, providing deep coverage to ensure complete extinguishment of the fire. After spraying, the robot continuously monitors the on-site status and reports it to the cloud until the fire is confirmed to be extinguished, at which point it returns to its standby position.

[0033] The wheeled robot is equipped with a servo-driven robotic arm, and its end effector integrates a dual-mode fire extinguishing device for dry powder and heptafluoropropane. The dry powder injection module is driven by a high-pressure gas cylinder, with a nozzle diameter of 20mm, a flow rate adjustment range of 2~10kg / s, and a coverage area of ​​5~15m². The gas injection module has a built-in heptafluoropropane storage tank, and the nozzle is designed with a porous diffusion structure to ensure uniform distribution of the extinguishing gas. The electromagnetic switching valve is a Festo MHJ9 series, with a response time of <10ms, supporting precise flow control via PWM signals.

[0034] After the firefighting mission is completed, the system enters the reset phase. The wheeled robot autonomously returns to the charging station to perform a self-test procedure, including checking the remaining fire extinguishing agent, calibrating the robotic arm joints, and assessing the battery status; the quadruped robot performs a secondary scan of the fire area, updating the damaged area markers and obstacle distribution in the environmental map; the cloud platform automatically generates an event analysis report, recording in detail key parameters such as the time of the fire, response delay, and consumption of fire extinguishing agents, and pushes it to the user terminal management system via API interface.

[0035] Fire extinguishing device: Used for precise control of the spraying of extinguishing agents. Mounted on the robotic arm of the wheeled robot, it is equipped with various extinguishing agent tanks depending on the type of fire.

[0036] During the system initialization phase, the quadruped robot moves at a low speed in an open area of ​​the target region. The LiDAR and inertial odometry work synchronously to perform a 3D scan of the target area, constructing an initial 3D map based on the LIO-SAM (Liberation Inertial Tightly Coupled Synchronous Localization and Mapping) algorithm. This algorithm optimizes the laser point cloud and IMU pose information through tight coupling, eliminating point cloud distortion caused by robot motion, and generating a 3D grid map containing obstacle heights, ground slopes, and semantic labels (such as "channel" and "equipment area"). Map data is stored on the robot's local solid-state drive and uploaded to the cloud platform in real time.

[0037] During the actual inspection phase, the robot updates the map every 10 seconds based on the initial map framework and real-time sensor data to ensure map accuracy. It retains the laser point cloud data from the most recent 20 seconds and performs joint optimization with the real-time IMU pose, dynamically correcting the coordinates of newly added obstacles (such as temporary shelves or relocated equipment) on the map, maintaining a positioning accuracy within ±2cm. The updated map is synchronized to the cloud, providing an accurate environmental model for multi-robot collaboration and global path planning.

[0038] like Figure 1 As shown, the specific process of using the system proposed in this invention for fire detection and response is as follows: Step 1: Create an elevation map centered on the quadruped robot. This invention constructs a 2.5D elevation map suitable for complex terrain environments during fires, and uses it as a unified environmental cognition basis for collaborative inspection and response by multi-quadruped robots. Compared to traditional two-dimensional planar grid maps, the 2.5D elevation map provides precise height information for each grid (each grid contains planar coordinates).<x, y> The map representation, including its corresponding height value z), accurately depicts three-dimensional features such as terrain undulations, obstacle heights, and rubble accumulation patterns while maintaining the advantages of lightweight storage and computational efficiency of grid maps. This map representation significantly reduces data redundancy and is specifically designed for the movement characteristics of quadruped robots (requiring the perception of foothold height and stability) and typical environmental features of fire scenarios (irregular collapses, sloping ground, and large obstacles). It provides crucial environmental model support for the robot's accurate perception, dynamic obstacle avoidance, collaborative path planning, and final fire source location and disposal in high-temperature, smoke, and complex rubble environments, greatly improving the system's autonomy, navigation efficiency, and mission reliability in extreme fire scenarios.

[0039] The specific mapping process is as follows: ① Robot pose estimation based on visual inertial odometry (VIO), such as Figure 3 As shown, the details are as follows: A quadruped robot equipped with an RGB camera and an IMU sensor array is used to construct a visual inertial odometry (VIO) system, enabling real-time synchronous acquisition of multi-source sensor data. ORB feature points in the images are obtained and tracked using a feature point extraction algorithm. Simultaneously, IMU data undergoes pre-integration processing. After acquiring the sensor data, a tightly coupled fusion strategy is used to deeply correlate visual feature points with IMU measurement information. Joint state estimation is then performed based on an extended Kalman filter algorithm to accurately calculate the robot's pose parameters in the unknown environment, providing a high-precision positioning foundation for subsequent elevation map construction.

[0040] ② Create and update a local map centered on the robot, such as... Figure 4 As shown, the details are as follows: The quadruped robot is equipped with a depth camera. It acquires 3D point cloud data of the environment through depth images and downsamples the point cloud data. Combined with the robot's pose information estimated in real-time in step ①, the downsampled point cloud data is transformed into a global coordinate system. Subsequently, the 3D point cloud is projected onto a 2D grid map plane of a preset resolution. For each grid cell... Obtain the position of each grid cell in the environment map in three-dimensional space. , means as follows: ; This includes grid cells On the map plane and Coordinates, and the corresponding terrain elevation values .

[0041] Collect all point clouds that fall within the horizontal range of this grid. , means as follows: ; Calculate the statistical values ​​(maximum and average) of the heights of these points as the height value of the grid cell. (Note: The choice of statistical value depends on the requirements. Taking the maximum value is beneficial for obstacle identification, while taking the average value is beneficial for terrain modeling.)

[0042] Therefore, the final local map is one that includes the horizontal position of each grid cell. and corresponding height value 2.5D elevation raster map.

[0043] The robot-centric map update and map fusion are completed during the movement process.

[0044] Map Update: As the robot moves and new sensor data is continuously collected, the system dynamically updates its local map. New areas: For newly observed areas, directly fill in the raster height value.

[0045] Repeated observation areas: For grids with existing height values, the height value `h(i, j)` of the grid is updated using a fusion strategy (such as taking the weighted average of the number of observations or Bayesian update) based on the new observation point cloud, so as to fuse multiple observation information and improve accuracy and robustness.

[0046] ③ Global collaborative elevation map construction: Each quadruped robot independently constructs a local 2.5D elevation map, which is then stitched and merged at the command and control center or a designated "leader" robot through the communication network between the robots and map fusion algorithms, ultimately forming a globally unified 2.5D elevation map covering the entire task area.

[0047] Input: Local elevation maps {LocalMap_k} (k=1,..,N) of each robot and their corresponding high-precision global poses are provided by VIO combined with loop closure detection, multi-robot relative positioning, or external global positioning sources such as UWB.

[0048] Map fusion process: 1. Coordinate Alignment: Using the pose information reported by each robot, all local maps are converted to a unified global coordinate system.

[0049] 2. Overlapping Area Handling: For overlapping areas covered by multiple local maps, an optimized fusion strategy is used to calculate the final height value of each grid cell in that area. The strategy needs to consider: Timeliness of observations: More recent observations are likely to be more reliable.

[0050] The quality of the observation can be weighted according to the confidence level of the sensor and the uncertainty of the robot pose estimation.

[0051] Fire scene characteristics: For example, special marking or processing of height information of suspected fire sources or high-temperature areas.

[0052] 3. Non-overlapping area filling: Directly fill local map data from a single source.

[0053] Output: A globally consistent 2.5D elevation map GlobalMap(x, y, h).

[0054] Dynamic updates: The global map is dynamically maintained and updated based on the local update information continuously uploaded by each robot.

[0055] This multi-robot map collaborative construction and fusion mechanism based on 2.5D elevation mapping is a key innovative element supporting subsequent multi-robot collaborative fire search, optimal path planning (considering terrain), task allocation (based on terrain accessibility), and collaborative response. It ensures that all robots share a unified and accurate three-dimensional environmental understanding in complex and dynamically changing fire scenes.

[0056] Step Two: This step proposes a accessibility quantification model for legged robots, transforming the geometric features of elevation maps into motion constraints for the robot, and designs a path planning algorithm based on a dynamic accessibility grid. This scheme solves the problem of traditional path planning neglecting the kinematic characteristics of legged robots in complex fire terrain, significantly improving the robot's passage efficiency and safety in scenarios such as ruins, slopes, and irregular surfaces.

[0057] The quadruped robot plans its path using an elevation map, enabling it to patrol along a designated trajectory while carrying visual sensors. The path planning employs a method based on... The obstacle avoidance algorithm, taking into account the passability characteristics of legged robots, can calculate the passability value of grid cells in the elevation map. The calculation formula is as follows: ; In the formula , value , The larger the value, the higher the passability of the current grid for the legged robot. This is the current grid height. The current grid slope, The current grid roughness, It depends on the maximum value of the robot's structure.

[0058] slope The height change rate of the grid in the x and y directions is calculated using the central difference method, and the magnitude of the slope vector (i.e., the slope value) is synthesized. ; in, and These represent the physical dimensions of the grid in the x and y directions, respectively. This slope value... Indicates the rate of change.

[0059] Roughness Calculated using the standard deviation of point cloud height within a raster cell: ;

[0060] in, The number of point clouds falling into the raster. Let k be the height of the k-th point in the point cloud. This represents the average height.

[0061] In this system's application scenario, hill climbing is not considered, and the surface roughness is a constant value, so the passability value... Only with grid height Related. When Value less than threshold If the area is deemed impassable, obstacle avoidance is performed during path planning.

[0062] Obstacle avoidance path planning: Accessibility map building: The grid markers are used as obstacles, forming a binary obstacle map. This is the passability threshold. .

[0063] The planning algorithm uses an improved A* algorithm to search for the optimal path on the reachability map, where: Cost function : ; in, From the starting point to the current node The actual cost of movement, This provides a heuristic estimate of the distance from the current node to the destination, using Euclidean distance.

[0064] Terrain adaptation improvement: In calculating movement costs At that time, a passability indicator was introduced. As a penalty, i.e. from the node Move to the current node The cost is: ; in, It is the Euclidean distance between the two nodes. Terrain penalty coefficient ( ).

[0065] Dynamic obstacle avoidance: Real-time integration of new perception data to update the accessibility map, triggering path replanning when new obstacles or terrain changes are detected.

[0066] Step 3: During the inspection process, the quadruped robot uses its high-performance edge computing platform to perform fire and smoke target detection on real-time environmental images.

[0067] This invention employs the YOLOv8 lightweight target detection algorithm, improving the model's inference efficiency on edge devices by structurally pruning and formatting the original model file. During detection, the robot calculates the confidence level of the flame target based on image recognition results. When the confidence level exceeds a preset threshold, it is determined to be a fire point; otherwise, it continues to perform the inspection task according to the planned trajectory. This mechanism enables rapid identification and accurate location of fires, providing a reliable basis for subsequent emergency response.

[0068] Step Four: After detecting a fire, the quadruped robot uploads alarm information and real-time image data to the local area network cloud platform via edge devices. The cloud platform supports intelligent dispatching in both manned and unmanned modes: In manned mode, the system synchronously pushes alarm information and real-time images to the monitoring terminal, allowing on-duty personnel to manually verify the fire situation based on environmental characteristics; after confirming the fire, the quadruped robot can be remotely controlled via the cloud platform to carry out the task with fire extinguishing equipment, and relevant departments can be linked to initiate emergency response procedures; when the fire exceeds the system's handling capacity, the platform automatically triggers the fire alarm communication function. In unmanned mode, the cloud platform automatically generates instructions based on preset rules to control the quadruped robot equipped with fire extinguishing equipment to carry out fire extinguishing operations, while simultaneously sending early warning information to staff through multiple channels (SMS, APP push, etc.), realizing human-machine and multi-machine collaborative handling of fire incidents.

[0069] Step 5: After the cloud platform triggers the wheeled robot to perform the firefighting task, it plans the optimal path for the robot to reach the target point based on the global elevation map and the coordinates of the fire point.

[0070] Multi-objective path planning based on risk perception: By associating environmental objects with fire risks through semantically enhanced elevation maps, path decision-making is optimized to achieve the dual objectives of "safety and efficiency", breaking through the limitations of traditional planning that only considers geometric obstacles.

[0071] Meanwhile, the platform uses a deep semantic segmentation network to enhance the environmental information of the global map. By recognizing scene objects such as shelves and power distribution cabinets, it dynamically labels semantic tags such as "flammable material storage area" and "electrical equipment area".

[0072] This semantic segmentation technology, based on a convolutional neural network architecture and combined with instance segmentation algorithms, achieves accurate classification and labeling of objects in complex scenes, providing path planning algorithms with multi-dimensional environmental data including terrain information, obstacle distribution, and risk levels. The optimized path planning results and control commands are transmitted in real-time to the wheeled robot's main control unit via a 5G communication link, ensuring its efficient and safe movement towards the fire. The specific technical solution is as follows: 1. Semantic map construction: The DeepLabV3+ semantic segmentation network is used to parse the global map and identify preset key objects (e.g., shelves, electrical boxes, etc.).

[0073] Define risk quantification rules and label each grid cell with a risk value. : ; 2. Cascaded Filtering Path Planning Using the improved D* Lite algorithm, the cost function integrates 3D accessibility and fire risk.

[0074] Step 1: Terrain accessibility screening Exclude all ( A grid (with a static stability threshold for quadruped robots) is used to generate terrain that can be accessed through the region.

[0075] Step 2: Fire Risk Screening Exclude all areas where the terrain allows passage. ( The grid (NFPA risk assessment standard) is used to generate safe zones.

[0076] Output: Set of safety grids: ; The essence of this step is to construct a three-dimensional corridor that robots can safely traverse.

[0077] Step 3: Shortest Path Search Within the safe region S, the traditional A* algorithm is used to plan the shortest path, with the cost function being the Euclidean distance: ; in, For actual mobility costs, The heuristic function is as follows: ; in, Representing the The location of the step It is Step position, These are the coordinates of the target point.

[0078] 3. Communication and Execution The robot can receive a path point sequence, maximum speed, and risk avoidance instructions via a 5G network with a transmission latency of ≤50ms.

[0079] Step Six: After the wheeled robot arrives at the fire location, it immediately activates the fire extinguishing device and performs fire extinguishing operations such as high-pressure spraying or dry powder covering.

[0080] In summary, this invention employs a quadruped robot for inspection, which possesses strong maneuverability and obstacle-crossing capabilities, enabling flexible movement in complex terrain environments and making it suitable for inspection tasks. Simultaneously, a wheeled robot equipped with a robotic arm is used to grasp fire extinguishing devices for fire suppression. Compared to the quadruped robot, the wheeled robot offers faster movement speed and greater stability, enabling rapid and precise fire suppression. This design solves the problem of personnel being unable to promptly reach the fire scene and effectively handle fires in unmanned areas, significantly improving the efficiency and safety of firefighting operations by using robots to replace manual labor.

[0081] The fire and smoke detection system proposed in this invention is developed based on a cloud-edge-device architecture. The edge side consists of a vision sensor mounted on a quadruped robot for image data acquisition; the side side comprises an edge computing device installed on the robot's back for image detection and recognition, uploading the results to a cloud platform; the cloud side is a host computer control program platform that can display the images uploaded by the quadruped robot's vision sensor in real time and analyze the recognition results. If a fire is detected, the platform controls the quadruped robot to carry firefighting equipment for disposal. This architecture significantly reduces the cloud computing load and effectively alleviates the bandwidth pressure on data transmission. The cloud-based control system proposed in this invention can plan inspection routes for a quadruped robot according to preset procedures and display monitoring images in real time. Furthermore, the system can establish remote communication with fire departments via a communication module. When the fire exceeds the robot's ability to handle it independently, the system can directly issue a warning to the fire department, enabling human intervention in fire suppression. The entire process is simple to operate and easy to control.

[0082] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content regarding a multi-robot collaborative fire detection and response system and method, as well as some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0083] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MCU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0084] This invention provides a concept and method for a fire detection and response system based on multi-robot collaboration. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A fire detection and response system based on multi-robot collaboration, characterized in that, include: Quadruped robot platforms, cloud platforms, and wheeled robot platforms; among them... The quadruped robot platform is used to detect fires and send information to the cloud platform in real time; the cloud platform receives information from the quadruped robot platform and schedules and controls the quadruped robot platform and the wheeled robot platform; the wheeled robot platform is used to receive instructions from the cloud platform to perform fire extinguishing operations.

2. The fire detection and response system based on multi-robot collaboration according to claim 1, characterized in that, The quadruped robot platform includes: A quadruped robot, and a lidar, inertial odometry, vision sensor, fire and smoke detection module, alarm module, and main control module installed on the quadruped robot; wherein, The quadruped robot is a vehicle with a lidar that scans upwards in a hemispherical shape mounted on its top, a vision sensor mounted on its head, and an inertial odometer for measuring motion status. The fire and smoke detection module performs multi-scale feature extraction based on the images acquired by the visual sensor to obtain the bounding box coordinates of the flame and smoke and the corresponding confidence level. The alarm module triggers an alarm based on the confidence level. The main control module generates a local map based on information obtained from lidar, inertial odometry, and visual sensors, interacts with the cloud platform, and controls the overall operation of the quadruped robot platform.

3. A fire detection and response system based on multi-robot collaboration according to claim 2, characterized in that, The cloud platform includes: Based on the local maps generated by all quadruped robot platforms, a global map is generated, and inspection path planning is performed for collaborative control of the quadruped robot platforms. Based on the fire information obtained from all quadruped robot platforms, a disposal path is planned to control the wheeled robot platforms to handle the fire.

4. A fire detection and response system based on multi-robot collaboration according to claim 3, characterized in that, The wheeled robot platform includes: A wheeled robot and a fire extinguishing device mounted on the wheeled robot are used to execute the scheduling commands of the cloud platform to handle fire situations.

5. A method for fire detection and response based on multi-robot collaboration, characterized in that, Using the system described in any one of claims 1-4 for fire detection and response includes the following steps: Step 1: Create a local elevation map centered on the quadruped robot and merge and update it into a global elevation map; Step 2: Construct a quantitative model of the quadruped robot's passability and perform inspection path planning; Step 3: Use a quadruped robot to conduct inspections along the planned inspection path and detect fire and smoke targets. Step 4: Obtain the target detection results of fire and smoke through the cloud platform and plan the fire response path; Step 5: Use the wheeled robot to handle the fire according to the planned fire handling path.

6. A fire detection and handling method based on multi-robot collaboration according to claim 5, characterized in that, Step 1, which involves creating a local elevation map centered on the quadruped robot and merging and updating it into a global elevation map, includes: Step 1-1: Perform pose estimation for the quadruped robot based on visual inertial odometry; Steps 1-2: Create a local elevation map centered on the quadruped robot and update it in real time; Steps 1-3 involve fusing the local elevation maps created by all quadruped robots to construct a global elevation map.

7. A fire detection and handling method based on multi-robot collaboration according to claim 6, characterized in that, The steps 1-2 described above, which involve creating and updating a local elevation map centered on the quadruped robot in real time, include: Step 1-2-1, Local Elevation Map Construction, as detailed below: The three-dimensional point cloud data of the environment is calculated from the depth image acquired by the quadruped robot, and the three-dimensional point cloud data is downsampled. Based on the quadruped robot pose obtained in step 1-1, the downsampled 3D point cloud data is converted to the global coordinate system; Project the 3D point cloud onto a 2D raster map plane of a preset resolution; for each raster cell... The position of each grid cell in the computational environment map in three-dimensional space. , means as follows: ； Among them, grid unit The coordinates on the environmental map plane are: and The corresponding terrain height value is ; Collect all point clouds that fall within the horizontal range of this grid. , means as follows: ； The height of the grid cell is calculated based on a preset statistical value of the height of all points. ; The final local map contains the horizontal position of each raster cell. and corresponding height value Local elevation map; Step 1-2-2, local elevation map update, details are as follows: For new areas observed by the quadruped robot's sensors, the grid height value is directly filled in; For recurring regions observed by the quadruped robot's sensors—that is, grids with existing height values—the height values ​​of these grids are updated using a pre-defined fusion strategy based on the new observation point cloud. .

8. A fire detection and handling method based on multi-robot collaboration according to claim 7, characterized in that, The fusion of local elevation maps created for all quadruped robots as described in steps 1-3 includes: Step 1-3-1: Coordinate alignment. Using the pose information of each quadruped robot, all local elevation maps are converted to a unified global coordinate system. Step 1-3-2, Overlapping Area Processing: For overlapping areas of different local elevation maps, a preset fusion strategy is used to calculate the final height value of each grid cell in the overlapping area. Step 1-3-3: Fill non-overlapping areas by directly filling with local elevation map data from a single source; Steps 1-3-4: Dynamic update. The global elevation map is updated in real time based on the local elevation maps uploaded by each quadruped robot.

9. A fire detection and handling method based on multi-robot collaboration according to claim 8, characterized in that, Step 2, which involves constructing a quadruped robot passability quantification model and planning inspection paths, includes: Step 2-1: Calculate the passability values ​​for the raster cells in the global elevation map, as follows: ； Among them, grid unit Passability value , For the current grid cell height, For the current grid cell The slope, For the current grid cell roughness, , and It depends on the preset threshold of the quadruped robot structure. , and For control coefficients; Slope value The calculation method is as follows: ； in, and Representing grid cells Physical dimensions in the x and y directions; roughness The calculation method is as follows: ； in, To fall into the grid cell The number of point clouds, For the first point cloud The height of each point Average height; Step 2-2: Plan the inspection path based on the passability value. When the passability value of a grid cell is less than the threshold, the area is determined to be impassable, and obstacle avoidance is performed in the inspection path planning, as follows: Step 2-2-1, Accessibility Map Construction, will The grid cells are marked as obstacles, forming a binary obstacle map, where This is the passability threshold; Step 2-2-2: The inspection path planning uses the improved A* method to search for the optimal path on the accessibility map. Let the cost function of the inspection path be... , means as follows: ； in, From the starting point to the current node The actual cost of movement, This is a heuristic estimate of the distance from the current node to the destination. Step 2-2-3, Terrain Adaptation Improvement, involves calculating the actual movement cost. At that time, a passability indicator was introduced. As a penalty, that is, from the previous node Move to the current node The cost is: ； in, It is the Euclidean distance between the two nodes. This is the terrain penalty coefficient; Step 2-2-4, Dynamic obstacle avoidance: When a new obstacle or terrain change is detected, path replanning is triggered.

10. A fire detection and handling method based on multi-robot collaboration according to claim 9, characterized in that, Step 4, which involves planning the fire response route, includes: Step 4-1, semantic map construction, as follows: A semantic segmentation network is used to parse the global elevation map and identify preset key objects; Define risk quantification rules, and assign each grid cell based on the identified key objects. Mark risk value ; Step 4-2, cascading filtering path planning, as follows: Step 4-2-1: Terrain accessibility filtering, i.e., excluding all... The grid cells generate terrain that can be accessed through the region; Step 4-2-2, Fire Risk Screening, details are as follows: Exclude all areas where the terrain allows passage. The grid is used to generate a safe area, which is a set of safe grid cells. , means as follows: ； in, Risk threshold; Step 4-2-3, shortest path search, details are as follows: In the safe area The A* method is used to plan the shortest path, and the cost function is given by... , means as follows: ； in, For actual mobility costs, A heuristic function is represented as follows: ； in, Representing the The location of the step, It is Step position, These are the coordinates of the target point.

Images