A fire detection and disposal system and method based on multi-robot cooperation

Abstract

The application provides a fire detection and disposal system and method based on multi-robot cooperation, wherein the system comprises a quadruped robot platform, a cloud platform and a wheel-foot robot platform; the quadruped robot platform is used for detecting a fire and sending information to the cloud platform in real time; the cloud platform receives the information of the quadruped robot platform, and schedules and controls the quadruped robot platform and the wheel-foot robot platform; the wheel-foot robot platform is used for receiving the instructions of the cloud platform and performing fire extinguishing disposal; and the method uses the above system to detect and dispose the fire.

Description

TECHNICAL FIELD

[0001] The present application relates to a fire detection and disposal system and method, in particular to a fire detection and disposal system and method based on multi-robot cooperation. BACKGROUND

[0002] In many unmanned environments, such as warehouses, field base stations, unmanned operation areas of large factories, etc., once a fire occurs, due to the lack of monitoring means, it is often not possible to dispose of the fire as soon as it occurs. The traditional fixed fire detection equipment is limited by the installation position, has a detection blind area, and cannot be flexibly moved to the fire source for fire extinguishing operation until the fire spreads significantly. The delay in discovering the fire and the lack of mobility of the fire extinguishing device not only causes huge property losses, but also poses a serious threat to the surrounding environment and personnel safety. The prior art lacks a fully automatic fire detection and disposal system, and cannot solve the problem of timely discovery and handling of fire in unmanned environments. SUMMARY

[0003] The technical problem to be solved by the present application is to provide a fire detection and disposal system and method based on multi-robot cooperation to solve the problems of the prior art.

[0004] In order to solve the above technical problems, the present application discloses a fire detection and disposal system and method based on multi-robot cooperation, wherein the system comprises:

[0005] a quadruped robot platform, a cloud platform and a wheel-foot robot platform; wherein,

[0006] The quadruped robot platform is used to detect fire and send information to the cloud platform in real time; the cloud platform receives information from the quadruped robot platform, schedules and controls the quadruped robot platform and the wheel-foot robot platform; the wheel-foot robot platform is used to receive instructions from the cloud platform and execute fire extinguishing disposal.

[0007] Further, the quadruped robot platform comprises:

[0008] a quadruped robot, and a laser radar, an inertial odometry, a vision sensor, a fire smoke detection module, an alarm module and a main control module arranged on the quadruped robot; wherein,

[0009] The quadruped robot is a carrier, a laser radar scanning upward in a hemispherical shape is installed on the top, a vision sensor is installed on the head, and an inertial odometry for measuring the motion state is carried;

[0010] The fire smoke detection module performs multi-scale feature extraction according to the image obtained by the vision sensor, obtains the boundary box coordinates of the flame and the smoke and the corresponding confidence;

[0011] The alarm module alarms according to the confidence level.

[0012] The master module generates a local map according to information obtained by the laser radar, the inertial odometry and the visual sensor, interacts with the cloud platform, and controls the overall operation of the quadruped robot platform.

[0013] Further, the cloud platform comprises:

[0014] According to the local maps generated by all the quadruped robot platforms, a global map is generated, and a patrol path is planned for cooperative control of the quadruped robot platforms.

[0015] According to the fire information obtained by all the quadruped robot platforms, a disposal path is planned for controlling the wheel-foot robot platform to dispose of the fire.

[0016] Further, the wheel-foot robot platform comprises:

[0017] The wheel-foot robot and the fire extinguishing device arranged on the wheel-foot robot are used to execute the scheduling command of the cloud platform and dispose of the fire.

[0018] The application also proposes a fire detection and disposal method based on multi-robot cooperation, which uses the aforementioned system to detect and dispose of the fire, and comprises the following steps:

[0019] Step 1: establishing a local elevation map centered on the quadruped robot and fusing and updating it into a global elevation map;

[0020] Step 2: constructing a quadruped robot passability quantification model and planning a patrol path;

[0021] Step 3: using the quadruped robot to patrol according to the planned patrol path and detecting a fire smoke target;

[0022] Step 4: obtaining the fire smoke target detection result through the cloud platform and planning a fire disposal path;

[0023] Step 5: using the wheel-foot robot to dispose of the fire according to the planned fire disposal path.

[0024] Further, step 1 of establishing a local elevation map centered on the quadruped robot and fusing and updating it into a global elevation map comprises:

[0025] Step 1-1: estimating the pose of the quadruped robot based on a visual inertial odometry;

[0026] Step 1-2: establishing a local elevation map centered on the quadruped robot and updating it in real time;

[0027] Step 1-3, fuse the local elevation maps of all quadruped robots to build a global elevation map.

[0028] Further, the establishment of the local elevation map centered on the quadruped robot and the real-time updating in step 1-2 include:

[0029] Step 1-2-1, local elevation map construction, specifically as follows:

[0030] Through the depth image obtained by the quadruped robot, the three-dimensional point cloud data of the environment is calculated and the three-dimensional point cloud data is down-sampled;

[0031] According to the quadruped robot pose obtained in step 1-1, the down-sampled three-dimensional point cloud data is converted to the global coordinate system;

[0032] Project the three-dimensional point cloud onto a two-dimensional grid map plane with a predetermined resolution; for each grid cell , calculate the position of each grid cell in the three-dimensional space in the environment map , represented as follows:

[0033] ;

[0034] Wherein, the coordinates of the grid cell on the environment map plane are and , and the height value corresponding to the terrain is ;

[0035] Collect all point clouds falling within the horizontal range of the grid , represented as follows:

[0036] ;

[0037] Calculate the predetermined statistical value of the height of all points as the height value of the grid cell ;

[0038] The finally established local map is a local elevation map containing the horizontal position of each grid cell and the corresponding height value ;

[0039] Step 1-2-2, local elevation map updating, specifically as follows:

[0040] For the new area observed by the sensor of the quadruped robot, directly fill in the grid height value;

[0041] For the repeated area observed by the sensor of the quadruped robot, that is, the grid with existing height value, according to the new observation point cloud, update the height value of the grid by using a predetermined fusion strategy .

[0042] Further, the fusion of the local elevation maps of all quadruped robots in steps 1-3 includes:

[0043] Step 1-3-1, coordinate alignment, using the pose information of each quadruped robot, converting all local elevation maps to a unified global coordinate system;

[0044] Step 1-3-2, overlapping area processing, for the overlapping area of different local elevation maps, using a preset fusion strategy to calculate the final height value of each grid in the overlapping area;

[0045] Step 1-3-3, non-overlapping area filling, directly filling the local elevation map data from a unique source;

[0046] Step 1-3-4, dynamic update, the global elevation map is updated in real time according to the local elevation map uploaded by each quadruped robot in real time.

[0047] Further, the construction of the passability quantification model and the inspection path planning of the quadruped robot in step 2 includes:

[0048] Step 2-1, calculating the passability value of the grid unit in the global elevation map, as follows:

[0049] ;

[0050] Wherein, the passability value of the grid unit is , is the height of the current grid unit , is the slope of the current grid unit , is the roughness of the current grid unit , , and are preset threshold values depending on the structure of the quadruped robot, , and are control coefficients;

[0051] The calculation method of the slope value is as follows:

[0052] ;

[0053] Wherein, and represent the physical dimensions of the grid unit in the x and y directions respectively;

[0054] roughness The calculation method is as follows:

[0055] ;

[0056] in, To fall into the grid cell The number of point clouds, For the first point cloud The height of each point Average height;

[0057] Step 2-2: Plan the inspection path based on the passability value. When the passability value of a grid cell is less than the threshold, the area is determined to be impassable, and obstacle avoidance is performed in the inspection path planning, as follows:

[0058] Step 2-2-1, Accessibility Map Construction, will The grid cells are marked as obstacles, forming a binary obstacle map, where This is the passability threshold;

[0059] Step 2-2-2: The inspection path planning uses the improved A* method to search for the optimal path on the accessibility map. Let the cost function of the inspection path be... , means as follows:

[0060] ;

[0061] in, From the starting point to the current node The actual cost of movement, This is a heuristic estimate of the distance from the current node to the destination.

[0062] Step 2-2-3, Terrain Adaptation Improvement, involves calculating the actual movement cost. At that time, a passability indicator was introduced. As a penalty, that is, from the previous node Move to the current node The cost is:

[0063] ;

[0064] in, It is the Euclidean distance between the two nodes. This is the terrain penalty coefficient;

[0065] Step 2-2-4, Dynamic obstacle avoidance: When a new obstacle or terrain change is detected, path replanning is triggered.

[0066] Further, the fire disposition path planning in step 4 includes:

[0067] Step 4-1, semantic map construction, specifically as follows:

[0068] The global elevation map is parsed using a semantic segmentation network to identify preset key objects;

[0069] Define risk quantification rules, and according to the identified key objects, assign a risk value to each grid cell ;

[0070] Step 4-2, cascade screening path planning, specifically as follows:

[0071] Step 4-2-1, terrain passability screening, i.e. excluding all grid cells to generate terrain passable areas;

[0072] Step 4-2-2, fire risk screening, specifically as follows:

[0073] Within the terrain passable area, all grids are excluded to generate a safe area, i.e. a set of safe grid cells , represented as follows:

[0074] ;

[0075] wherein, is a risk threshold;

[0076] Step 4-2-3, shortest path search, specifically as follows:

[0077] Within the safe area , the A* method is used to plan the shortest path, with the cost function being , represented as follows:

[0078] ;

[0079] wherein, is the actual moving cost, is the heuristic function, represented as follows:

[0080] ;

[0081] wherein, represents the position of the th step, is the position of the th step, is the target point coordinate.

[0082] Advantages: ​

[0083] 1. This invention constructs a heterogeneous collaborative operation system of quadruped robots and wheeled-legged robots. Legged robots are assigned inspection and response tasks based on their terrain adaptability. Quadruped robots are used for fire inspection, while wheeled-legged robots are used for fire response. The quadruped robots, leveraging their excellent terrain adaptability, autonomously construct environmental maps and perform environmental inspections, intelligent fire and smoke identification, precise positioning, and alarm tasks. Data is uploaded to the cloud in real time via a 5G network. The wheeled-legged robots, with their high mobility and large load capacity due to their wheeled movement, are equipped with fire extinguishing devices and respond rapidly to commands from the cloud platform, performing precise fire extinguishing tasks.

[0084] 2. This invention employs a cloud-based intelligent management platform that integrates multimodal data fusion and analysis functions. It supports real-time display of video streams transmitted by the legged robot, visual detection results of fire and smoke, and the robot's real-time positioning. Users can remotely control the robot through text input, voice, and a visual interface, flexibly adjusting operational strategies to form an efficient human-machine interaction loop.

[0085] 3. This invention employs elevation map modeling technology to establish a local elevation map more adapted to the movement characteristics of quadruped robots, dynamically generating local 3D environment models that fit complex terrain. Through map fusion and map updating technologies, a robot-centric local elevation map is established. Compared to traditional point cloud maps and voxel maps, elevation map maps only need to calculate the height value of the highest point within the grid cell, thus significantly reducing the computational resource consumption during mapping and completely preserving the elevation information of environmental obstacles. This provides legged robots with accurate environmental perception data, supports dynamic obstacle avoidance and path planning, and significantly improves the efficiency and reliability of emergency response in complex fire scenarios. Attached Figure Description

[0086] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0087] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0088] Figure 2 This is a three-dimensional structural diagram of a quadruped robot.

[0089] Figure 3 This is a schematic diagram of the pose calculation process for a quadruped robot.

[0090] Figure 4 This is a schematic diagram of the process of generating and updating local maps for a quadruped robot. Detailed Implementation

[0091] The overall idea of the system is as follows: the quadruped robot carries high-precision visual sensors (RGB camera), laser radar, navigation control unit and wireless communication module; the wheel-foot robot is equipped with a fire extinguishing device (dry powder / gas injection system), a mechanical arm, a path planning module and a high-capacity battery; the cloud platform integrates an AI analysis engine, a real-time map database and a remote control interface. Specifically, the system mainly consists of a quadruped robot platform, a cloud platform and a wheel-foot robot platform.

[0092] First part, quadruped robot platform:

[0093] As the core mobile platform of the entire system, the quadruped robot carries laser radar, visual sensor, inertial odometer and fire smoke detection module. With stable mobile performance, it can adapt to complex and variable terrains. Its main control module is responsible for device coordination, data processing and communication interaction, etc., to ensure efficient and stable operation of the system.

[0094] The quadruped robot, as shown in Figure 2 The robot is selected for its high stability and strong load capacity. Its leg joints are designed with six degrees of freedom, with a load capacity of 30 kg, suitable for complex terrains such as factories and warehouses. The robot's main body is equipped with a high-precision laser radar on the top, achieving a hemispherical ultra-wide angle scan, which can measure the 360° three-dimensional space above the radar. The visual sensor uses a front camera, which is installed on the adjustable position of the robot's head to obtain the best field of view. At the same time, it is equipped with an inertial odometer to accurately measure the motion state of the robot. All sensor data is transmitted to the cloud platform in real time, running ROS (Robot Operating System) for multi-source data fusion and real-time processing.

[0095] Visual sensor module: installed on the quadruped robot platform, used to collect RGB images in space in real time during the movement of the quadruped robot, and transmit clear and comprehensive image data to the fire smoke detection module, providing key data sources for accurate identification of smoke sources and fire sources. It is the core component of the data acquisition front end in the entire fire detection process.

[0096] Fire smoke detection module: installed on the quadruped robot platform, using deep learning algorithms to perform in-depth analysis on images transmitted by the visual sensor. After learning from a large number of images, it accurately identifies smoke sources and fire sources and gives recognition confidence, providing key evidence for fire judgment. It is the core operation part of fire detection.

[0097] The fire smoke visual detection module is realized based on an improved YOLO v8 model, and the algorithm is optimized for the characteristics of fire and smoke. The model embeds a CBAM attention module in the CSPDarknet53 backbone network, improving the detection capability of flame edge features (such as flickering texture) and semi-transparent smoke. The visual data processing unit has two levels of AI recognition algorithms. First, the YOLO v7 model is used to detect fire in real-time returned images. This model is trained based on a dataset of 20,000 multi-scene flame images, covering complex scenes such as backlight, occlusion, and dynamic blur. It can achieve an accuracy of 85% under complex lighting conditions. During detection, the model performs multi-scale feature extraction on the input image and outputs the bounding box coordinates and corresponding confidence of fire and smoke. To meet the small target detection requirements, the model uses a feature fusion strategy to improve the fine-grained recognition accuracy and ensure that the detection rate of fire and smoke meets the actual application requirements. If no open flame is detected, the U-Net semantic segmentation model is used to analyze the semi-transparent smoke area in the image, and cross-validation is performed with the smoke sensor data to effectively eliminate dust, water mist, and other interference factors.

[0098] Alarm module: set on the quadruped robot platform, used to alarm immediately when the confidence output by the fire smoke detection module reaches the preset threshold. Send fire coordinates, detection time, and other information to the cloud platform through wireless communication, and trigger local sound and light alarm to alert the surrounding and timely deliver fire information.

[0099] Second part, cloud platform:

[0100] Used as the intelligent hub of the system, it receives real-time quadruped robot data and stores it. Through the visualization interface, the operator can view the patrol trajectory, image, and alarm details. It can automatically generate quadruped robot control instructions based on the fire, dispatch it to perform fire extinguishing tasks, and realize system collaborative operation.

[0101] Based on the environment map generated by the quadruped robot platform, the cloud platform uses Dijkstra algorithm to plan the global inspection path, highlights the high-risk areas, and generates a covering movement trajectory. The map is divided according to the area function, such as the storage area and passageway of the warehouse. The cloud platform automatically generates a global inspection path based on the map features, sets the robot inspection period and priority strategy. The path is sent to the quadruped robot navigation module in the form of a grid trajectory, giving priority to covering high-risk areas such as flammable material storage areas and electrical equipment areas. The quadruped robot moves along the planned path, the leg joints are driven by hydraulic pressure to adaptively adjust the gait, and the navigation module combines real-time sensor data to realize dynamic obstacle avoidance.

[0102] Third part, quadruped robot platform:

[0103] The wheel-foot robot is used to receive cloud platform instructions and perform fire extinguishing operations. The wheel-foot robot uses a wheel-foot combination movement method, uses the mounted mechanical arm to clamp the fire extinguishing device, and accurately navigates to the fire point according to the instructions, which is a key mobile device for fire extinguishing operations.

[0104] The wheel-foot robot adjusts the direction of the fire extinguishing nozzle through the mechanical arm, and sprays the fire extinguishing medium to the core area of the fire source. The fire extinguishing process adopts a hierarchical control strategy: in the first stage, ABC dry powder is sprayed to suppress open flames, and the duration is 10 seconds; in the second stage, heptafluoropropane gas is released to suffocate the fire, and deep coverage is implemented to ensure that the fire is completely extinguished. After the spraying is completed, the robot continues to monitor the site state and feeds back to the cloud until it is confirmed that the fire is eliminated and returns to the standby position.

[0105] The wheel-foot robot is equipped with a servo mechanical arm, and the end effector integrates a dry powder and heptafluoropropane dual-mode fire extinguishing device. The dry powder spraying module is driven by a high-pressure gas cylinder, the nozzle diameter is 20 mm, the flow adjustment range is 2-10 kg / s, and the coverage area is 5-15 m²; the gas spraying module has a built-in heptafluoropropane storage tank, and the nozzle is designed as a porous diffusion structure to ensure uniform distribution of the fire extinguishing gas. The electromagnetic on-off valve selects the Festo MHJ9 series, with a response time of <10 ms, supporting precise control of the flow through PWM signals.

[0106] After the fire extinguishing task is completed, the system enters the reset stage. The wheel-foot robot returns to the charging station autonomously to perform a self-checking program, including fire extinguishing agent remaining amount detection, mechanical arm joint calibration, and battery state evaluation; the quadruped robot performs a second scan on the fire area, updates the damaged area markers and obstacle distribution in the environment map; the cloud platform automatically generates an event analysis report, which records key parameters such as fire occurrence time, response delay, and fire extinguishing medium consumption in detail, and pushes it to the user terminal management system through the API interface.

[0107] Fire extinguishing device: used for precise control of fire extinguishing medium spraying. It is installed on the mechanical arm of the wheel-foot robot and is equipped with multiple fire extinguishing medium tanks according to the type of fire.

[0108] In the system initialization stage, the quadruped robot moves in the open area of the target area at low speed, and the laser radar and inertial odometer work synchronously to scan the target area in three dimensions, and an initial three-dimensional map is constructed based on the LIO-SAM (laser inertial tightly coupled simultaneous localization and mapping) algorithm. The algorithm optimizes the laser point cloud and IMU pose information through tight coupling, eliminates the point cloud distortion caused by robot movement, and generates a 3D grid map containing obstacle height, ground slope, and semantic labels (such as "passage" and "equipment area"). Map data is stored in the local solid state disk of the robot and is uploaded to the cloud platform in real time.

[0109] In the actual inspection stage, the robot updates the map every 10 seconds according to the initial map framework combined with real-time sensor data, to ensure the accuracy of the map. The laser point cloud data of the last 20 seconds is retained, and the real-time IMU pose is jointly optimized to dynamically correct the coordinates of the newly added obstacles (such as temporary shelves and equipment displacement) in the map, and the positioning accuracy is maintained within ±2 cm. The updated map is synchronized to the cloud to provide an accurate environment model for multi-robot cooperation and global path planning.

[0110] As shown in Figure 1 , the method for fire detection and disposal using the system proposed in the present application has the following specific process:

[0111] Step 1: Establish an elevation map centered on a quadruped robot

[0112] The present application constructs a 2.5D elevation map suitable for complex terrain environments of fire, and uses it as a unified environment cognition basis for multi-quadruped robot cooperative inspection and disposal. Compared with traditional two-dimensional plane grid maps, the 2.5D elevation map gives each grid accurate height information (each grid contains plane coordinates <x, y> and corresponding height value z), while maintaining the advantages of lightweight storage and computing efficiency of grid maps, it can accurately depict three-dimensional features such as terrain undulation, obstacle height, and debris accumulation pattern. This map expression significantly reduces data redundancy, and is particularly suitable for the motion characteristics of quadruped robots (need to perceive the height of the landing point and stability) and the typical environmental characteristics of fire scenes (irregular collapse, inclined ground, large obstacles). It provides crucial environmental model support for subsequent robot precise perception, dynamic obstacle avoidance, cooperative path planning, and ultimately fire source positioning and disposal in high-temperature, smoke, and complex debris environments, greatly improving the autonomy, navigation efficiency, and task reliability of the system in extreme fire scenarios.

[0113] The specific mapping process is as follows:

[0114] ① Robot pose estimation based on visual inertial odometry (VIO), as shown in Figure 3 , specifically as follows:

[0115] The quadruped robot is equipped with an RGB camera and an IMU sensor group to construct a visual inertial odometry (VIO) system, realizing real-time synchronous acquisition of multi-source sensor data. Feature point extraction algorithm is used to obtain ORB feature points in the image and track them; at the same time, the IMU data is pre-integrated. After acquiring the sensor data, the visual feature points and IMU measurement information are deeply related through a tight coupling fusion strategy, and joint state estimation is performed based on the extended Kalman filter algorithm to accurately calculate the pose parameters of the robot in the unknown environment, providing a high-precision positioning basis for subsequent elevation map construction.

[0116] ② Establish a local map centered on the robot and update it, as shown in Figure 4 , as follows:

[0117] The quadruped robot carries a depth camera. Through the depth image, it obtains the three-dimensional point cloud data of the environment and performs downsampling processing on the point cloud data. Combined with the real-time estimated robot pose information in step ①, the down-sampled point cloud data is converted to the global coordinate system. Then, the three-dimensional point cloud is projected onto a two-dimensional grid map plane with a preset resolution. For each grid cell , the position of each grid cell in the three-dimensional space in the environment map is obtained , and is represented as follows:

[0118] ;

[0119] , which includes the and coordinates of the grid cell on the map plane, and the height value corresponding to the terrain.

[0120] Collect all point clouds falling within the horizontal range of the grid , and represent them as follows:

[0121] ;

[0122] Calculate the statistical value (maximum value, average value) of the heights of these points as the height value of the grid cell. (Note: choose which statistical value according to the requirements, take the maximum value for obstacle recognition, and take the average value for terrain modeling).

[0123] Therefore, the final local map established is a 2.5D elevation grid map containing the horizontal position of each grid cell and the corresponding height value .

[0124] In the motion process, the map update and map fusion centered on the robot are completed.

[0125] Map update: as the robot moves and new sensor data is continuously collected, the system dynamically updates its local map:

[0126] New area: for newly observed areas, directly fill in the grid height value.

[0127] Repeated observation area: for the grid with existing height value, according to the new observation point cloud, adopt the fusion strategy (such as taking the weighted average of the observation times, Bayesian update) to update the height value `h(i, j)` of the grid, to fuse the multiple observation information, improve the accuracy and robustness.

[0128] ​③ Global collaborative height map construction:

[0129] The local 2.5D height maps constructed by each quadruped robot are spliced and fused at the background command and control center or the designated "leader" robot through the communication network between robots and map fusion algorithms, and finally a global unified 2.5D height map covering the entire task area is formed.

[0130] Input: Local height maps {LocalMap_k} (k=1,..,N) of each robot and their corresponding high-precision global poses provided by VIO combined with loop detection, multi-robot relative positioning or external global positioning sources such as UWB, etc.

[0131] Map fusion process:

[0132] 1. Coordinate alignment: Use the pose information reported by each robot to convert all local maps to a unified global coordinate system.

[0133] 2. Overlapping area processing: For overlapping areas covered by multiple local maps, an optimized fusion strategy is used to calculate the final height value of each grid in the area. The strategy needs to consider:

[0134] Observation timeliness: Updated observations may be more reliable.

[0135] Observation quality: Weighted according to sensor confidence and robot pose estimation uncertainty.

[0136] Fire scene characteristics: For example, special marking or processing of height information in suspected fire sources or high-temperature areas.

[0137] 3. Non-overlapping area filling: Directly fill the data from the only source local map.

[0138] Output: Global consistent 2.5D height map GlobalMap(x, y, h).

[0139] Dynamic update: The global map is dynamically maintained and updated according to the continuous local update information uploaded by each robot.

[0140] This multi-robot map collaborative construction and fusion mechanism based on 2.5D height map representation is a key innovative link to support subsequent multi-robot collaborative fire search, optimal path planning (considering terrain), task allocation (based on terrain accessibility) and collaborative disposal. It ensures that all robots share a unified and accurate three-dimensional environment awareness in a complex and dynamic fire scene.

[0141] Step two:

[0142] This step proposes a passability quantization model for the motion characteristics of a legged robot, which converts the geometric features of the elevation map into motion constraints for the robot, and designs a path planning algorithm based on a dynamic reachability grid. This scheme solves the problem of ignoring the kinematic characteristics of legged robots in traditional path planning in complex fire terrain, significantly improving the efficiency and safety of the robot in scenes such as ruins, slopes, and irregular surfaces.

[0143] The quadruped robot performs path planning in the elevation map, and carries a visual sensor to patrol along the trajectory. The path planning adopts an obstacle avoidance algorithm based on the A* algorithm, taking into account the passability characteristics of the legged robot, which can calculate the passability value of the grid cell in the elevation map as follows:

[0144] ;

[0145] wherein , the value , the larger the value, the higher the passability of the current grid for the legged robot, is the height of the current grid, is the slope of the current grid, is the roughness of the current grid, is the maximum value depending on the structure of the robot.

[0146] Slope The height change rate of the grid in the x and y directions is calculated by the central difference method, and the size of the slope vector (i.e. the slope value) is synthesized:

[0147] ;

[0148] wherein and represent the physical dimensions of the grid in the x and y directions. The slope value represents the height change rate.

[0149] Roughness is calculated by the standard deviation of the point cloud height within the grid cell:

[0150] ;

[0151] wherein is the number of point clouds falling into the grid, is the height of the kth point in the point cloud, is the average height.

[0152] In the use scenario of the system, the climbing task is not considered, and the surface roughness is a constant value, so the passability value​​ Only related to grid height When the value is less than the threshold , it is determined that the area is not passable, and obstacle avoidance processing is performed in path planning.

[0153] Obstacle avoidance path planning:

[0154] Reachability map construction: mark the grid as an obstacle to form a binary obstacle map. Wherein is the passability threshold, .

[0155] The planning algorithm uses an improved A* algorithm to search for the optimal path on the reachability map, wherein:

[0156] Cost function :

[0157] ;

[0158] Wherein, is the actual movement cost from the starting point to the current node , and is the heuristic estimate from the current node to the end point, where the Euclidean distance is used.

[0159] Terrain adaptation improvement: when calculating the movement cost , introduce the passability index as a penalty term, that is, the cost of moving from node to the current node is:

[0160] ;

[0161] Wherein, is the Euclidean distance between the two nodes, is the terrain penalty coefficient ( ).

[0162] Dynamic obstacle avoidance: real-time fusion of new perception data to update the reachability map, when new obstacles or terrain changes are detected, trigger path re-planning.

[0163] Step three: during the inspection process, the four-legged robot relies on the high-performance edge computing platform carried to perform fire smoke target detection on the real-time collected environmental images.

[0164] The application adopts a YOLOv8 lightweight target detection algorithm, improves the inference efficiency of the model on the edge device by performing structure pruning and format conversion on the original model file. During the detection process, the robot calculates the flame target confidence based on the image recognition result. When the confidence exceeds the preset threshold, it is determined as a fire point; otherwise, it continues to perform the inspection task according to the planned trajectory. This mechanism realizes the rapid identification and accurate positioning of fire, providing reliable basis for subsequent emergency response.

[0165] Step four: After the quadruped robot detects the fire point, the edge device uploads the alarm information and real-time image data to the local area network cloud platform. The cloud platform supports dual-mode intelligent scheduling of manned and unmanned modes: in the manned mode, the system synchronously pushes the alarm information and real-time image to the monitoring terminal for the duty personnel to manually review the fire in combination with environmental characteristics; after confirming the fire, the quadruped robot carrying the fire extinguishing device can be remotely controlled through the cloud platform to rush to the scene to perform the task, and the related departments can be linked to start the emergency disposal process; when the fire exceeds the disposal capacity of the system, the platform automatically triggers the fire alarm linkage function. In the unmanned mode, the cloud platform automatically generates instructions based on preset rules to control the quadruped robot carrying the fire extinguishing device to carry out fire extinguishing operation, and sends warning information to the staff through multiple channels (such as SMS and APP push), realizing the collaborative disposal of fire events by man and multiple machines.

[0166] Step five: After triggering the quadruped robot to perform the fire extinguishing task, the cloud platform plans the optimal path for the robot to go to the target point based on the global elevation map and the coordinates of the fire point.

[0167] Risk-aware multi-objective path planning: through the semantic-enhanced elevation map, the environment objects and fire risks are associated to realize the path decision of "safety-efficiency" dual-objective optimization, breaking through the limitation of traditional planning considering only geometric obstacles.

[0168] At the same time, the platform uses a deep semantic segmentation network to enhance the environmental information of the global map, and dynamically labels semantic labels such as "flammable material storage area" and "electrical equipment area" by identifying scene objects such as shelves and power distribution cabinets.

[0169] This semantic segmentation technology is based on a convolutional neural network architecture combined with an instance segmentation algorithm to realize accurate classification and labeling of objects in complex scenes, providing multi-dimensional environmental data including terrain information, obstacle distribution and risk level for the path planning algorithm. The optimized path planning result and control instruction are transmitted in real time to the main control unit of the quadruped robot through the 5G communication link, ensuring that it efficiently and safely travels to the fire point. The specific technical scheme is as follows:

[0170] 1. Semantic map construction:

[0171] The DeepLabV3+ semantic segmentation network is used to analyze the global map and identify the preset key objects (e.g., shelves, distribution boxes, etc.).

[0172] Define risk quantification rules to label risk values for each grid :

[0173] ;

[0174] 2, Cascade screening path planning

[0175] Use the improved D* Lite algorithm, and the cost function integrates three-dimensional passability and fire risk.

[0176] Step 1: Terrain passability screening

[0177] Exclude all grids that do not meet the terrain passability threshold of the quadruped robot, and generate a terrain passable area.

[0178] Step 2: Fire risk screening

[0179] In the terrain passable area, exclude all grids that do not meet the NFPA risk assessment standard, and generate a safe area. Output: Safe grid set:

[0180]

[0181] ;

[0182] The essence of this step is to construct a three-dimensional corridor that the robot can safely pass through.

[0183] Step 3: Shortest path search

[0184] In the safe area S, use the traditional A* algorithm to plan the shortest path, and the cost function is the Euclidean distance:

[0185] ;

[0186] Where, is the actual movement cost, is the heuristic function, as follows:

[0187] ;

[0188] Where, represents the position at step , is the position at step , is the target point coordinate.​​​​

[0189] 3, communication and execution

[0190] The <path point sequence, maximum speed, risk avoidance instruction> is issued to the robot through the 5G network, and the transmission delay is less than or equal to 50ms.

[0191] Step six: after the wheel-foot robot arrives at the fire point, the fire extinguishing device is started immediately, and high-pressure injection or dry powder covering and other fire extinguishing operations are performed.

[0192] In summary, the four-legged robot is used for inspection in the present application, wherein the four-legged robot has strong passability and obstacle crossing ability, can move flexibly in some complex terrain environment, and is suitable for performing an inspection task. At the same time, the wheel-foot robot carries a mechanical arm to clamp a fire extinguishing device for fire treatment. Compared with the four-legged robot, the wheel-foot robot has a faster movement speed and higher stability, and can quickly and accurately perform a fire extinguishing task. This design solves the problem that personnel cannot arrive at the fire scene in time and effectively dispose of the fire in an unmanned area, and uses a robot to replace manual fire treatment, which greatly improves the efficiency and safety of fire extinguishing operations.

[0193] The fire smoke detection system proposed in the present application is developed based on a cloud edge architecture, the end side is composed of a visual sensor carried by a four-legged robot, which is used for image data acquisition, the edge side is an edge computing device installed on the back of the robot, which is used for image detection and recognition, and the recognition result is uploaded to the cloud platform, and the cloud side is a control program platform of the upper computer, which can display the image uploaded by the visual sensor of the four-legged robot in real time, and judge and analyze the recognition result. If it is judged that there is a fire, the platform controls the wheel-foot robot to carry a fire extinguishing device for disposal. This architecture can significantly reduce the cloud computing load and effectively relieve the bandwidth pressure of data transmission

[0194] The cloud control system proposed in the present application can plan an inspection route for the four-legged robot according to a preset disposal program, and display monitoring images in real time. In addition, the system can establish remote communication with the fire department through a communication module. When the scale of the fire exceeds the ability of the robot to dispose alone, the system can directly warn the fire department to realize human intervention in fire disposal. The whole process operation is simple and easy to control.

[0195] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium is capable of storing a computer program, the computer program is executable to run some or all steps of the invention content and each embodiment of the fire detection and disposal system and method based on multi-robot cooperation provided by the present application when executed by the data processing unit. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0196] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present application can be realized by means of a computer program and its corresponding general hardware platform. Based on such understanding, the technical solutions in the embodiments of the present application can be embodied in the form of a computer program, i.e., a software product, which can be stored in a storage medium and includes a plurality of instructions for causing a device (which can be a personal computer, a server, a single-chip microcomputer, an MCU or a network device, etc.) containing a data processing unit to execute the method described in each embodiment or some parts of the embodiments of the present application.

[0197] The present application provides a thought and method of a fire detection and disposal system and method based on multi-robot cooperation. There are many methods and ways to realize the technical solutions, and the above description is only the preferred embodiment of the present application. It should be pointed out that for ordinary skilled in the art, some improvements and refinements can be made without departing from the principles of the present application, and these improvements and refinements should also be regarded as the protection scope of the present application. The components not explicitly described in the embodiments can be realized by using existing technologies.

Claims

1. A fire detection and response system based on multi-robot collaboration, characterized in that, include: Quadruped robot platforms, cloud platforms, and wheeled robot platforms; among them... The quadruped robot platform is used to detect fires and send information to the cloud platform in real time; the cloud platform receives information from the quadruped robot platform and schedules and controls the quadruped robot platform and the wheeled robot platform; the wheeled robot platform is used to receive instructions from the cloud platform to perform fire extinguishing operations. The scheduling and control of the quadruped robot platform and the wheeled robot platform includes the following steps: Step 1: Create a local elevation map centered on the quadruped robot and merge and update it into a global elevation map; Step 2: Construct a quantitative model of the quadruped robot's passability and perform inspection path planning; Step 3: Use a quadruped robot to conduct inspections along the planned inspection path and detect fire and smoke targets. Step 4: Obtain the target detection results of fire and smoke through the cloud platform and plan the fire response path; Step 5: Use the wheeled robot to handle the fire according to the planned fire handling path; Step 1, which involves creating a local elevation map centered on the quadruped robot and merging and updating it into a global elevation map, includes: Step 1-1: Perform pose estimation for the quadruped robot based on visual inertial odometry; Steps 1-2: Create a local elevation map centered on the quadruped robot and update it in real time; Steps 1-3: Merge the local elevation maps created by all quadruped robots to construct a global elevation map; The steps 1-2 described above, which involve creating and updating a local elevation map centered on the quadruped robot in real time, include: Step 1-2-1, Local Elevation Map Construction, as detailed below: The three-dimensional point cloud data of the environment is calculated from the depth image acquired by the quadruped robot, and the three-dimensional point cloud data is downsampled. Based on the quadruped robot pose obtained in step 1-1, the downsampled 3D point cloud data is converted to the global coordinate system; Project the 3D point cloud onto a 2D raster map plane of a preset resolution; for each raster cell... The position of each grid cell in the computational environment map in three-dimensional space. , means as follows: ； Among them, grid unit The coordinates on the environmental map plane are: and The corresponding terrain height value is ; Collect all point clouds that fall within the horizontal range of this grid. , means as follows: ； The height of the grid cell is calculated based on a preset statistical value of the height of all points. ; The final local map contains the horizontal position of each raster cell. and corresponding height value Local elevation map; Step 1-2-2, local elevation map update, details are as follows: For new areas observed by the quadruped robot's sensors, the grid height value is directly filled in; For recurring regions observed by the quadruped robot's sensors—that is, grids with existing height values—the height values ​​of these grids are updated using a pre-defined fusion strategy based on the new observation point cloud. ; The fusion of local elevation maps created for all quadruped robots as described in steps 1-3 includes: Step 1-3-1: Coordinate alignment. Using the pose information of each quadruped robot, all local elevation maps are converted to a unified global coordinate system. Step 1-3-2, Overlapping Area Processing: For overlapping areas of different local elevation maps, a preset fusion strategy is used to calculate the final height value of each grid cell in the overlapping area. Step 1-3-3: Fill non-overlapping areas by directly filling with local elevation map data from a single source; Steps 1-3-4: Dynamic update. The global elevation map is updated in real time based on the local elevation maps uploaded by each quadruped robot. Step 4, which involves planning the fire response route, includes: Step 4-1, semantic map construction, as follows: A semantic segmentation network is used to parse the global elevation map and identify preset key objects; Define risk quantification rules, and assign each grid cell based on the identified key objects. Mark risk value ; Step 4-2, cascading filtering path planning, as follows: Step 4-2-1, terrain accessibility filtering, i.e., excluding all accessibility values. The grid cells, Generate terrain passability zones based on the passability threshold; Step 4-2-2, Fire Risk Screening, details are as follows: Exclude all areas where the terrain allows passage. The grid is used to generate a safe area, which is a set of safe grid cells. , means as follows: ； in, Risk threshold; Step 4-2-3, shortest path search, details are as follows: In the safe area The A* method is used to plan the shortest path, and the cost function is given by... , means as follows: ； in, For actual mobility costs, A heuristic function is represented as follows: ； in, Representing the The location of the step, It is the first Step position, These are the coordinates of the target point.

2. The fire detection and response system based on multi-robot collaboration according to claim 1, characterized in that, The quadruped robot platform includes: A quadruped robot, and a lidar, inertial odometry, vision sensor, fire and smoke detection module, alarm module, and main control module installed on the quadruped robot; wherein, The quadruped robot is a vehicle with a lidar that scans upwards in a hemispherical shape mounted on its top, a vision sensor mounted on its head, and an inertial odometer for measuring motion status. The fire and smoke detection module performs multi-scale feature extraction based on the images acquired by the visual sensor to obtain the bounding box coordinates of the flame and smoke and the corresponding confidence level. The alarm module triggers an alarm based on the confidence level. The main control module generates a local map based on information obtained from lidar, inertial odometry, and visual sensors, interacts with the cloud platform, and controls the overall operation of the quadruped robot platform.

3. A fire detection and response system based on multi-robot collaboration according to claim 2, characterized in that, The cloud platform includes: Based on the local maps generated by all quadruped robot platforms, a global map is generated, and inspection path planning is performed for collaborative control of the quadruped robot platforms. Based on the fire information obtained from all quadruped robot platforms, a disposal path is planned to control the wheeled robot platforms to handle the fire.

4. A fire detection and response system based on multi-robot collaboration according to claim 3, characterized in that, The wheeled robot platform includes: A wheeled robot and a fire extinguishing device mounted on the wheeled robot are used to execute the scheduling commands of the cloud platform to handle fire situations.

5. A method for fire detection and response based on multi-robot collaboration, characterized in that, Using the system described in any one of claims 1-4 for fire detection and response includes the following steps: Step 2, which involves constructing a quadruped robot passability quantification model and planning inspection paths, includes: Step 2-1: Calculate the passability values ​​for the raster cells in the global elevation map, as follows: ； Among them, grid unit Passability value , For the current grid cell height, For the current grid cell The slope, For the current grid cell roughness, , and It depends on the preset threshold of the quadruped robot structure. , and For control coefficients; Slope value The calculation method is as follows: ； in, and Representing grid cells Physical dimensions in the x and y directions; roughness The calculation method is as follows: ； in, To fall into the grid cell The number of point clouds, For the first point cloud The height of each point Average height; Step 2-2: Plan the inspection path based on the passability value. When the passability value of a grid cell is less than the threshold, the area is determined to be impassable, and obstacle avoidance is performed in the inspection path planning, as follows: Step 2-2-1, Accessibility Map Construction, will The grid cells are marked as obstacles, forming a binary obstacle map; Step 2-2-2: The inspection path planning uses the improved A* method to search for the optimal path on the accessibility map. Let the cost function of the inspection path be... , means as follows: ； in, From the starting point to the current node The actual cost of movement, This is a heuristic estimate of the distance from the current node to the destination. Step 2-2-3, Terrain Adaptation Improvement, involves calculating the actual movement cost. At that time, a passability indicator was introduced. As a penalty, that is, from the previous node Move to the current node The cost is: ； in, It is the Euclidean distance between the two nodes. This is the terrain penalty coefficient; Step 2-2-4, Dynamic obstacle avoidance: When a new obstacle or terrain change is detected, path replanning is triggered.

Images