Special robot cooperative auxiliary rescue control method for building fire fighting

Abstract

The application discloses a special robot cooperative auxiliary rescue control method for building fire fighting. In the real fire source determination process, the application introduces the features of the flame centroid displacement and the flame roughness, so that the problem that the mobile flame cannot be accurately recognized in the indoor environment is solved. In addition, a multi-sensor fusion recognition algorithm is adopted, so that the recognition misjudgment rate is greatly reduced. When the building fire fighting robots are dispatched, the number of the building fire fighting robots K is used to adjust the edge bar items of the fire source convex hull, so that more convenient and reasonable robot spraying station positions are realized. The application provides a complete, safe and fast emergency fire extinguishing scheme from discovering a suspected fire source, judging the authenticity and category of the fire source and dispatching the building fire fighting robots to extinguish the fire.

Description

Technical Field

[0001] This invention belongs to the field of mobile robot technology and relates to a special robot collaborative assisted rescue control method for building fire fighting. Background Technology

[0002] Different buildings have different functions, so specific analysis is needed for efficient fire fighting. In order to improve fire fighting efficiency, some buildings use mobile robots for patrol. Mobile robots can detect flames and track them in real time during patrols using flame detection and tracking technology, which is also an important technology for robots to achieve fire fighting.

[0003] Traditional fire detection technologies typically employ sensors such as light, temperature, smoke, and gas sensors to capture early warning signs of fires and determine whether a fire has occurred. However, these devices are affected by physical factors such as detection distance and sensor installation location, as well as interference from the natural environment, resulting in low prediction accuracy and limited effectiveness in complex environments. Using only video images for flame detection also has limitations; when the fire source is obscured by other objects, it becomes difficult to detect. Therefore, real-time flame tracking through multi-sensor fusion within the mission space is of great significance. This invention proposes a special robot-assisted rescue control method for building fire fighting. Summary of the Invention

[0004] The first objective of this invention is to address the shortcomings of existing technologies by providing a special robot collaborative assisted rescue control method for building fire fighting, comprising a special robot and a trolley that moves the robot. The special robot collaborative assisted rescue method includes the following steps:

[0005] Step (1): Equip each building with multiple building fire-fighting robots and smoke sensors. The building fire-fighting robots will patrol regularly. Each building will construct a composite semantic-grid map containing the semantic information of the current building site through the lidar and depth camera A of the special robot.

[0006] Step (2): Use a quadcopter drone to search for suspected fire sources, determine the type of combustible material of the actual fire source, and obtain the coordinates of the actual fire source.

[0007] Step (3): Assess the fire intensity and send the fire semantic information marked on the composite semantic-raster map to the building fire robot; the building fire robot goes to the fire point;

[0008] Step (4): Obtain the potential expansion area of ​​the fire source and construct an evaluation convex hull function to evaluate the potential expansion area of ​​the fire source;

[0009] Step (5): The quadcopter drone controller uses particle filtering to predict the temperature of surrounding objects for three cycles. If the object will catch fire after one cycle, the building fire robot closest to the object will be dispatched to spray water to cool it down for one cycle.

[0010] Step (6): Adjust the distribution of all building fire-fighting robots to be executed near the fire source;

[0011] Step (7): Adjust the spray direction of all building fire-fighting robots to be executed;

[0012] Step (8): When the building fire robot A starts using the equipment to extinguish the fire, update the shape of the convex hull in real time, repeat step (4), and start timing the building fire robot A; if the building fire robot A extinguishes the fire at a certain target point for more than t0, the adjacent building fire robot B and building fire robot C approach the building fire robot A to help the building fire robot A extinguish the fire until the target point of the spray station of the building fire robot A changes, and the building fire robot A restarts the timing; the building fire robot B and building fire robot C return to their original positions.

[0013] The beneficial effects of this invention are:

[0014] This invention incorporates the characteristics of flame centroid displacement and flame roughness into the real fire source identification process, thereby solving the problem of inaccurate identification of moving flames in indoor environments. Furthermore, it employs a multi-sensor fusion identification algorithm, significantly reducing the false positive rate.

[0015] When scheduling building fire-fighting robots, this invention adjusts the side entries of the fire source convex hull according to the number K of building fire-fighting robots to achieve a more convenient and reasonable robot spraying station position.

[0016] This invention proposes a complete, safe, and rapid emergency firefighting solution that involves detecting a suspected fire source, determining the authenticity and type of the fire source, and dispatching building firefighting robots to extinguish the fire. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0018] Figure 1 A multi-sensor fusion flame detection architecture;

[0019] Figure 2 This describes the overall process of the solution. Detailed Implementation

[0020] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0021] Example: Figures 1 to 2 As shown, the present invention provides a special robot collaborative assisted rescue control method for building fire fighting. The method is based on configuring multiple mobile building fire fighting robots. Each building fire fighting robot is equipped with a quadcopter drone on its back and is equipped with at least a lidar and a depth camera A. The quadcopter drone is equipped with a depth camera B and an infrared detector.

[0022] The mobile building firefighting robot in this embodiment is a quadruped tracked robot.

[0023] The present invention provides a special robot collaborative assisted rescue control method for building fire fighting, comprising the following steps:

[0024] Step (1): Equip each building with multiple building fire-fighting robots and smoke sensors. The building fire-fighting robots will patrol regularly. Each building will construct a composite semantic-grid map containing the semantic information of the current building site through the lidar and depth camera A of the special robot. The construction of the composite semantic-grid map is a conventional technology and will not be explained in detail.

[0025] The site semantic information includes passable areas, passage hazards, entrances and exits, and obstacles.

[0026] Step (2): Use a quadcopter drone to search for suspected fire sources, determine the type of combustible material in the actual fire source, and obtain the coordinates of the actual fire source. Specifically:

[0027] 2-1 When the smoke sensor on a certain floor of the building issues an alarm, the control center issues an order to dispatch a quadcopter drone;

[0028] 2-2 A quadcopter drone flew to the floor where the alarm was triggered, patrolled each room to search for suspected fire sources, and determined whether it was a real fire; specifically:

[0029] 2-2-1 When the quadcopter drone hovers, depth camera B acquires infrared and visible light video streams of the suspected fire source area. The presence of a suspected fire source area is determined using background subtraction. If no fire source area is found, the process ends; otherwise, step 2-2-2 is executed. The background subtraction method described is a conventional existing technology and will not be explained in detail.

[0030] 2-2-2 Determining the actual fire source from the infrared video stream of the suspected fire source area.

[0031] 2-2-2-1 The Otsu threshold segmentation foreground extraction algorithm is used to segment the suspected fire source region. The Otsu threshold segmentation foreground extraction algorithm is a conventional existing technology and will not be described in detail.

[0032] 2-2-2-2 For suspected fire source areas, features such as fire source roughness, centroid displacement, circularity, rectangularity, eccentricity, area change rate, and Hu moment are extracted. Specifically:

[0033] (1) Flame roughness FR is defined as:

[0034]

[0035] Where L is the perimeter of the suspected fire source area; L TH Let be the perimeter of the convex hull of the suspected fire source area. This convex hull represents the polygon that encompasses the area of ​​the suspected fire source and has the smallest possible area.

[0036] (2) Extraction of the centroid of the fire source: The brightness of the suspected fire source area gradually decreases from the center to the edge. Based on this feature, the centroid can be determined. If the fire source image is I(x,y), then the center (x0,y0) of the suspected fire source area is:

[0037]

[0038]

[0039] Based on the images I1(x,y) and I2(x,y) of two adjacent frames, the fire source center (x,y) is obtained according to formula (2). 0,1 y 0,1 ) and (x 0,2 y 0,2 Furthermore, according to formula (3), the displacement D of the centroid is obtained as follows:

[0040]

[0041] (3) The formula for calculating the roundness C is as follows:

[0042]

[0043] Where A is the area of ​​the suspected fire source and L is the perimeter of the suspected fire source.

[0044] (4) The formula for calculating the rectangularity R is as follows:

[0045]

[0046] Where S R Let A be the area of ​​the smallest bounding rectangle of the suspected fire source region.

[0047] (5) The formula for calculating the eccentricity T is as follows:

[0048]

[0049] Where W represents the width of the suspected fire source area and H represents the height of the suspected fire source area.

[0050] (6) The formula for calculating the area change rate α is as follows:

[0051]

[0052] Where A i+1 and A i This represents the area of ​​the suspected fire source region in the two consecutive frames.

[0053] (7) The (p+q)th order invariant moment m of Hu's moment p,q (p+q)th order central moment μ p,q and normalized central moments η p,q The calculation formula is as follows:

[0054]

[0055]

[0056]

[0057]

[0058] Where M and N are the image sizes along the X and Y axes within the digital image I(x,y). The center of the radius; m 10 Denotes the first-order invariant moment, m 00 It represents the zeroth order invariant moment.

[0059] Hu's moment is η p,q The linear combination of Hu moments is defined as follows:

[0060] Hu1=η 20 +η 02 Formula (12)

[0061] Where η 20 η 02 Both represent the second-order normalized central moments.

[0062] 2-2-2-3 Extract all features from the images in the infrared video stream, normalize all features, and then input them into the infrared video fire identification classifier SVM to determine whether it is a real fire. If the identification result is a real fire, output 1; otherwise, output 0.

[0063] 2-2-3 Determine the actual fire source from the visible light video stream of the suspected fire source area.

[0064] 2-2-3-1 The SSD_MobileNetV3 model is used to identify the suspected fire source in the visible light video stream of the suspected fire source area and determine whether it is a real fire. If the identification result is a real fire, output 1, otherwise output 0.

[0065] 2-2-4 The fusion flame recognition algorithm is used to fuse the recognition results of the infrared video fire recognition classifier SVM and the SSD_MobileNetV3 model. If both recognition results are 0, it is determined to be a non-fire. If both recognition results are 1, it is determined to be a real fire, and then step 2-3 is performed. If only one of them is 1, it is further determined whether the temperature of the image in the infrared video stream has increased. If it has increased, it is a real fire, and step 2-3 is performed. Otherwise, it is a non-fire.

[0066] 2-3 The infrared detector of the quadcopter drone detects the flame flicker frequency of the real flame. Based on the characteristics of the flame flicker frequency and the objects around the flame, the type of combustible material is identified. If it is an electrical fire, the control center issues an instruction to the building fire robot to use dry powder to extinguish the fire. If it is a wood fire, the control center issues an instruction to the building fire robot to use water to extinguish the fire.

[0067] The specific steps for identifying and determining the type of combustible material based on the characteristics of flame flicker frequency and objects surrounding the flame are:

[0068] The flame flashing spectrum is obtained by an infrared detector on a quadcopter drone. It is then compared with the labeled flame flashing spectrum stored in the quadcopter drone database to find the k closest flame flashing spectra, k≥1, and thus preliminarily determine the type of combustible material. Then, the YOLOv3 target recognition algorithm is used to identify objects around the flame, and the final combustible material category is obtained from the k categories based on the category of the surrounding objects.

[0069] 2-4 Calculate the location of the current fire source and semantically label the current fire in the composite semantic-raster map, specifically:

[0070] 2-4-1 Calculate the pose of the quadcopter UAV in the world coordinate system as T wr Where x1, y1, z1 are the Cartesian coordinates of the quadcopter UAV in the world coordinate system, X', Y', Z' represent the Cartesian coordinate system in which the quadcopter UAV is located, X, Y, Z represent the Cartesian coordinate system in the world coordinate system, and cos∠X'X is the angle between the X-axis of the world coordinate system and the X' axis of the quadcopter UAV coordinate system.

[0071]

[0072] 2-4-2 Calculate the fire source pose matrix P in the coordinate system of the quadcopter UAV. rh x2, y2, z2 are the Cartesian coordinates of the fire source in the quadcopter drone coordinate system, X”, Y”, Z” represent the Cartesian coordinate system in which the fire source is located, and cos∠X”X’ is the angle between the X” axis in the fire source coordinate system and the X’ axis in the quadcopter drone coordinate system.

[0073]

[0074] 2-4-3 Calculate the pose matrix of the fire source in the world coordinate system as P wh :

[0075] P wh =T wr *P rh Formula (15)

[0076] Furthermore, according to P wh The Cartesian coordinates of the fire source in the world coordinate system are x3, y3, z3.

[0077] 2-4-4 Based on (x3, y3, z3), fire semantic information is marked on the composite semantic-raster map.

[0078] Step (3): Assess the fire intensity and send the fire semantic information marked on the composite semantic-raster map to the building fire-fighting robot; the building fire-fighting robot then goes to the fire location; specifically:

[0079] 3-1 The quadcopter UAV controller rates the fire intensity of the fire source according to formula (17), and judges the spread of the fire by analyzing the fire intensity, the damage around the scene and the detection of surrounding objects, and sends the fire type and the spread of the fire to the background command center.

[0080] P = A * H * D0 Formula (16)

[0081] Where P is the fire intensity rating parameter, A is the fire area, H is the fire height, and D0 is the damage coefficient of the surrounding environment.

[0082] The method of analyzing the fire, assessing the damage to the surrounding area, and detecting nearby objects to determine the spread of the fire is a conventional technique and will not be explained in detail.

[0083] 3-2 The back-end command center calculates the equipment information required for firefighting and the number K of building firefighting robots to be deployed, based on the type of combustible materials and the spread of the fire, where 1 ≤ K ≤ K. max K max This indicates the total number of building firefighting robots in the current building; then, the information on the equipment needed for firefighting is sent to the building firefighting robots in the building where the fire is currently taking place.

[0084] If P < 5m 3 is a first-level fire, then dispatch one building fire-fighting robot; if 5m 3 < P < 10m 3 is a second-level fire, two building fire-fighting robots are needed to extinguish the fire; if 10m 3 < P is a third-level fire, more building fire-fighting robots need to be dispatched.

[0085] 3-3 The background command center updates the expansion area of the fire source in real time according to the category of combustibles in the fire and the spread situation of the fire; the calculation of the expansion area of the fire source is a conventional technology, so it will not be elaborated.

[0086] Set a pose information Q (Q can be 2m) outside the flame expansion area as the initial navigation point of the building fire-fighting robot, and display the initial navigation point on the composite semantic-grid map.

[0087] 3-4 The composite semantic-grid map updates the spatial positions of all building fire-fighting robots and quadrotor UAVs in real time, and plans the optimal path for each executing building fire-fighting robot to reach the initial navigation point of the fire source through the A*(A-Star) algorithm. Sort the optimal path lengths of each executing building fire-fighting robot from low to high, and select the top K building fire-fighting robots. Then assign priorities to these K building fire-fighting robots. The shorter the Euclidean distance from the building fire-fighting robot to the fire source, the higher the priority.

[0088] 3-5 When multiple building fire-fighting robots are about to collide when moving to the target point and their paths overlap, adopt a collision avoidance strategy of priority arbitration:

[0089] When a collision occurs, the building fire-fighting robot with a higher priority continues to move forward, and the building fire-fighting robot with a lower priority selects an avoidance method according to the behavior costs of the two strategies of waiting in place and re-planning a new path.

[0090] The behavior cost is the sum of the path increase amount and the time increase amount caused by the collision avoidance behavior. The path increase amount is the difference between the length of the originally planned path and the length of the re-planned path. The time increase amount is the time required for the waiting strategy or the time to re-plan a new path.

[0091] 3-4 When the building fire-fighting robot encounters an obstacle when navigating to the initial navigation point, the lidar obtains point cloud information, fuses the odometer information, combines with the composite semantic-grid map, and constructs a new local map through coordinate transformation fusion to re-plan the local path of the building fire-fighting robot.

[0092] Step (4): Obtain the possible expansion area of the fire source, and construct an evaluation convex hull function to evaluate the possible expansion area of the fire source;

[0093] 4-1 The temperature information of the fire source is obtained from infrared images acquired by a quadcopter drone. Based on the temperature threshold f, a number of pixels in the infrared image are obtained as a point set. The Graham algorithm is used to create a convex hull of the potential expansion region of the fire source using these point sets. The shape of the convex hull is updated in real time as the firefighting continues.

[0094] 4-2 If there are multiple convex hulls within the field, an evaluation convex hull function V is constructed as the evaluation standard for the order of rescues. The higher V is, the higher the priority of the convex hull.

[0095] For the evaluation of the convex hull function, see formula (17):

[0096] V = A hull ∑ i:1→n w i ×L i Formula (17)

[0097] Among them, w i A represents the weight of the i-th influencing factor. hull L represents the area of ​​the convex hull. i This represents the value of the i-th influencing factor, which includes the fire level, other influencing factors on surrounding buildings, such as the number of surrounding combustibles, the rate of fire spread, and the extent of damage to buildings.

[0098] Step (5): The quadcopter drone controller uses particle filtering to predict the temperature of surrounding objects over three cycles. If the object will catch fire after one cycle, the building fire robot closest to the object will be dispatched to spray water to cool it down for one cycle.

[0099] The particle filtering method described is a conventional technique, so it will not be explained in detail.

[0100] Step (6): Adjust the distribution of all pending building firefighting robots near the fire source.

[0101] If the number of edges F of the convex hull obtained in step 4-1 is equal to the number of building fire-fighting robots K in the current building, then the K building fire-fighting robots will be distributed at the center of each edge of the convex hull as the spray station target points of the building fire-fighting robots.

[0102] If the number of edges F of the convex hull obtained in step 4-1 is less than the number of building fire-fighting robots K in the current building, then select F from the K building fire-fighting robots and distribute them at the center of each edge of the convex hull as the spray station target points of the building fire-fighting robots.

[0103] If the number of edges F of the convex hull obtained in step 4-1 is greater than the number K of the building fire-fighting robots in the current building, then determine whether there are at least two intersecting edges of the convex hull. If there are, select two intersecting edges of the convex hull and extend them to form a new convex hull. Repeat the above operation until the number of edges of the new convex hull is equal to K. Then distribute K building fire-fighting robots at the center of each edge of the new convex hull as the spray station target points of the building fire-fighting robots. If there are no intersecting edges, construct the minimum circumcircle and distribute K building fire-fighting robots at equal intervals on the edges of the minimum circumcircle.

[0104] Step (7): Adjust the spray direction of all building fire robots to be executed.

[0105] The Cartesian coordinates of the current fire source calculated in steps 2-4 are used to adjust the spray direction in combination with the distribution of the building fire-fighting robot near the fire source.

[0106] The distance to the fire source is determined by the Y-axis. When the center of the fire is above the image, the fire source moves forward; when it is below, it moves backward. The X-axis determines the directional deviation of the fire source. A PID algorithm controls the fire source's movement, ensuring the fire-fighting robot's nozzles are aimed at the fire source to extinguish it. After extinguishing the fire, each fire-fighting robot returns to its original position.

[0107] Step (8): When the building fire-fighting robot A starts using its equipment to extinguish the fire, update the shape of the convex hull in real time, repeat step (4), and start timing the building fire-fighting robot A. If the building fire-fighting robot takes more than t0 to extinguish the fire at a certain target point, the adjacent building fire-fighting robots B and C move closer to the building fire-fighting robot A to help it extinguish the fire until the target point of the spray station of the building fire-fighting robot A changes, and the building fire-fighting robot A restarts its timing. The building fire-fighting robots B and C return to their original positions.

[0108] As a preferred option, when the building fire-fighting robot arrives near the fire source and detects people nearby, it displays the fastest and safest escape route to the exit on the building fire-fighting robot's screen and provides voice prompts to assist people in escaping, thus shortening the escape time.

[0109] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A special robot collaborative assisted rescue control method for building fire fighting, the method being based on configuring multiple mobile building fire fighting robots, each building fire fighting robot carrying a quadcopter drone on its back, and equipped with at least a lidar and a depth camera A; the quadcopter drone is equipped with a depth camera B and an infrared detector; characterized in that... The method includes the following steps: Step (1): Equip each building with multiple building fire-fighting robots and smoke sensors. The building fire-fighting robots will patrol regularly. Each building will construct a composite semantic-grid map containing the semantic information of the current building site through the lidar and depth camera A of the special robot. Step (2): Use a quadcopter drone to search for suspected fire sources, determine the type of combustible material in the actual fire source, and obtain the coordinates of the actual fire source. Specifically: 2-1 When the smoke sensor on a certain floor of the building issues an alarm, the control center issues an order to dispatch a quadcopter drone; 2-2 A quadcopter drone flew to the floor where the alarm was triggered, patrolled each room to search for suspected fire sources, and determined whether it was a real fire; specifically: 2-2-1 When the quadcopter drone is hovering, the depth camera B acquires the infrared video stream and visible light video stream of the suspected fire source area. The background subtraction method is used to determine whether there is a suspected fire source area. If not, the process ends; otherwise, step 2-2-2 is executed. 2-2-2 Perform actual fire source determination on the infrared video stream of suspected fire source areas; specifically: 2-2-2-1 The Otsu threshold segmentation foreground extraction algorithm is used to segment the suspected fire source region; 2-2-2-2 For suspected fire source areas, features such as fire source roughness, centroid displacement, circularity, rectangularity, eccentricity, area change rate, and Hu moment are extracted. Specifically: (1) Flame roughness FR is defined as: Official (1) in The perimeter of the suspected fire source area; Let be the perimeter of the convex hull of the suspected fire source area. This convex hull represents the polygon with the smallest area that encompasses the suspected fire source area. (2) Extraction of the centroid of the fire source: The brightness of the suspected fire source area gradually decreases from the center to the edge. Based on this feature, the centroid can be determined. If the fire source image is... So, the center of the suspected fire source area ( , )for: Official (2) Based on the images of two adjacent frames , According to formula (2), the center of the fire source is obtained. and The displacement of the centroid is further obtained according to formula (3). for: Official (3) (3) Circularity The calculation formula is as follows: Official (4) in The area of ​​the suspected fire source. The perimeter of the suspected fire source; (4) Rectangularity The calculation formula is as follows: Official (5) in Let the area be the smallest bounding rectangle of the suspected fire source area. The area of ​​the suspected fire source; (5) Eccentricity The calculation formula is as follows: Official (6) Where W is the width of the suspected fire source area and H is the height of the suspected fire source area; (6) Area change rate The calculation formula is as follows: Official (7) in and The area of ​​the suspected fire source region in the two consecutive frames; (7) Hu's moment Invariant Moments , order central moments and normalized central moments The calculation formula is as follows: Official (8) Official (9) , Official (10) Official (11) Where M and N are digital images Image size along the inner X and Y axes Center of the square; Indicates the first-order invariant moment. Hu is The linear combination of Hu moments is defined as follows: Official (12) in Both represent the second-order normalized central moments; 2-2-2-3 Extract all features from the images in the infrared video stream, normalize all features, and then input them into the infrared video fire identification classifier SVM to determine whether it is a real fire. If the identification result is a real fire, output 1; otherwise, output 0. 2-2-3 Determine the real fire source in the visible light video stream of the suspected fire source area; specifically: use the SSD_MobileNetV3 model to identify the suspected fire source in the visible light video stream of the suspected fire source area and determine whether it is a real fire. If the identification result is a real fire, output 1, otherwise output 0. 2-2-4 The fusion flame recognition algorithm is used to fuse the recognition results of steps 2-2-2 to 2-2-3. If both recognition results are 0, it is determined to be a non-fire. If both recognition results are 1, it is determined to be a real fire, and step 2-3 is performed. If only one of them is 1, it is further determined whether the temperature of the image in the infrared video stream has increased. If it has increased, it is a real fire, and step 2-3 is performed. Otherwise, it is a non-fire. 2-3 The infrared detector of the quadcopter drone detects the flame flicker frequency of the real flame. Based on the characteristics of the flame flicker frequency and the objects around the flame, the type of combustible material is identified. If it is an electrical fire, the control center issues an instruction to the building fire robot to use dry powder to extinguish the fire. If it is a wood fire, the control center issues an instruction to the building fire robot to use water to extinguish the fire. 2-4 Calculate the location of the current fire source and semantically label the current fire in the composite semantic-raster map; Step (3): Assess the fire intensity and send the fire semantic information marked on the composite semantic-raster map to the building fire robot; the building fire robot then goes to the fire location. Step (4): Obtain the potential expansion area of ​​the fire source and construct an evaluation convex hull function to evaluate the potential expansion area of ​​the fire source; Step (5): The quadcopter drone controller uses particle filtering to predict the temperature of surrounding objects for three cycles. If the object will catch fire after one cycle, the building fire robot closest to the object will be dispatched to spray water to cool it down for one cycle. Step (6): Adjust the distribution of all building firefighting robots to be executed near the fire source; specifically: If the number of edges F of the convex hull obtained in step 4-1 is equal to the number of building fire-fighting robots K in the current building, then the K building fire-fighting robots will be distributed at the center of each edge of the convex hull as the spray station target points of the building fire-fighting robots. If the number of edges F of the convex hull obtained in step 4-1 is less than the number of building fire-fighting robots K in the current building, then select F from the K building fire-fighting robots and distribute them at the center of each edge of the convex hull as the spray station target points of the building fire-fighting robots. If the number of edges F of the convex hull obtained in step 4-1 is greater than the number K of the building fire-fighting robots in the current building, then determine whether there are at least two intersecting edges of the convex hull. If there are, select two intersecting edges of the convex hull and extend them to form a new convex hull. Repeat the above operation until the number of edges of the new convex hull is equal to K. Then distribute K building fire-fighting robots at the center of each edge of the new convex hull as the spray station target points of the building fire-fighting robots. If there are no intersecting edges, construct the minimum circumcircle and distribute K building fire-fighting robots at equal intervals on the edges of the minimum circumcircle. Step (7): Adjust the spray direction of all building fire-fighting robots to be executed and extinguish the fire.

2. The method according to claim 1, characterized in that... The site semantic information mentioned in step (1) includes the passable area of ​​the site, the danger of the passage, the entrance and exit, and obstacles.

3. The method according to claim 1, characterized in that... Steps 2-3, which describe identifying the type of combustible material based on the characteristics of flame flicker frequency and objects surrounding the flame, specifically involve: The flame flashing spectrum is obtained by an infrared detector on a quadcopter drone. It is then compared with the labeled flame flashing spectrum stored in the quadcopter drone database to find the k closest flame flashing spectra, k≥1, and thus preliminarily determine the type of combustible material. Then, the YOLOv3 target recognition algorithm is used to identify objects around the flame, and the final combustible material category is obtained from the k categories based on the category of the surrounding objects.

4. The method according to claim 1, characterized in that... Steps 2-4 are as follows: 2-4-1 Calculate the pose of the quadcopter UAV in the world coordinate system as follows ,in , , The coordinates of the quadcopter UAV in the world coordinate system are Cartesian coordinates. , , This represents the Cartesian coordinate system in which the quadcopter drone is located. , , This represents the Cartesian coordinate system in the world coordinate system. For the world coordinate system Axis and quadcopter UAV coordinate system The included angle of the axis; Official (13) 2-4-2 Calculate the fire source pose matrix in the coordinate system of the quadcopter UAV as follows: , , , Let the Cartesian coordinates of the fire source be in the coordinate system of the quadcopter UAV. , , The Cartesian coordinate system representing the location of the fire source. In the fire source coordinate system Axis and quadcopter UAV coordinate system The included angle of the axis; Official (14) 2-4-3 Calculate the pose matrix of the fire source in the world coordinate system as follows: : Official (15) Furthermore, based on Obtain the Cartesian coordinates of the fire source in the world coordinate system. , , ; 2-4-4 According to ( , , Fire semantic information is marked under the composite semantic-raster map.

5. The method according to claim 1, characterized in that... Step (3) specifically involves: 3-1 The quadcopter UAV controller rates the fire intensity of the fire source according to formula (17), and judges the spread of the fire by analyzing the fire intensity, the damage around the scene and the detection of surrounding objects, and sends the fire information to the background command center. The fire type and the spread of the fire are also sent. Official (16) in For fire intensity rating parameters, The height of the fire The coefficient of damage to the surrounding environment; 3-2 The back-end command center calculates the equipment information required for firefighting and the number K of building firefighting robots to be deployed, based on the type of combustible materials and the spread of the fire, where 1 ≤ K ≤ K. max K max This indicates the total number of building firefighting robots in the current building; then, it sends the equipment information required for firefighting to the building firefighting robots in the building where the fire is currently taking place. 3-3 The back-end command center updates the expansion area of ​​the fire source in real time based on the type of combustibles and the spread of the fire. Set a pose information of Q outside the flame expansion zone as the initial navigation point of the building fire robot, and display the initial navigation point on the composite semantic-grid map; 3-4 Composite Semantic-Grid Map: Real-time updates of the spatial positions of all building firefighting robots and quadcopter drones. The A*(A-Star) algorithm is used to plan the optimal path for each building firefighting robot to reach the initial navigation point of the fire source. The optimal path lengths of each building firefighting robot are sorted from low to high, and the top K building firefighting robots are selected. These K building firefighting robots are then assigned priorities, with higher priority given to robots with shorter Euclidean distances from the fire source. 3-5 If multiple building firefighting robots are about to collide while moving towards the target point, a collision avoidance strategy based on priority arbitration will be adopted: When a collision occurs, the high-priority building firefighting robot continues to move forward, while the low-priority building firefighting robot chooses an avoidance method based on the behavioral costs of either waiting in place or replanning a new path. The cost of the action is the sum of the increase in path length and the increase in time caused by the collision avoidance action. The increase in path length is the difference between the originally planned path length and the replanned path length; the increase in time is the time required to wait for the strategy or to replan a new path. 3-6 When the building fire-fighting robot encounters an obstacle while navigating to the initial navigation point, the lidar acquires point cloud information, fuses it with odometry information, combines it with a composite semantic-grid map, and constructs a new local map through coordinate transformation and fusion, thereby replanning the local path of the building fire-fighting robot.

6. The method according to claim 1, characterized in that... Step (4) specifically involves: 4-1 The temperature information of the fire source is obtained from infrared images collected by a quadcopter drone, and then a temperature threshold is established. A set of pixels is obtained from an infrared image. The Graham algorithm is used to create a convex hull for the potential expansion region of the fire source. The shape of the convex hull is updated in real time as the firefighting process continues. 4-2 If multiple convex hulls exist within the site, construct an evaluation convex hull function. As a criterion for evaluating the order of firefighting, A higher value indicates a higher priority for the convex hull. For the evaluation of the convex hull function, see formula (17): Official (17) in Representing the The weight of each influencing factor This represents the area of ​​the convex hull. Representing the The values ​​of each influencing factor.

7. The method according to claim 1, characterized in that... When the building fire-fighting robot A begins to use its equipment to extinguish the fire, the shape of the convex hull is updated in real time, step (4) is repeated, and the building fire-fighting robot A is timed; if the building fire-fighting robot A takes longer than a certain amount of time to extinguish the fire at a certain target point... Then, the adjacent building fire-fighting robots B and C approach building fire-fighting robot A and help it extinguish the fire until the target point of building fire-fighting robot A's spray station changes, at which point building fire-fighting robot A restarts its timer; building fire-fighting robots B and C return to their original positions.

Images