A multi-UAV collaboration method based on multi-sensor fusion SLAM for firefighting

Abstract

The invention discloses a multi-UAV cooperation method of multi-sensor fusion SLAM for fire fighting, which is based on a multi-UAV system, and the cooperation method includes: step S1, multi-sensor registration for each UAV; step S2, each UAV obtains environmental information through airborne sensors, and generates a three-dimensional point cloud map of the search environment in real time, and feeds back the position information of each UAV to the base station; step S3, each UAV respectively uses multi-strip cooperative search and multi-angle cooperative search methods to search for flame positions, wherein one UAV obtains the flame position through a fusion algorithm, and based on the three-dimensional point cloud map of the search environment established by it, the three-dimensional coordinates of the flame in the global coordinate system are obtained through coordinate calculation; step S4, the three-dimensional coordinates of the flame are transmitted back to the server, and the server controls multiple UAVs nearby to perform cooperative fire extinguishing operations through data fed back by the base station. It realizes autonomous perception of UAVs in fire environments and completes fire extinguishing tasks.

Description

Technical Field

[0001] The invention belongs to the technical field of unmanned aerial vehicles, and in particular relates to a multi-unmanned aerial vehicle coordination method for multi-sensor fusion SLAM for fire fighting. Background Art

[0002] With the rapid development of drone technology, the current research on drone collaborative technology continues to deepen. Nowadays, multi-drone collaborative tasks have been applied in various fields. Drones can perform various complex maneuvers and reach places that humans cannot reach, thereby completing various relatively "dangerous" tasks. In recent years, there have been reports of fires from time to time. Whether it is a wild or urban fire, drones have a certain place to use. At the same time, due to the large scope of the fire and insufficient personnel, multi-drone autonomous collaborative firefighting has become a very valuable research topic.

[0003] In addition, the current vision and radar SLAM technology is developing rapidly, and multi-sensor fusion technology has also developed to a certain extent. These technologies are of great help to the autonomous positioning of drones, and also greatly improve the ability of drones to perform tasks in unknown environments. The autonomous fire extinguishing system of multiple drones is of great help to the fire situation awareness and fire extinguishing task execution in large-scale complex environments, which can greatly improve the speed and quality of fire extinguishing task completion. Summary of the invention

[0004] The purpose of the present invention is to provide a multi-UAV collaboration method for multi-sensor fusion SLAM for firefighting, so as to realize autonomous perception of UAVs in fire environment and complete firefighting tasks.

[0005] The present invention adopts the following technical solution: a multi-UAV coordination method for multi-sensor fusion SLAM for firefighting,

[0006] It is based on a multi-UAV system, including a base station, which connects multiple UAVs with data; each UAV is equipped with airborne sensors, including binocular cameras, laser radars and flame detectors;

[0007] The multi-UAV collaboration method includes the following:

[0008] Step S1, performing multi-sensor registration on each UAV;

[0009] Step S2: Each drone obtains environmental information through an onboard sensor, generates a three-dimensional point cloud map of the search environment in real time, and feeds back the location information of each drone to the base station;

[0010] Step S3, each UAV uses multi-strip collaborative search and multi-angle collaborative search methods to search for the flame position, wherein one UAV obtains the flame position through a fusion algorithm, and obtains the three-dimensional coordinates of the flame in the global coordinate system through coordinate calculation based on the three-dimensional point cloud map of the search environment established by the UAV;

[0011] Step S4: The three-dimensional coordinates of the flame are transmitted back to the server, and the server controls multiple nearby drones to perform coordinated fire extinguishing operations based on the data fed back by the base station.

[0012] Furthermore, in step S1, multi-sensor registration includes time registration and space registration; time registration is to convert the observation information collected by each sensor from asynchronous time to the same time; the method of spatial registration is to convert the data observed by each sensor in different coordinate systems on the same platform and in different coordinate systems on different platforms to the same platform and the same coordinate system; the coordinate system includes radar coordinate system, camera coordinate system, image pixel coordinate system and infrared sensor coordinate system.

[0013] Furthermore, step S1 also includes joint calibration of the laser radar and the binocular camera, and the specific method is as follows:

[0014] The 2D point coordinates of the binocular camera are obtained by calculating the vertex coordinates of the calibration plate, and the 3D point coordinates of the lidar are obtained by fitting the boundary line of the calibration plate. Finally, the PNP problem is solved to obtain the corresponding coordinate transfer matrix, as shown in the following formula:

[0015]

[0016] Among them, c x ,c y is the translation parameter; f x =αf,f y =βf is the normalization factor, which indicates the scaling of coordinates. The unit is pixel. u,v are the normalized plane homogeneous coordinate positions of the points. x,y,z are the homogeneous coordinate positions of the points. r 11 To r 33 are the parameters of the rotation matrix, t1, t2, t3 refer to the translation amounts in the xyz axes, and γ is the camera intrinsic parameter;

[0017] The rotation and translation matrix between corresponding points is obtained by ICP method;

[0018] Assuming that two sets of point cloud pairs have been obtained, the objective function to be minimized is as follows, and finally the rotation matrix R and translation matrix t between the binocular camera and the lidar can be obtained:

[0019]

[0020] Among them, P i is the 3D point in the first set of data, Qi are the 3D points in the second set of data.

[0021] Furthermore, the specific method of step S3 is:

[0022] Target flame recognition is performed using multiple sensors. It is necessary to determine the boundary intersection and union ratio of targets detected by different sensors, that is, to calculate the ratio of the intersection and union of the rectangular box areas, take the maximum value, and set a threshold. If the value exceeds the threshold, it is considered to be the same target. Assuming that the visual bounding box is A and the bounding box obtained by the flame sensor data through the algorithm conversion is B, the calculation formula for the intersection and union ratio of the two is:

[0023]

[0024] Furthermore, the specific method of step S4 is:

[0025] 4.1) The target flame is regarded as the center of the formation structure and the formation is constructed around it. That is, the formation control strategy is to regard the target as part of the formation structure itself and maintain the tracking state by controlling the formation structure;

[0026] 4.2) Use Leader-Follower to build a UAV formation, with the first UAV that discovers and locates the target fire as the Leader and the other UAVs as Followers;

[0027] When a target flame is found, the Follower obtains the location of the Leader and the target through the base station, and approaches the target and performs fire extinguishing operations in a preset formation.

[0028] Furthermore, in a multi-UAV system, each UAV includes:

[0029] A binocular camera, installed in the middle of the drone's nose, with a diagonal viewing angle of 120°, a horizontal viewing angle of 65°, a vertical viewing angle of 55°, and an IMU frequency of 200Hz;

[0030] A laser radar, mounted on top of the drone;

[0031] A flame detector is mounted directly below the fuselage of the drone and detects directly forward;

[0032] A fire extinguishing device is mounted under the drone.

[0033] The beneficial effects of the present invention are as follows: the present invention adopts a SLAM method combining visual and radar information, and adds infrared sensors at the same time, adopts a multi-sensor data fusion algorithm to fuse the data obtained by the sensors, and outputs the fused results to facilitate the positioning of the fire. The present invention adopts multi-sensor spatiotemporal registration to solve the problem of different sensor working platforms or different sampling intervals and coordinate systems for providing measurement information, and the existence of deviations and observation errors in the sensors themselves. The present invention adopts multi-UAV collaboration, adds positioning, planning, navigation, and obstacle avoidance algorithms based on the data obtained by the sensors plus the relative measurement data between the UAVs, and realizes a multi-UAV collaborative autonomous firefighting system, which effectively improves the efficiency of firefighting, reduces the time of firefighting, and protects the safety of life and property to the greatest extent possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of the structure of a multi-UAV system of a multi-UAV coordination method of a multi-sensor fusion SLAM for firefighting according to the present invention;

[0035] Figure 2 It is a structural schematic diagram of a camera model of a multi-UAV collaboration method of a multi-sensor fusion SLAM for firefighting according to the present invention;

[0036] Figure 3 A schematic diagram of a basic framework of data fusion of a multi-UAV collaboration method for multi-sensor fusion SLAM for firefighting according to the present invention;

[0037] Figure 4 This is a flow chart of a flame detection fusion algorithm of a multi-UAV collaboration method of a multi-sensor fusion SLAM for fire fighting according to the present invention;

[0038] Figure 5 This is a schematic diagram of the three-UAV formation structure of a multi-UAV coordination method of multi-sensor fusion SLAM for firefighting in the present invention. DETAILED DESCRIPTION

[0039] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] The present invention provides a multi-UAV coordination method for multi-sensor fusion SLAM for fire fighting, which is based on a multi-UAV system, such as Figure 1 As shown, it includes a base station, and the base station data is connected to multiple drones; each drone is provided with an airborne sensor, and the airborne sensor includes a binocular camera, a laser radar and a flame detector;

[0041] The multi-UAV collaboration method includes the following contents:

[0042] Step S1, performing multi-sensor registration on each of the drones;

[0043] Step S2: Each UAV obtains environmental information through the airborne sensor, generates a three-dimensional point cloud map of the search environment in real time, and feeds back the location information of each UAV to the base station;

[0044] Step S3, each of the UAVs searches for the flame position by using a multi-strip collaborative search and a multi-angle collaborative search method, wherein one of the UAVs obtains the flame position by using a fusion algorithm, and obtains the three-dimensional coordinates of the flame in a global coordinate system by using coordinate calculation based on a three-dimensional point cloud map of the search environment established by the UAV;

[0045] Step S4: The three-dimensional coordinates of the flame are transmitted back to the server, and the server controls multiple nearby drones to perform coordinated fire extinguishing operations based on the data fed back by the base station.

[0046] Among them, in the present invention, multiple drones are controlled by centralized clusters to achieve multi-machine collaboration. The system has a control unit, and the data information of multiple drones is gathered to the central control unit. The central control unit performs unified planning based on global information. The general structure is as follows: Figure 1 As shown:

[0047] The information of each drone includes data from lidar, camera and flame sensor, which are initially processed by the onboard computer and transmitted to the central control unit. The drone's mission is then planned and deployed according to the fire environment and mission requirements (such as firefighting and search). A fire extinguishing device is mounted underneath each drone, which can extinguish the fire autonomously when a flame is discovered. At the same time, the central control unit deploys the drone group according to the size of the fire and the drone's firefighting capability, and ensures the firefighting effect through multi-strip collaborative search and multi-angle collaborative firefighting operations.

[0048] In some embodiments, in step S1, multi-sensor registration includes time registration and space registration; the time registration is to convert the observation information collected by each sensor from asynchronous time to the same time; the spatial registration method is to complete the conversion of the data observed by each sensor in different coordinate systems on the same platform and different coordinate systems on different platforms to the same platform and the same coordinate system; the coordinate system includes radar coordinate system, camera coordinate system, image pixel coordinate system and infrared sensor coordinate system.

[0049] Among them, due to the different working platforms of sensors or the different sampling intervals and coordinate systems of providing observation information, the existence of sensor deviations and observation errors makes multi-sensors need data registration before fusion, that is, the spatiotemporal registration process usually required for the "error-free" conversion of multi-sensor data, which mainly includes two parts: time registration and space registration. Time registration mainly converts the observation information collected by each sensor from asynchronous moments to the same moment. Spatial registration mainly completes the conversion of observation data from different coordinate systems of the same platform or different platforms to the same coordinate system of the same platform.

[0050] Time alignment: Different sensors have different sampling frequencies, so it is difficult to completely align the data at the same time, so the data needs to be aligned in time. Here, thread synchronization is used to create radar data receiving threads, camera data receiving threads, and infrared sensor data receiving threads. Each time radar data is collected, the image data and infrared sensor data of the current frame are obtained, thereby achieving time alignment of multiple sensors.

[0051] Spatial registration: Image quality has a great impact on the accuracy of SLAM. To ensure the quality of image information and establish an accurate camera model, it is necessary to obtain accurate camera parameters. The mathematical model of the camera describes the process of projecting a three-dimensional scene into a two-dimensional image after transformation. Once the camera mathematical model is determined, the position of the projection point of a certain point in the three-dimensional scene in the two-dimensional image can be known. In order to minimize the influence of external factors and improve the accuracy of SLAM, it is necessary to calibrate the internal and external parameters of the camera.

[0052] Camera model such as Figure 2 As shown, point O is the origin of the coordinates. A point P on the real object is projected to point P' on the physical imaging plane O'-x'-y'. At this time, the image obtained is an inverted image, so there is a negative sign in the coordinates of point P', which needs to be normalized to the imaging plane to simplify the model.

[0053] Normalized point P coordinates [X, Y, Z] T with P' coordinates [X',Y',Z'] T The coordinate relationship is as follows:

[0054]

[0055]

[0056] Where f is the distance from the physical imaging plane to the pinhole.

[0057] Set a pixel plane in the physical plane as the sampling and quantization area to get the pixel coordinates [u, v] of P' T :

[0058]

[0059] Among them, c x ,c y is the translation parameter; f x =αf,f y =βf is the normalization factor, which indicates the scaling of coordinates, and its unit is pixel.

[0060] According to formula (0.2), the camera's intrinsic parameter matrix K can be obtained:

[0061]

[0062] After obtaining the camera intrinsic parameters, the camera's rotation matrix R and translation vector T can be used to estimate the camera pose. As the estimated target parameters, the extrinsic parameter matrix describes the world coordinates P of point P. w and pixel coordinates P uv The projection transformation relationship is:

[0063] ZP uv =K(RP w +t)=KTP w (0.4),

[0064] The intrinsic parameters represent the projection relationship between the object point and the image point in the camera coordinate system. R is the rotation matrix of the camera's external parameters, and T is the translation vector of the camera's external parameters. The two determine the orientation and position of the camera installation. The external parameters represent the geometric transformation relationship between the camera coordinate system and the world coordinate system.

[0065] In some embodiments, step S1 further includes joint calibration of the laser radar and the binocular camera, and the specific method is:

[0066] As the two most important sensors for autonomous perception, LiDAR and camera have their own characteristics, making sensor fusion a better detection solution. LiDAR has high resolution, high ranging accuracy, and good real-time performance, but LiDAR is greatly affected by the material properties of obstacles and weather factors (such as rain and fog). Cameras provide rich color and feature information and can detect objects of interest through the most advanced algorithms, but the two-dimensional images obtained from the camera do not have depth of field information, and the three-dimensional information of obstacles cannot be obtained through the image. At the same time, in an environment where a fire occurs, the camera is greatly affected by smoke and fire, and other sensors are needed to assist it. Therefore, LiDAR and camera data fusion is used here to project the LiDAR point cloud onto the image plane.

[0067] The traditional joint calibration of LiDAR and camera is realized by the correspondence between the camera 2D points and the LiDAR 3D points. The 2D point coordinates of the binocular camera are obtained by calculating the vertex coordinates of the calibration plate, and the 3D point of the LiDAR is obtained by fitting the boundary line of the calibration plate to obtain the vertex 3D coordinates; finally, the PNP problem is solved to obtain the corresponding coordinate transfer matrix, as shown in the following formula:

[0068]

[0069] Among them, c x ,c y is the translation parameter; f x =αf,f y =βf is the normalization factor, which indicates the scaling of coordinates. The unit is pixel. u,v are the normalized plane homogeneous coordinate positions of the points. x,y,z are the homogeneous coordinate positions of the points. r 11 To r 33 are the parameters of the rotation matrix, t1, t2, t3 refer to the translation amounts in the xyz axes, and γ is the camera intrinsic parameter;

[0070] The rotation and translation matrix between corresponding points is obtained by ICP method;

[0071] Assuming that two sets of point cloud pairs have been obtained, the objective function to be minimized is as follows, and finally the rotation matrix R and translation matrix t between the binocular camera and the lidar can be obtained:

[0072]

[0073] Among them, P i is the 3D point in the first set of data, Q i are the 3D points in the second set of data.

[0074] In some embodiments, the specific method of step S3 is:

[0075] Target flame recognition is performed using multiple sensors. It is necessary to determine the boundary intersection and union ratio of targets detected by different sensors, that is, to calculate the ratio of the intersection and union of the rectangular box areas, take the maximum value, and set a threshold. If the value exceeds the threshold, it is considered to be the same target. Assuming that the visual bounding box is A and the bounding box obtained by the flame sensor data through the algorithm conversion is B, the calculation formula for the intersection and union ratio of the two is:

[0076]

[0077] Among them, this system is mainly composed of two subsystems and an auxiliary system. The two subsystems are the visual inertial navigation system and the lidar inertial navigation system, and the auxiliary system is the infrared sensor system. The two subsystems adopt a tightly coupled method. The visual inertial navigation system uses the estimation of the lidar inertial navigation for initialization, and combines the measurement of the lidar with the depth obtained by vision to improve the accuracy; at the same time, the lidar inertial navigation system uses the estimated value of the visual inertial navigation as the initial value for frame matching, and uses vision for loop detection. The visual detection results are used to optimize the laser inertial navigation system. In this way, the drone can still work when one of the lidar or vision fails, which greatly improves the robustness of the system. In addition, the infrared sensor system is an auxiliary system, and its data is used for auxiliary detection of fire points. The data fusion framework is as follows Figure 3 shown.

[0078] The first step of data fusion requires combining the radar coordinate system, camera coordinate system, image pixel coordinate system, infrared sensor coordinate system and other coordinate systems. Coordinate system 1 is conducive to the accurate measurement of the distance and direction of obstacles and objects to be detected in the environment by multiple sensors. The external parameter calibration of the camera can map the radar scanning points to the image, and at the same time superimpose the infrared sensor data to obtain a more accurate position of the flame, which enhances the robustness of the system and effectively improves the stability of the drone and the accuracy of detection.

[0079] In addition, since the sensor may miss detection or report false alarms in complex environments, a flame detection fusion algorithm is set up. The flow chart is as follows: Figure 4 As shown:

[0080] In some embodiments, the specific method of step S4 is:

[0081] 4.1) The target flame is regarded as the center of the formation structure, and the formation is constructed around it. That is, the formation control strategy is to regard the target as part of the formation structure itself and maintain the tracking state by controlling the formation structure. Figure 5 As shown in the figure, taking three UAVs as an example, the three UAVs and the ground target form a four-unit formation structure centered on the target in a two-dimensional plane. The main control quantities are the distance coordination between each UAV and the target, and the phase relationship between each UAV.

[0082] 4.2) Use Leader-Follower to build a UAV formation, with the first UAV that discovers and locates the target fire as the Leader and the other UAVs as Followers;

[0083] When a target flame is found, the Follower obtains the location of the Leader and the target through the base station, and approaches the target and performs fire extinguishing operations in a preset formation.

[0084] In some embodiments, in a multi-UAV system, each of the UAVs comprises:

[0085] A binocular camera is installed in the middle of the nose of the UAV, with a diagonal viewing angle of 120°, a horizontal viewing angle of 65°, a vertical viewing angle of 55°, and an IMU frequency of 200 Hz;

[0086] A laser radar installed on the top of the drone;

[0087] A flame detector is mounted on the UAV directly below the fuselage and detects directly forward;

[0088] A fire extinguishing device is mounted below the UAV.

[0089] The laser radar is a 32-line laser radar installed on the top of the drone, and each laser scanning beam adopts radial circular scanning. The basic idea of ​​the radar SLAM used in the present invention is to divide the task into two parts, one is the high-frequency motion estimation problem, and the other is the low-frequency environment mapping problem.

[0090] After the radar receives external environment data, it first reconstructs the point cloud map. The radar's odometer performs motion estimation and coordinate conversion at a fixed frequency (such as 10Hz). At the same time, the mapping thread constructs a three-dimensional map at a lower frequency (such as 2Hz). There is also another thread that optimizes the posture. This multi-threaded parallel structure ensures the real-time performance of the system.

[0091] Among them, the infrared sensor (flame detector) can perform non-contact temperature measurement. The ambient temperature is measured at a long distance through the infrared sensor to obtain a thermal image of the environment. The sensor is very sensitive to flames and detects flames through a special receiving tube. The flame brightness is converted into a level signal by the infrared detector and input into the onboard computer for processing, so as to obtain the approximate direction of the flame in the fire scene and prepare for fire extinguishing.

[0092] The flame detector of the present invention is mounted directly below the fuselage and detects directly forward. It performs flame detection based on the principle of radiant light. After the probe receives the radiation emitted by the flame, it judges the flame according to the intensity. After the output data enters the onboard computer, the data is processed based on the algorithm in the onboard computer. The processed data is fused and analyzed with the data of vision and lidar SLAM to obtain a more accurate flame position, thereby controlling the drone to go to the vicinity of the flame to perform firefighting tasks such as fire extinguishing.

[0093] The present invention adopts multi-sensor fusion SLAM. In the case of fire, the environment is relatively complex. The SLAM method using a single sensor is not stable enough. When only visual SLAM is used, the smoke and fire at the fire scene are very strong, which interferes with the vision very much, causing the drone to be extremely unstable or even explode. If only radar SLAM is used, it is impossible to distinguish the fire site. Therefore, a SLAM method combining visual and radar information is adopted, and an infrared sensor is added at the same time. A multi-sensor data fusion algorithm is used to fuse the data obtained by the sensor, and the fused result is output, which is convenient for locating the fire site.

[0094] The present invention adopts multi-sensor spatiotemporal registration. Different sensor working platforms or sampling intervals and coordinate systems that provide measurement information are different, and the sensors themselves have deviations and observation errors, which requires multi-sensor data registration for fusion, and usually requires multiple sensors to perform "error-free" conversion. It mainly includes two parts: time registration and space registration. In response to the problem of spatiotemporal registration, the present invention proposes a calibration algorithm for 3D points corresponding to 3D points. Time registration mainly converts the observation information collected by each sensor from asynchronous moments to the same moment. The spatial registration principle is mainly to convert the data observed by each sensor in different coordinate systems on the same platform and different coordinate systems on different platforms to the same platform and the same coordinate system. After completing the spatiotemporal registration of cameras, lidars, infrared sensors, etc., the detection results are in the same spatiotemporal coordinate system and can be further processed.

[0095] The present invention adopts multi-UAV collaboration. Based on the data obtained by the sensor and the relative measurement data between the UAVs, positioning, planning, navigation, and obstacle avoidance algorithms are added to realize a multi-UAV collaborative autonomous firefighting system, which effectively improves the firefighting efficiency, reduces the firefighting time, and protects the safety of life and property to the greatest extent possible.

Claims

1. A multi-UAV collaboration method for multi-sensor fusion SLAM for firefighting, characterized in that: It is based on a multi-UAV system, including a base station, the base station data is connected to multiple UAVs; each UAV is provided with an airborne sensor, the airborne sensor includes a binocular camera, a laser radar and a flame detector; The multi-UAV collaboration method includes the following contents: Step S1, performing multi-sensor registration on each of the drones; Step S2: Each UAV obtains environmental information through the airborne sensor, generates a three-dimensional point cloud map of the search environment in real time, and feeds back the location information of each UAV to the base station; Step S3, each of the UAVs searches for the flame position by using a multi-strip collaborative search method and a multi-angle collaborative search method, wherein one of the UAVs obtains the flame position by using a fusion algorithm, and obtains the three-dimensional coordinates of the flame in a global coordinate system by using coordinate calculation based on a three-dimensional point cloud map of the search environment established by the UAV; The specific method of step S3 is: Target flame recognition is performed using multiple sensors. It is necessary to determine the boundary intersection and union ratio of targets detected by different sensors, that is, to calculate the ratio of the intersection and union of the rectangular box areas, take the maximum value, and set a threshold. If the value exceeds the threshold, it is considered to be the same target. Assuming that the visual bounding box is A and the bounding box obtained by the flame sensor data through the algorithm conversion is B, the calculation formula for the intersection and union ratio of the two is: Step S4: The three-dimensional coordinates of the flame are transmitted back to the server, and the server controls multiple nearby drones to perform coordinated fire extinguishing operations based on the data fed back by the base station.

2. A multi-UAV coordination method for multi-sensor fusion SLAM for firefighting as claimed in claim 1, characterized in that: In step S1, the multi-sensor registration includes time registration and space registration; the time registration is to convert the observation information collected by each sensor from asynchronous time to the same time; the spatial registration method is to convert the data observed by each sensor in different coordinate systems on the same platform and different coordinate systems on different platforms to the same platform and the same coordinate system; the coordinate system includes a radar coordinate system, a camera coordinate system, an image pixel coordinate system and an infrared sensor coordinate system.

3. A multi-UAV coordination method for multi-sensor fusion SLAM for firefighting as claimed in claim 2, characterized in that: The step S1 also includes joint calibration of the laser radar and the binocular camera, and the specific method is as follows: The 2D point coordinates of the binocular camera are obtained by calculating the vertex coordinates of the calibration plate, and the 3D point of the laser radar is obtained by fitting the boundary line of the calibration plate to obtain the vertex 3D coordinates; finally, the PNP problem is solved to obtain the corresponding coordinate transfer matrix, as shown in the following formula: Among them, c x ,c y is the translation parameter; f x =αf,f y =βf is the normalization factor, which indicates the scaling of coordinates. The unit is pixel. u,v are the normalized plane homogeneous coordinate positions of the points. x,y,z are the homogeneous coordinate positions of the points. r 11 To r 33 are the parameters of the rotation matrix, t1, t2, t3 refer to the translation amounts in the xyz axes, and γ is the camera intrinsic parameter; The rotation and translation matrix between corresponding points is obtained by ICP method; Assuming that two sets of point cloud pairs have been obtained, the objective function to be minimized is as follows, and finally the rotation matrix R and translation matrix t between the binocular camera and the lidar can be obtained: Among them, P i is the 3D point in the first set of data, Q i are the 3D points in the second set of data.

4. The multi-UAV coordination method of multi-sensor fusion SLAM for fire fighting as claimed in claim 1, characterized in that: The specific method of step S4 is: 4.1) The target flame is regarded as the center of the formation structure and the formation is constructed around it. That is, the formation control strategy is to regard the target as part of the formation structure itself and maintain the tracking state by controlling the formation structure; 4.2) Use Leader-Follower to build a UAV formation, with the first UAV that discovers and locates the target fire as the Leader and the other UAVs as Followers; When a target flame is found, the Follower obtains the location of the Leader and the target through the base station, and approaches the target and performs fire extinguishing operations in a preset formation.

5. A multi-UAV coordination method for multi-sensor fusion SLAM for firefighting as claimed in claim 1 or 2, characterized in that: In the multi-UAV system, each UAV comprises: A binocular camera is installed in the middle of the nose of the UAV, with a diagonal viewing angle of 120°, a horizontal viewing angle of 65°, a vertical viewing angle of 55°, and an IMU frequency of 200 Hz; A laser radar installed on the top of the drone; A flame detector is mounted on the UAV directly below the fuselage and detects directly forward; A fire extinguishing device is mounted below the UAV.

Images