A collaborative air-ground life detection and mapping method for earthquake relief scenarios

Abstract

A method for collaborative air-ground life detection and mapping applied to earthquake relief scenarios. The present invention relates to a method for collaborative air-ground life detection and mapping, and belongs to the technical field of robot active mapping and exploration algorithms. The purpose of the present invention is to solve the problem that existing methods only focus on map establishment and environmental exploration, and do not organically combine functions such as life detection with map establishment. The established maps often cannot play a targeted guiding role in search and rescue missions, resulting in low search and rescue efficiency and poor safety. The method of the present invention includes collaborative mapping of drones and unmanned vehicles, which can actively explore in the absence of a complete map, detect signs of life, and further supplement the map. The life sign information is integrated with the three-dimensional map to generate a point cloud map containing life sign information, providing targeted guidance for earthquake relief and effectively improving the efficiency and safety of search and rescue.

Description

Technical Field

[0001] The present invention relates to an air-ground collaborative life detection and mapping method, and belongs to the technical field of robot active mapping and exploration algorithms. Background Art

[0002] Disasters such as earthquakes, mudslides, fires, and coal mine collapses have caused significant casualties and property losses. Traditional methods for earthquake relief efforts rely heavily on manual search and rescue, which is not only inefficient and labor-intensive, but also extremely dangerous. Therefore, the introduction of drones and unmanned vehicles (UAVs) in these scenarios is highly significant. However, remote control of unmanned equipment not only requires skilled operators but also requires constant monitoring of sensor data, which carries significant uncertainty. Furthermore, any life signs detected can be difficult to clearly convey to rescue personnel.

[0003] With the development of science and technology, multi-agent collaborative mapping technology has received considerable attention and research. However, the vast majority of research focuses only on map building and environmental exploration, without organically integrating functions such as life detection with map building. As a result, the resulting maps often fail to provide targeted guidance for search and rescue missions. Summary of the Invention

[0004] The purpose of this invention is to solve the problem that existing methods only focus on map creation and environmental exploration, but do not organically combine functions such as life detection with map creation. The maps created often cannot provide targeted guidance for search and rescue missions, resulting in low search and rescue efficiency and poor safety. This invention proposes an air-ground collaborative life detection and mapping method for earthquake relief scenarios.

[0005] A method for air-ground collaborative life detection and mapping applied in earthquake relief scenarios has the following specific steps:

[0006] Step 1: Create a 3D point cloud map of the unmanned vehicle;

[0007] Step 2: Create a 3D point cloud map of the drone;

[0008] Step 3: Use the NDT algorithm to match the 3D point cloud map of the unmanned vehicle and the 3D point cloud map of the drone to obtain the relative pose A of the 3D point cloud map of the unmanned vehicle and the 3D point cloud map of the drone;

[0009] Step 4: Based on the relative pose A of the unmanned vehicle's 3D point cloud map and the drone's 3D point cloud map, the unmanned vehicle's 3D point cloud map and the drone's 3D point cloud map are merged with the unmanned vehicle's 3D point cloud map, using the unmanned vehicle's mapping origin as the origin of the coordinate system.

[0010] Step 5: Update the UAV's pose Tbias×A based on the relative pose A of the 3D point cloud map of the unmanned vehicle and the 3D point cloud map of the UAV, so that the UAV maps the newly received point cloud into the merged map.

[0011] Step 6: Repeat steps 1 and 2 to continuously expand and merge the map, and execute steps 7 and 14 at the same time;

[0012] Step 7: Convert the merged map expanded in step 6 into an octree map, find the octree map boundary, obtain candidate points through clustering, threshold processing, and finding the center of gravity, and the candidate points form a candidate location list F; calculate the utility of the candidate points and select the candidate point with the highest utility as the target point of the UAV;

[0013] The path from the current position of the UAV to the candidate point with the highest utility is used as the planned path of the UAV;

[0014] Step 8: Remove the candidate locations F with the target point of the drone as the center and a radius of R. obs Candidate points of the unmanned vehicle are obtained, and a list of candidate points for the unmanned vehicle is obtained.

[0015] Using A * The algorithm calculates the current position of the unmanned vehicle Middle The path of candidate points

[0016]

[0017] in, For path The first point;

[0018] For path Middle points;

[0019] The length of the path from the current position of the unmanned vehicle to each candidate point is recorded as

[0020] Step 9: Assume the average speed of the drone is Calculate the estimated time of the drone's path from its current position to each candidate point

[0021] Step 10: Current location of the unmanned vehicle The path of each candidate point in Calculate the utility and select the candidate point with the highest utility as the target point of the unmanned vehicle;

[0022] The path from the current position of the autonomous vehicle to the candidate point with the highest utility is used as the planned path of the autonomous vehicle;

[0023] Step 11: Control the unmanned vehicle and the drone to move toward the selected target point;

[0024] Step 12: Determine whether the UAV and the unmanned vehicle have reached the target point. If so, proceed to step 13; if not, proceed to step 11.

[0025] Step 13: Determine whether the exploration is completed. If so, end; if not, proceed to step 6;

[0026] Step 14: Perform infrared thermal imaging detection of unmanned vehicles and drones on the merged map.

[0027] The beneficial effects of the present invention are:

[0028] The present invention proposes a collaborative air-ground life detection and mapping method for earthquake relief and disaster relief, involving the collaborative mapping of drones and unmanned vehicles. This method can proactively explore without a complete map, detecting life signs and further supplementing the map. This method also fuses life sign information with a three-dimensional map to generate a point cloud map containing this information, providing targeted guidance for earthquake relief and disaster relief, effectively improving the efficiency and safety of search and rescue efforts.

[0029] This invention provides an air-ground collaborative life detection and mapping method. It uses infrared thermal imagers for life detection and combines the results with a constructed three-dimensional map. This method not only presents the overall earthquake relief site environment but also identifies areas where potential life is detected. The locations, thermal images, and RGB images of these areas are recorded, and these areas are marked in the constructed global point cloud map. Finally, a complete three-dimensional map containing potential life signs is formed. The vehicles used are drones and unmanned vehicles, both equipped with lidar, visual cameras, and infrared thermal imagers.

[0030] This paper proposes a collaborative air-ground life detection and mapping method that creates a three-dimensional map of the disaster area, including information on potential life signs. Compared with traditional manual overlay search methods, this method significantly improves the efficiency of searching for survivors in disaster areas, reduces the risk of searching, and provides targeted guidance to searchers.

[0031] The present invention adopts an air-ground collaborative exploration system to make up for the shortcomings of unmanned vehicles' limited movement and insufficient endurance of drones, and has the advantages of fast exploration speed, strong maneuverability and long endurance.

[0032] The present invention designs a boundary-based active exploration method for heterogeneous vehicles, which comprehensively considers the utility equation to achieve the effect of obtaining more information in a shorter time (exploring a larger area in less time), greatly improving the efficiency of exploration. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0034] Specific embodiment 1: This embodiment is an air-ground collaborative life detection and mapping method applied in earthquake relief scenarios. The specific process is as follows:

[0035] Step 1: Create a 3D point cloud map of the unmanned vehicle (the map continues to grow);

[0036] Step 2: Create a 3D point cloud map of the drone (the map keeps getting bigger);

[0037] Step 3: Use the NDT algorithm to match the 3D point cloud map of the unmanned vehicle and the 3D point cloud map of the drone to obtain the relative pose A of the 3D point cloud map of the unmanned vehicle and the 3D point cloud map of the drone;

[0038] NDT matching is performed only once;

[0039] After NDT matching is successful, the relative pose between point clouds can be directly output;

[0040] Step 4: Based on the relative pose A of the unmanned vehicle's 3D point cloud map and the drone's 3D point cloud map, the unmanned vehicle's 3D point cloud map and the drone's 3D point cloud map are merged with the unmanned vehicle's 3D point cloud map, using the unmanned vehicle's mapping origin as the origin of the coordinate system.

[0041] Step 5: Update the drone’s pose T based on the relative pose A of the unmanned vehicle’s 3D point cloud map and the drone’s 3D point cloud map. bias ×A(UAV's T bias , Step 20, Update T bias : ), so that the UAV maps the newly received point cloud into the merged map;

[0042] Step 6: Repeat steps 1 and 2 to continuously expand and merge the map, and execute steps 7 and 14 at the same time;

[0043] Step 7: Convert the merged map expanded in step 6 into an octree map, find the octree map boundary, obtain candidate points through clustering, threshold processing, and finding the center of gravity, and the candidate points form a candidate location list F; calculate the utility of the candidate points and select the candidate point with the highest utility as the target point of the UAV;

[0044] The path from the current position of the UAV to the candidate point with the highest utility is used as the planned path of the UAV;

[0045] Step 8: Remove the candidate locations F with the target point of the drone as the center and a radius of R. obs (Set) candidate points, get the list of candidate points for the unmanned vehicle

[0046] Using A * The algorithm calculates the current position of the unmanned vehicle Middle The path of candidate points

[0047]

[0048] in, For path The first point;

[0049] For path Middle points;

[0050] The length of the path from the current position of the unmanned vehicle to each candidate point is recorded as

[0051] For those who plan UAVs first and then UAVs, the plan before steps 4 and 5 is for UAVs, and after steps 4 and 5 is for UAVs;

[0052] Step 9: Assume the average speed of the drone is Calculate the estimated time of the drone's path from its current position to each candidate point

[0053] A three-dimensional raster map with a fixed resolution is stored in an octree map, which is equivalent to dividing the space into countless small cubes with a fixed resolution, such as (0.5m×0.5m×0.5m). Each small cube is called a voxel block. The resolution is determined by the actual application.

[0054] Step 10: Current location of the unmanned vehicle The path of each candidate point in Calculate the utility and select the candidate point with the highest utility as the target point of the unmanned vehicle;

[0055] The path from the current position of the autonomous vehicle to the candidate point with the highest utility is used as the planned path of the autonomous vehicle;

[0056] Step 11: The central control terminal sends control instructions to the host computers of the unmanned vehicle and drone, and sends them to the underlying controller via the serial port to control the unmanned vehicle and drone to move toward the selected target point;

[0057] According to the planned path points and the specific unmanned vehicle / drone model and control principle, the control instructions that need to be sent are calculated and sent.

[0058] Depending on the type of drone or unmanned vehicle, the control commands issued are also different. The signal is usually in the form of PWM signal sent to the servos and motors on the drone or unmanned vehicle.

[0059] Step 12: Determine whether the UAV and the unmanned vehicle have reached the target point. If so, proceed to step 13; if not, proceed to step 11. The target point is to cover the entire area.

[0060] Step 13: Determine whether the exploration is completed. If so, end; if not, proceed to step 6;

[0061] The space to be explored must be specified in advance, for example, exploring within a space of 100m×100m×10m. When the proportion of unexplored areas reaches a certain requirement (a value set in advance, adjusted according to actual conditions), for example, if the exploration degree is set to 90%, the exploration is considered complete. In the octree map, the exploration is considered complete when the ratio of the occupied area marked as 1 to the explored blank area marked as 0 exceeds 90%.

[0062] Step 14: Perform infrared thermal imaging detection of unmanned vehicles and drones on the merged map.

[0063] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that: in step 1, a three-dimensional point cloud map of the unmanned vehicle is established; the specific process is:

[0064] Step 1: Calculate the curvature c of each point in the laser point cloud based on the laser point cloud data received by the unmanned vehicle's laser radar, and obtain the unmanned vehicle laser point cloud P based on the curvature. UGV Plane points and unmanned vehicle laser point cloud P UGV edge points;

[0065] Step 1 and 2: For the unmanned vehicle laser point cloud P UGV edge points (m), extract the two edge points closest to the edge point in the previous frame, and calculate the distance d from the edge point to the straight line formed by the two closest edge points ε i(m), i=1,2,…,m, m is the number of edge points;

[0066] For unmanned vehicle laser point cloud P UGVPlane points (n), extract the three non-collinear planar points closest to the plane point in the previous frame, and calculate the distance from the plane point to the plane formed by the three non-collinear planar points closest to the plane point (n), n is the number of plane points;

[0067] The calculation formula is as follows:

[0068]

[0069] Among them, d εi is the distance from the ith edge point to the straight line formed by the two nearest edge points, X (k+1,i) is the i-th edge point of the k+1 frame, X (k,j) and X (k,l) is the value of X in the kth frame (k+1,i) The two closest edge points;

[0070]

[0071] in, For the The distance from a plane point to the plane formed by the three non-collinear plane points closest to the plane point, is the k+1th frame plane points, and is the kth frame with The three closest non-collinear plane points; · represents the dot product.

[0072] Step 13: Set the relative pose transformation T between each two frames scan The initial value is

[0073] Step 14: d εi and Perform nonlinear optimization to solve all d εi and The sum of the distances is minimized, and the relative pose transformation T between each two frames is obtained. scan ; The expression is:

[0074]

[0075] Among them, m is the number of edge points, n is the number of plane points;

[0076] Take f(T scan ) is minimum when T scan is the relative pose transformation T between each two frames scan ;

[0077] Since the front end is the matching between frames, the relative pose transformation T between each two frames is obtained scan ;

[0078] Step 15: Set the initial relative position of the lidar on the unmanned vehicle relative to the origin of the world coordinate system (the lidar of the unmanned vehicle is used as the origin, which is set and unchanged)

[0079] Step 16: Based on the initial relative pose and the relative pose transformation T between each two frames scan , transform the rough relative pose of the current lidar on the unmanned vehicle relative to the origin of the mapping world coordinate system:

[0080] Get the rough relative position of the current lidar on the unmanned vehicle relative to the origin of the mapping world coordinate system

[0081] Since this process will produce cumulative errors over time, point cloud to map matching is required to obtain a more accurate pose relationship.

[0082] Step 17: Assume that the exact relative position of the current lidar on the unmanned vehicle relative to the origin of the mapping world coordinate system is

[0083] Step 18: Map the current laser point cloud on the unmanned vehicle to the world coordinate system of the map, and obtain the current laser point cloud P′ on the unmanned vehicle in the world coordinate system of the map. UGV , the expression is:

[0084]

[0085] Among them, P UGV For laser point cloud, is the exact relative pose; represents the dot product; P′ UGV The current laser point cloud on the unmanned vehicle in the world coordinate system for mapping;

[0086] Step 19: Map the current laser point cloud P′ on the unmanned vehicle to the world coordinate system UGV Match with the local map (that is, the local point cloud) to obtain the accurate relative pose

[0087] The local map is:

[0088] With the unmanned vehicle as the center, take a cube with a side length of d, and the point cloud map inside the cube is the local point cloud map P loc ;

[0089] Step 10: Update Tbias:

[0090]

[0091] Step 11: Transform the precise relative pose of the current frame lidar on the unmanned vehicle relative to the origin of the world coordinate system for mapping Map the current point cloud to the global map (world coordinate system);

[0092] Step 12: Repeat steps 11 to 11 to update the unmanned vehicle's three-dimensional point cloud map, thereby completing the map creation (the map continues to grow).

[0093] Other steps and parameters are the same as those in the first embodiment.

[0094] Specific embodiment three: This embodiment differs from specific embodiment one or two in that: in step one, the curvature c of each point in the laser point cloud is calculated based on the laser point cloud data received by the laser radar of the unmanned vehicle, and the laser point cloud P of the unmanned vehicle is obtained based on the curvature. UGV Plane points and unmanned vehicle laser point cloud P UGV The specific process is:

[0095] The processing of point clouds from drones and autonomous vehicles is similar. For drone point clouds, only the corresponding pose transformation is required, and then they are mapped to the global map coordinate system for point cloud matching. Therefore, we will first describe the processing of point clouds received by autonomous vehicles.

[0096] When the laser radar of the unmanned vehicle receives the laser point cloud P UGV , calculate the laser point cloud P UGV The curvature c of each point in;

[0097] Based on laser point cloud P UGV The curvature c of each point in the image is arranged from small to large, and the points corresponding to the curvatures of the top n positions are used as the laser point cloud P of the unmanned vehicle. UGV Plane points of

[0098] Based on laser point cloud P UGV The curvature c of each point in the image is arranged from large to small, and the points corresponding to the curvatures in the top m positions are used as the laser point cloud P of the unmanned vehicle. UGV edge points;

[0099] The calculation formula is as follows:

[0100]

[0101] Where S is the set of points selected from the row of the selected point, with the selected point as the center. (A frame of data scanned by the laser radar can be regarded as an M×N matrix. The curvature calculation is the curvature of each point relative to the row where the point is located. That is, the points selected are all points in the same row as the point. S is the number of points obtained). i′ is the i′th point in the set S, i′∈S; X j′ is the j′th point in the set S, j′∈S.

[0102] Other steps and parameters are the same as those in the first or second embodiment.

[0103] Specific embodiment 4: This embodiment differs from any one of specific embodiments 1 to 3 in that: in step 19, the current laser point cloud P′ on the unmanned vehicle is mapped to the world coordinate system of the map. UGV Match with the local map (that is, the local point cloud) to obtain the accurate relative pose

[0104] The local map is:

[0105] With the unmanned vehicle as the center, take a cube with a side length of d, and the point cloud map inside the cube is the local point cloud map P loc ;

[0106] The matching process is:

[0107] The processing of point clouds from drones and autonomous vehicles is similar. For drone point clouds, only the corresponding pose transformation is required, and then they are mapped to the global map coordinate system for point cloud matching. Therefore, we will first describe the processing of point clouds received by autonomous vehicles.

[0108] Step 191: For the current laser point cloud P′ on the unmanned vehicle UGV Edge points (many), extract the local point cloud map P loc The two edge points closest to the edge point are used to calculate the distance d between the edge point and the straight line formed by the two closest edge points. εi (a lot of);

[0109] For the current laser point cloud P′ on the unmanned vehicle UGV Plane points (many), extract local point cloud map P loc The three non-collinear plane points closest to the plane point are used to calculate the distance from the plane point to the plane formed by the three non-collinear plane points closest to the plane point. (a lot of);

[0110] Step 192: Put the current laser point cloud P′ on the unmanned vehicle UGV Match with the local map (that is, the local point cloud) to obtain the accurate relative pose The process is:

[0111]

[0112] Among them, m is the number of edge points, n is the number of plane points;

[0113] Take the envoy The minimum corresponding The precise relative position transformation of the current laser radar on the unmanned vehicle relative to the origin of the mapping world coordinate system

[0114] The other steps and parameters are the same as those in the first to third embodiments.

[0115] Specific embodiment 5: This embodiment differs from any one of specific embodiments 1 to 4 in that: in step 2, a three-dimensional point cloud map of a drone is established (the map is continuously enlarged); the specific process is:

[0116] Step 2. Calculate the curvature c of each point in the laser point cloud based on the laser point cloud data received by the UAV's laser radar, and obtain the UAV laser point cloud P based on the curvature. UAV Plane points and UAV laser point cloud P UAV edge points;

[0117] Step 2: For the UAV laser point cloud P UAV edge points (m), extract the two edge points closest to the edge point in the previous frame, and calculate the distance d from the edge point to the straight line formed by the two closest edge points ε i(m), i=1,2,…,m, m is the number of edge points;

[0118] For UAV laser point cloud P UAV Plane points (n), extract the three non-collinear planar points closest to the plane point in the previous frame, and calculate the distance from the plane point to the plane formed by the three non-collinear planar points closest to the plane point (n), n is the number of plane points;

[0119] Step 2 and 3: Set the relative pose transformation T between each two frames scan The initial value is

[0120] Step 24: d εi and Perform nonlinear optimization to solve all d εi and The sum of the distances is minimized, and the relative pose transformation T between each two frames is obtained. scan ;

[0121] Step 25: Set the initial relative pose of the UAV's LiDAR relative to the origin of the world coordinate system (the UAV's LiDAR is the origin, set and unchanged)

[0122] Step 26: Based on the initial relative pose and the relative pose transformation T between each two frames scan , transform the rough relative pose of the current lidar on the drone relative to the origin of the mapping world coordinate system:

[0123] Get the rough relative pose of the current lidar on the drone relative to the origin of the mapping world coordinate system

[0124] Since this process will produce cumulative errors over time, point cloud to map matching is required to obtain a more accurate pose relationship.

[0125] Step 27: Assume that the exact relative position of the current lidar on the drone relative to the origin of the mapping world coordinate system is

[0126] Step 28: Map the current laser point cloud on the UAV to the world coordinate system of the map, and obtain the current laser point cloud P′ on the UAV in the world coordinate system of the map UAV , the expression is:

[0127]

[0128] Among them, P UAV For laser point cloud, is the exact relative pose; represents the dot product; P′ UAV The current laser point cloud on the UAV in the world coordinate system of the map;

[0129] Step 29: Map the current laser point cloud P′ on the drone to the world coordinate system UAV Match with the local map (that is, the local point cloud) to obtain the accurate relative pose

[0130] The local map is:

[0131] With the drone as the center, take a cube with a side length of d, and the point cloud map inside the cube is the local point cloud map P loc ;

[0132] Step 20: Update T bias :

[0133]

[0134] Step 21: Transform the precise relative position of the current laser radar on the drone relative to the origin of the world coordinate system for mapping Map the current point cloud to the global map (world coordinate system);

[0135] Step 22: Repeat steps 21 to 21 to update the drone's three-dimensional point cloud map, thereby achieving map creation (the map continues to grow).

[0136] The specific process of each step is the same as that of step 1 to create a UAV 3D point cloud map.

[0137] Other steps and parameters are the same as those in Specific Embodiments 1 to 4-1.

[0138] Specific embodiment 6: This embodiment differs from any one of specific embodiments 1 to 5 in that: in step 7, the merged map expanded in step 6 is converted into an octree map, the octree map boundary is found, candidate points are obtained through clustering, threshold processing, and finding the center of gravity, and the candidate points form a candidate position list F; the utility of the candidate points is calculated, and the candidate point with the highest utility is selected as the target point of the drone;

[0139] The path from the current position of the UAV to the candidate point with the highest utility is used as the planned path of the UAV;

[0140] The specific process is:

[0141] Step 7.1. Convert the merged map into an octree map;

[0142] Unexplored areas are recorded as 0.5, occupied areas are recorded as 1, and explored empty areas are recorded as 0;

[0143] The octree map consists of cubes, each representing a voxel block;

[0144] Initially, the occupancy rate of each voxel block is set to 0.5;

[0145] If there is one or more unexplored areas in the spatial 26 neighborhood of the blank area grid (if there is one around a square, only this square is the boundary), then the blank area is defined as the boundary area;

[0146] A three-dimensional raster map with a fixed resolution is stored in an octree map, which is equivalent to dividing the space into countless small cubes with a fixed resolution, such as (0.5m×0.5m×0.5m). Each small cube is called a voxel block. The resolution is determined by the actual application.

[0147] As the drone and the unmanned vehicle move, the boundaries within the lidar detection range near the drone and the unmanned vehicle are continuously updated according to their positions in the established map.

[0148] Step 72: Randomly sample all boundary areas and perform clustering on the sampled boundary areas to obtain candidate points and form a list F of boundary candidate target points. The specific process is as follows:

[0149] The adjacent boundaries are clustered into one category and the pre-set threshold T is used to h Remove the classes with smaller numbers and use the centroids of each remaining class as candidate points to form a list F of candidate boundary target points;

[0150] Step 7.3. Use A * The algorithm calculates the path R from the current position of the drone to the δth candidate point in F δ (R1, R2, R3);

[0151]

[0152] in, For path R δ The first point;

[0153] For path R δ The jth point in

[0154] The length of the path from the current position of the drone to each candidate point is recorded as L δ ,

[0155] for point with dot The Euclidean distance of For path R δ The kth point in For path R δ The k+1th point in the

[0156] The path from the current position to the first candidate point of F is R1, where R1 is a set of points, which are

[0157] For those who plan UAVs first and then UAVs, the plan before steps 4 and 5 is for UAVs, and after steps 4 and 5 is for UAVs;

[0158] Step 7.4: Assume the average speed of the drone is v ave , calculate the estimated time T(R δ )=L δ / v ave ;

[0159] A three-dimensional raster map with a fixed resolution is stored in an octree map, which is equivalent to dividing the space into countless small cubes with a fixed resolution, such as (0.5m×0.5m×0.5m). Each small cube is called a voxel block. The resolution is determined by the actual application.

[0160] Step 75: Calculate the path R from the current position of the drone to each candidate point in F δ Calculate the utility and select the candidate point with the highest utility as the target point of the UAV;

[0161] The path from the current position of the UAV to the candidate point with the highest utility is used as the planned path of the UAV.

[0162] Other steps and parameters are the same as those in Specific Implementations 1 to 5-1.

[0163] Specific embodiment seven: This embodiment differs from one of the specific embodiments one to six in that: in step seven-five, the path R from the current position of the drone to each candidate point in F is calculated. δ Calculate the utility and select the candidate point with the highest utility as the target point of the UAV;

[0164] The path from the current position of the UAV to the candidate point with the highest utility is used as the planned path of the UAV;

[0165] The specific process is:

[0166] Step 751: Calculate the map entropy H(x) of each candidate point x; the specific process is:

[0167] There are M (16×3600) hormone bundles at the candidate point x;

[0168] Each hormone beam passes through the voxel block in the octree map. When it reaches the voxel block with an occupation probability of 0.5 (stop the next calculation), the map entropy of the voxel block with an occupation probability of 0.5 and the map entropy of each voxel block passed through are calculated, and the map entropy of the voxel block with an occupation probability of 0.5 and each voxel block passed through are added together as the map entropy of a hormone beam.

[0169] Each voxel block has an occupancy probability, which is continuously updated based on the received point cloud. The occupancy rate of each voxel block is updated according to the following formula:

[0170]

[0171] Among them, P(n|z 1:t ) represents the occupancy probability of voxel block n from the 1st to the tth observation, z 1:trepresents the first to t-th observation, n represents the n-th voxel block, P(n|z t ) represents the occupancy probability of voxel block n under the t-th observation, P(n|z 1:t-1 ) represents the occupancy probability of voxel block n from the 1st to the t-1th observation, z t represents the t-th observation, P(n) represents the occupancy probability of voxel block n;

[0172] The map entropy of M (16×3600) hormone bundles is summed as the map entropy H(x) of each candidate point x;

[0173] The calculation formula of the map entropy of each voxel block v is as follows:

[0174] Η(v)=-P O (v)ln(P O (v))-(1-P O (v))ln(1-P O (v))

[0175] Where Η(v) is the map entropy of voxel block v, P O (v) is the occupancy probability of voxel block v;

[0176] A three-dimensional raster map with a fixed resolution is stored in an octree map, which is equivalent to dividing the space into countless small cubes with a fixed resolution, such as (0.5m×0.5m×0.5m). Each small cube is called a voxel block. The resolution is determined by the actual application.

[0177] Step 752: Calculate the utility of each candidate point x. The calculation formula is as follows:

[0178]

[0179] Select the candidate point with the highest utility as the target point of the UAV;

[0180] The path from the current position of the drone to the target point of the drone in F is used as the planned path of the drone; for example, if R6 is the largest, the path corresponding to R6 is the planned path of the drone.

[0181] The other steps and parameters are the same as those in the first to sixth embodiments.

[0182] Specific embodiment eight: This embodiment differs from any one of the specific embodiments one to seven in that: in step ten, the current position of the unmanned vehicle is The path of each candidate point in Calculate the utility and select the candidate point with the highest utility as the target point of the unmanned vehicle;

[0183] The path from the current position of the autonomous vehicle to the candidate point with the highest utility is used as the planned path of the autonomous vehicle;

[0184] The specific process is:

[0185] Step 10.1. Calculate the map entropy H(x) of each candidate point x; the specific process is:

[0186] There are M (16×3600) hormone bundles at the candidate point x;

[0187] Each hormone beam passes through the voxel block in the octree map. When it reaches the voxel block with an occupation probability of 0.5 (stop the next calculation), the map entropy of the voxel block with an occupation probability of 0.5 and the map entropy of each voxel block passed through are calculated, and the map entropy of the voxel block with an occupation probability of 0.5 and each voxel block passed through are added together as the map entropy of a hormone beam.

[0188] Each voxel block has an occupancy probability, which is continuously updated based on the received point cloud. The occupancy rate of each voxel block is updated according to the following formula:

[0189]

[0190] Among them, P(n|z 1:t ) represents the occupancy probability of voxel block n from the 1st to the tth observation, z 1:t represents the first to t-th observation, n represents the n-th voxel block, P(n|z t ) represents the occupancy probability of voxel block n under the t-th observation, P(n|z 1:t-1 ) represents the occupancy probability of voxel block n from the 1st to the t-1th observation, z t represents the tth (current) observation, P(n) represents the occupancy probability of voxel block n;

[0191] The map entropies of M (16×3600) hormone bundles are summed up as the map entropy H(x) of a candidate point x;

[0192] Step 10.2: Calculate the utility of each candidate point x. The calculation formula is as follows:

[0193]

[0194] Select the candidate point with the highest utility as the target point of the unmanned vehicle;

[0195] The path from the current position of the unmanned vehicle to the candidate point with the highest utility is used as the planned path of the unmanned vehicle; for example, if R6 is the largest, the path corresponding to R6 is the planned path of the drone.

[0196] Other steps and parameters are the same as those in Specific Embodiments 1 to 7-1.

[0197] Specific embodiment 9: This embodiment differs from any one of specific embodiments 1 to 8 in that: in step 14, infrared thermal imaging detection of unmanned vehicles and drones is performed on the combined map; the specific process is:

[0198] Step 14: Install infrared thermal imaging sensors on the drone and the unmanned vehicle, and receive thermal imaging information sent back by the infrared thermal imaging sensors;

[0199] Step 142: Determine whether there is an area representing human body temperature in the thermal image information returned by the infrared thermal imaging sensor. If so, execute step 143; if not, execute step 141;

[0200] The human body temperature is 30°C≤T≤38°C, where T is the human body temperature;

[0201] Step 14: Binarize the thermal image of the area with human body temperature according to the set threshold, cluster it after binarization, and eliminate the class with small number of pixels according to the preset threshold, and calculate the centroid of each remaining class;

[0202] Step 14: Select an unselected centroid from the centroids of each remaining class;

[0203] Step 14: Set the depth value of the center of gravity selected in step 14:4 in the camera coordinate system to Z;

[0204] Step 14: Convert the selected center of gravity from the camera coordinate system to the world coordinate system according to the pinhole camera model;

[0205] Step 147: Determine whether there is a point cloud (in the point cloud map) within a radius d around the selected centroid. If there is a point cloud, there is an obstacle and proceed to step 148. If there is no point cloud, set Z = Z + s, where s is the step size, and proceed to steps 14 and 15.

[0206] Step 148: Mark the point cloud within the range of the selected centroid radius d on the merged map (mark the point cloud within this range) and record the thermal image information at this time;

[0207] Step 149: Determine whether all the centers of gravity in the frame have been traversed. If so, execute step 140; if not, execute step 144;

[0208] The information transmitted back is a thermal image frame by frame, and the subsequent processing is frame by frame;

[0209] Step 140: Determine whether the exploration is completed. If so, end; if not, execute step 141;

[0210] The space to be explored must be specified in advance, for example, exploring within a space of 100m×100m×10m. When the proportion of unexplored areas reaches a certain requirement (a value set in advance, adjusted according to actual conditions), for example, if the exploration degree is set to 90%, the exploration is considered complete. In the octree map, the exploration is considered complete when the ratio of the occupied area marked as 1 to the blank area explored as 0 exceeds 90%.

[0211] The other steps and parameters are the same as those in the specific implementation modes 1 to 8-1.

[0212] Specific embodiment 10: This embodiment differs from any one of specific embodiments 1 to 9 in that in step 146, the selected center of gravity is converted from the camera coordinate system to the world coordinate system according to the pinhole camera model; the specific process is:

[0213] According to the pinhole camera model, the pixel coordinates [u,v] T The coordinates [X, Y, Z] in the camera coordinate system T Relationship:

[0214]

[0215] where f x 、f y 、c x and c y is the camera internal parameter, the unit is pixel;

[0216] Z is the Z axis in the camera coordinate system, that is, the depth value of the center of gravity in the camera coordinate system is Z;

[0217] P uv It is the homogeneous coordinate form in the pixel coordinate system;

[0218] It is defined as K is the camera intrinsic parameter matrix and P is the coordinate in the camera coordinate system;

[0219] The superscript T means to find the transpose;

[0220] Since [X,Y,Z] T In the camera coordinate system, the camera coordinate system needs to be converted to the world coordinate system. The calculation formula is:

[0221]

[0222] in is the homogeneous transformation matrix of the camera coordinate system relative to the world coordinate system, P W is the coordinate of the target point in the world coordinate system;

[0223] The homogeneous transformation matrix of the camera coordinate system relative to the world coordinate system

[0224] is the relative position of the unmanned vehicle relative to the origin W of the mapping world coordinate system, is the relative position of the camera relative to the unmanned vehicle;

[0225] The unmanned vehicle is a drone or an unmanned vehicle;

[0226] described The acquisition process is:

[0227] In the merged map, the relative position of the unmanned vehicle relative to the origin W of the world coordinate system can be obtained.

[0228] described The acquisition process is:

[0229] According to the installation position and angle of the thermal imaging camera, the relative position of the camera relative to the unmanned vehicle can be obtained

[0230] Given any Z value and pixel coordinates [u,v] T , you can calculate the coordinates in the world coordinate system. The calculation formula is as follows:

[0231]

[0232] The other steps and parameters are the same as those in the specific implementation modes 1 to 9-1.

[0233] The following examples are used to verify the beneficial effects of the present invention:

[0234] Example 1:

[0235] This embodiment is an air-ground collaborative mapping method.

[0236] The unmanned vehicle is equipped with a central controller, router, unmanned vehicle host computer, laser radar, thermal imaging camera, and monocular camera; the drone is equipped with a drone sub-end host computer, laser radar, thermal imaging camera, and monocular camera. The overall system adopts a distributed structure. The drone and unmanned vehicle receive sensor information and transmit it to the central control terminal via wireless network. The central control terminal processes the data and implements a series of functions such as mapping, planning, and life information detection. It also transmits control instructions to the drone and unmanned vehicle via wireless network. The overall system structure is as follows: Figure 1 shown.

[0237] The drone and the autonomous vehicle are initially initialized at the same location. The collaboratively constructed map is based on the autonomous vehicle's mapping coordinate system. Whenever the autonomous vehicle receives a new laser point cloud, it extracts feature points, matches frames between them, and matches frames with the local map. Finally, it updates the accumulated error and maps the current frame's point cloud to the autonomous vehicle's global map.

[0238] For drones, the steps for establishing maps are similar to those for unmanned vehicles. However, before NDT matching is successfully performed, the local map must be matched with the local map of the unmanned vehicle after each frame of point cloud is processed. If the match is successful, the global maps of the two are fused. The maps subsequently established by the drone and the unmanned vehicle are supplemented based on this fused map.

[0239] Once the unmanned vehicle and drone maps are matched and integrated, their poses are calculated in the same coordinate system. This allows for accurate positional information and the 3D point cloud map created from the start of the operation to the current state, enabling subsequent active exploration and the integration of life detection information. Once NDT matching is initialized, subsequent exploration tasks, including exploration, can begin.

[0240] Example 2:

[0241] This implementation is an active exploration method based on boundaries.

[0242] After obtaining the fused point cloud map in the central controller, it is first converted into an octree map. Unoccupied voxels with nearby voxels of unknown occupancy are defined as boundaries. All boundary voxels in the octree map are found. Through clustering and thresholding, multiple large-scale boundaries are obtained, and their respective centroids are selected as candidate points for exploration. To evaluate the effectiveness of reaching each candidate point and obtain the most information in the shortest time, it is necessary to obtain the time from the current position to the candidate point and the map entropy obtained at the candidate point.

[0243] Due to the high maneuverability of drones, the mission objectives of the drones are calculated first. For the calculation of time, it is first necessary to obtain the path from the current position to the candidate position. This part is implemented using the A* algorithm. Assuming that the drone moves at a constant speed vave, the required time can be obtained by calculating the quotient of the total length of the path and vave. For the calculation of map entropy, at each candidate point, the sum of the information entropy of the space passed by all hormone beams in one frame is calculated as the map entropy of this point. After obtaining the map entropy and the expected time consumption, the utility of reaching each candidate point can be obtained according to the utility equation, and the candidate point with the highest utility is selected as the target point of the drone.

[0244] To prevent the autonomous vehicle and drone from exploring the same area, the selected candidate points are eliminated, removing those near the drone's path. The above steps are repeated, and the A* algorithm is used to calculate the path and time required to reach the eliminated candidate points. Since the candidate points are a subset of the original candidate points, there is no need to repeatedly calculate the map entropy of each point. Instead, the utility is directly calculated, and the largest target is selected as the next target for the autonomous vehicle.

[0245] The central control terminal sends control instructions to the host computer of the unmanned vehicle and drone, and sends them to the underlying controller via the serial port to control the unmanned vehicle and drone to move toward the selected target point.

[0246] Example 3:

[0247] This embodiment is a mapping method for infrared thermal imaging detection.

[0248] The image of the infrared thermal imaging sensor is analyzed, and the area close to the human body temperature is extracted. Clustering and threshold processing are performed, and the center of gravity of each area is obtained.

[0249] Its center of gravity is mapped from a two-dimensional plane to a global map according to the single-aperture camera model. Due to the lack of depth data, its depth Z in the camera coordinate system is gradually increased according to the step size s until the maximum detection range is reached or an obstacle is encountered.

[0250] Once mapped onto the global map, if there are obstacles nearby, the point cloud near that point is marked in the point cloud map, effectively marking the location of potential life signs on the 3D map. This location, along with the thermal imaging and monocular camera image data at that moment, is also recorded, providing more detailed and targeted information for subsequent searches.

[0251] The present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A collaborative air-ground life detection and mapping method for earthquake relief scenarios, characterized by: The specific process of the method is: Step 1: Create a 3D point cloud map of the unmanned vehicle; Step 2: Create a 3D point cloud map of the drone; Step 3: Use the NDT algorithm to match the 3D point cloud map of the unmanned vehicle and the 3D point cloud map of the drone to obtain the relative pose A of the 3D point cloud map of the unmanned vehicle and the 3D point cloud map of the drone; Step 4: Based on the relative pose A of the unmanned vehicle's 3D point cloud map and the drone's 3D point cloud map, the unmanned vehicle's 3D point cloud map and the drone's 3D point cloud map are merged with the unmanned vehicle's 3D point cloud map, using the unmanned vehicle's mapping origin as the origin of the coordinate system. Step 5: Update the UAV's pose Tbias×A based on the relative pose A of the 3D point cloud map of the unmanned vehicle and the 3D point cloud map of the UAV, so that the UAV maps the newly received point cloud into the merged map. Step 6: Repeat steps 1 and 2 to continuously expand and merge the map, and execute steps 7 and 14 at the same time; Step 7: Convert the merged map expanded in step 6 into an octree map, find the octree map boundary, obtain candidate points through clustering, threshold processing, and finding the center of gravity, and the candidate points form a candidate location list F; calculate the utility of the candidate points and select the candidate point with the highest utility as the target point of the UAV; The path from the current position of the UAV to the candidate point with the highest utility is used as the planned path of the UAV; Step 8: Remove the candidate locations F with the target point of the drone as the center and a radius of R. obs Candidate points of the unmanned vehicle are obtained, and a list of candidate points for the unmanned vehicle is obtained. Using A * The algorithm calculates the current position of the unmanned vehicle Middle The path of candidate points in, For path The first point; For path Middle points; The length of the path from the current position of the unmanned vehicle to each candidate point is recorded as Step 9: Assume the average speed of the drone is Calculate the estimated time of the drone's path from its current position to each candidate point Step 10: Current location of the unmanned vehicle The path of each candidate point in Calculate the utility and select the candidate point with the highest utility as the target point of the unmanned vehicle; The path from the current position of the autonomous vehicle to the candidate point with the highest utility is used as the planned path of the autonomous vehicle; Step 11: Control the unmanned vehicle and the drone to move toward the selected target point; Step 12: Determine whether the UAV and the unmanned vehicle have reached the target point. If so, proceed to step 13; if not, proceed to step 11. Step 13: Determine whether the exploration is completed. If so, end; if not, proceed to step 6; Step 14: Perform infrared thermal imaging detection of unmanned vehicles and drones on the merged map.

2. The air-ground collaborative life detection and mapping method for earthquake relief scenarios according to claim 1 is characterized by: In step 1, a 3D point cloud map of the unmanned vehicle is established; the specific process is as follows: Step 1: Calculate the curvature c of each point in the laser point cloud based on the laser point cloud data received by the unmanned vehicle's laser radar, and obtain the unmanned vehicle laser point cloud P based on the curvature. UGV Plane points and unmanned vehicle laser point cloud P UGV edge points; Step 1 and 2: For the unmanned vehicle laser point cloud P UGV The edge point of the frame is extracted, and the two edge points closest to the edge point in the previous frame are extracted. The distance d from the edge point to the straight line formed by the two closest edge points is calculated. ε i, i=1,2,…,m, m is the number of edge points; For unmanned vehicle laser point cloud P UGV The plane point is extracted, and the three non-collinear plane points closest to the plane point in the previous frame are extracted. The distance from the plane point to the plane formed by the three non-collinear plane points closest to the plane point is calculated. n is the number of plane points; The calculation formula is as follows: Among them, d εi is the distance from the ith edge point to the straight line formed by the two nearest edge points, X (k+1,i) is the i-th edge point of the k+1 frame, X (k,j) and X (k,l) is the value of X in the kth frame (k+1,i) The two closest edge points; in, For the The distance from a plane point to the plane formed by the three non-collinear plane points closest to the plane point, is the k+1th frame plane points, and is the kth frame with The three closest non-collinear plane points; represents the dot product; Step 13: Set the relative pose transformation T between each two frames scan The initial value is Step 14: d εi and Perform nonlinear optimization to solve all d εi and The sum of the distances is minimized, and the relative pose transformation T between each two frames is obtained. scan ; The expression is: Among them, m is the number of edge points, n is the number of plane points; Take f(T scan ) is minimum when T scan is the relative pose transformation T between each two frames scan ; Step 15: Set the initial relative position of the lidar on the unmanned vehicle relative to the origin of the mapping world coordinate system Step 16: Based on the initial relative pose and the relative pose transformation T between each two frames scan , transform the rough relative pose of the current lidar on the unmanned vehicle relative to the origin of the mapping world coordinate system: Get the rough relative position of the current lidar on the unmanned vehicle relative to the origin of the mapping world coordinate system Step 17: Assume that the exact relative position of the current lidar on the unmanned vehicle relative to the origin of the mapping world coordinate system is Step 18: Map the current laser point cloud on the unmanned vehicle to the world coordinate system of the map, and obtain the current laser point cloud P′ on the unmanned vehicle in the world coordinate system of the map. UGV , the expression is: Among them, P UGV For laser point cloud, is the exact relative pose; represents the dot product; P′ UGV The current laser point cloud on the unmanned vehicle in the world coordinate system for mapping; Step 19: Map the current laser point cloud P′ on the unmanned vehicle to the world coordinate system UGV Match with the local map to obtain accurate relative pose The local map is: With the unmanned vehicle as the center, take a cube with a side length of d, and the point cloud map inside the cube is the local point cloud map P loc ; Step 10: Update Step 11: Transform the precise relative position of the current laser radar on the unmanned vehicle relative to the origin of the world coordinate system for mapping Map the current point cloud to the global map; Step 12: Repeat steps 11 to 11 to update the unmanned vehicle's three-dimensional point cloud map, thereby completing the map creation.

3. The air-ground collaborative life detection and mapping method for earthquake relief scenarios according to claim 2 is characterized by: In the step 11, the curvature c of each point in the laser point cloud is calculated based on the laser point cloud data received by the laser radar of the unmanned vehicle, and the laser point cloud P of the unmanned vehicle is obtained based on the curvature. UGV Plane points and unmanned vehicle laser point cloud P UGV The specific process is: When the laser radar of the unmanned vehicle receives the laser point cloud P UGV , calculate the laser point cloud P UGV The curvature c of each point in; Based on laser point cloud P UGV The curvature c of each point in the image is arranged from small to large, and the points corresponding to the curvatures of the top n positions are used as the laser point cloud P of the unmanned vehicle. UGV The plane point, Based on laser point cloud P UGV The curvature c of each point in the image is arranged from large to small, and the points corresponding to the curvatures in the top m positions are used as the laser point cloud P of the unmanned vehicle. UGV edge points; The calculation formula is as follows: Among them, S is the set of points selected on the left and right sides of the selected point in the row where the selected point is located; X i′ is the i′th point in the set S, i′∈S; X j′ is the j′th point in the set S, j′∈S.

4. The air-ground collaborative life detection and mapping method for earthquake relief scenarios according to claim 3 is characterized by: In step 19, the current laser point cloud P′ on the unmanned vehicle is mapped to the world coordinate system of the map. UGV Match with the local map to obtain accurate relative pose The local map is: With the unmanned vehicle as the center, take a cube with a side length of d, and the point cloud map inside the cube is the local point cloud map P loc ; The matching process is: Step 191: For the current laser point cloud P′ on the unmanned vehicle UGV Edge points of the local point cloud map P are extracted loc The two edge points closest to the edge point are used to calculate the distance d between the edge point and the straight line formed by the two closest edge points. εi ; For the current laser point cloud P′ on the unmanned vehicle UGV Plane points, extract local point cloud map P loc The three non-collinear plane points closest to the plane point are used to calculate the distance from the plane point to the plane formed by the three non-collinear plane points closest to the plane point. Step 192: Put the current laser point cloud P′ on the unmanned vehicle UGV Match with the local map to obtain accurate relative pose The process is: Among them, m is the number of edge points, n is the number of plane points; Take the envoy The minimum corresponding The precise relative position transformation of the current laser radar on the unmanned vehicle relative to the origin of the mapping world coordinate system 5. The method for air-ground collaborative life detection and mapping for earthquake relief and disaster relief according to claim 4 is characterized by: In step 2, a three-dimensional point cloud map of a drone is created; the specific process is as follows: Step 2. Calculate the curvature c of each point in the laser point cloud based on the laser point cloud data received by the UAV's laser radar, and obtain the UAV laser point cloud P based on the curvature. UAV Planar points and drone laser point cloud P UAV edge points; Step 2: For the UAV laser point cloud P UAV The edge point of the frame is extracted, and the two edge points closest to the edge point in the previous frame are extracted. The distance d from the edge point to the straight line formed by the two closest edge points is calculated. εi , i=1,2,…,m, m is the number of edge points; For UAV laser point cloud P UAV The plane point is extracted, and the three non-collinear plane points closest to the plane point in the previous frame are extracted. The distance from the plane point to the plane formed by the three non-collinear plane points closest to the plane point is calculated. n is the number of plane points; Step 2 and 3: Set the relative pose transformation T between each two frames scan The initial value is Step 24: d εi and Perform nonlinear optimization to solve all d ε i and The sum of the distances is minimized, and the relative pose transformation T between each two frames is obtained. scan ; Step 25: Set the initial relative position of the UAV's lidar relative to the origin of the mapping world coordinate system Step 26: Based on the initial relative pose and the relative pose transformation T between each two frames scan , transform the rough relative pose of the current lidar on the drone relative to the origin of the mapping world coordinate system: Get the rough relative pose of the current lidar on the drone relative to the origin of the mapping world coordinate system Step 27: Assume that the exact relative position of the current lidar on the drone relative to the origin of the mapping world coordinate system is Step 28: Map the current laser point cloud on the UAV to the world coordinate system of the map, and obtain the current laser point cloud P′ on the UAV in the world coordinate system of the map UAV , the expression is: Among them, P UAV For laser point cloud, is the exact relative pose; represents the dot product; P′ UAV The current laser point cloud on the UAV in the world coordinate system of the map; Step 29: Map the current laser point cloud P′ on the drone to the world coordinate system UAV Match with the local map to obtain accurate relative pose The local map is: With the drone as the center, take a cube with a side length of d, and the point cloud map inside the cube is the local point cloud map P loc ; Step 20: Update Step 21: Transform the precise relative pose of the current frame lidar on the drone relative to the origin of the world coordinate system Map the current point cloud to the global map; Step 22: Repeat steps 21 to 21 to update the drone's three-dimensional point cloud map, thereby completing the map creation.

6. The method for air-ground collaborative life detection and mapping for earthquake relief and disaster relief according to claim 5 is characterized by: In step 7, the merged map expanded in step 6 is converted into an octree map, the octree map boundary is found, candidate points are obtained by clustering, threshold processing, and finding the center of gravity, and the candidate points constitute a candidate location list F; the utility of the candidate points is calculated, and the candidate point with the highest utility is selected as the target point of the UAV; The path from the current position of the UAV to the candidate point with the highest utility is used as the planned path of the UAV; The specific process is: Step 7.

1. Convert the merged map into an octree map; Unexplored areas are recorded as 0.5, occupied areas are recorded as 1, and explored empty areas are recorded as 0; The octree map consists of cubes, each representing a voxel block; Initially, the occupancy rate of each voxel block is set to 0.5; If there is one or more unexplored areas in the spatial 26 neighborhood of the blank area grid, the blank area is defined as a boundary area; Step 72: Randomly sample all boundary areas and perform clustering on the sampled boundary areas to obtain candidate points and form a list F of boundary candidate target points. The specific process is as follows: The adjacent boundaries are clustered into one category and the pre-set threshold T is used to h Remove the classes with smaller numbers and use the centroids of each remaining class as candidate points to form a list F of candidate boundary target points; Step 7.

3. Use A * The algorithm calculates the path R from the current position of the drone to the δth candidate point in F δ ; in, For path R δ The first point; For path R δ The jth point in The length of the path from the current position of the drone to each candidate point is recorded as L δ , for point with dot The Euclidean distance of For path R δ The kth point in For path R δ The k+1th point in the Step 7.4: Assume the average speed of the drone is v ave , calculate the estimated time T(R δ )=L δ / v ave ; Step 75: Calculate the path R from the current position of the drone to each candidate point in F δ Calculate the utility and select the candidate point with the highest utility as the target point of the UAV; The path from the current position of the UAV to the candidate point with the highest utility is used as the planned path of the UAV.

7. The method for air-ground collaborative life detection and mapping for earthquake relief and disaster relief according to claim 6 is characterized by: In step 75, the path R from the current position of the UAV to each candidate point in F is calculated. δ Calculate the utility and select the candidate point with the highest utility as the target point of the UAV; The path from the current position of the UAV to the candidate point with the highest utility is used as the planned path of the UAV; The specific process is: Step 751: Calculate the map entropy H(x) of each candidate point x; the specific process is: There are M hormone bundles at the candidate point x; Each hormone bundle passes through the voxel block in the octree map. When it reaches the voxel block with an occupation probability of 0.5, the map entropy of the voxel block with an occupation probability of 0.5 and the map entropy of each voxel block passed through are calculated, and the map entropy of the voxel block with an occupation probability of 0.5 and each voxel block passed through are added together as the map entropy of a hormone bundle; The occupancy of each voxel block is updated according to the following formula: Among them, P(n|z 1:t ) represents the occupancy probability of voxel block n from the 1st to the tth observation, z 1:t represents the first to t-th observation, n represents the n-th voxel block, P(n|z t ) represents the occupancy probability of voxel block n under the t-th observation, P(n|z 1:t-1 ) represents the occupancy probability of voxel block n from the 1st to the t-1th observation, z t represents the t-th observation, P(n) represents the occupancy probability of voxel block n; The map entropies of the M hormone bundles are summed up as the map entropy H(x) of each candidate point x; The calculation formula of the map entropy of each voxel block v is as follows: Η(v)-P O (v)ln(P O (v))-(1-P O (v))ln(1-P). O (v)) Where Η(v) is the map entropy of voxel block v, P O (v) is the occupancy probability of voxel block v; Step 752: Calculate the utility of each candidate point x. The calculation formula is as follows: Select the candidate point with the highest utility as the target point of the UAV; The path from the current position of the UAV to the target point of the UAV in F is used as the planned path of the UAV.

8. The air-ground collaborative life detection and mapping method for earthquake relief scenarios according to claim 7 is characterized by: In step 10, the current position of the unmanned vehicle is The path of each candidate point in Calculate the utility and select the candidate point with the highest utility as the target point of the unmanned vehicle; The path from the current position of the autonomous vehicle to the candidate point with the highest utility is used as the planned path of the autonomous vehicle; The specific process is: Step 10.

1. Calculate the map entropy H(x) of each candidate point x; the specific process is: There are M hormone bundles at the candidate point x; Each hormone bundle passes through the voxel block in the octree map. When it reaches the voxel block with an occupation probability of 0.5, the map entropy of the voxel block with an occupation probability of 0.5 and the map entropy of each voxel block passed through are calculated, and the map entropy of the voxel block with an occupation probability of 0.5 and each voxel block passed through are added together as the map entropy of a hormone bundle; The occupancy of each voxel block is updated according to the following formula: Among them, P(n|z 1:t ) represents the occupancy probability of voxel block n from the 1st to the tth observation, z 1:t represents the first to t-th observation, n represents the n-th voxel block, P(n|z t ) represents the occupancy probability of voxel block n under the t-th observation, P(n|z 1:t-1 ) represents the occupancy probability of voxel block n from the 1st to the t-1th observation, z t represents the t-th observation, P(n) represents the occupancy probability of voxel block n; The map entropies of M (16×3600) hormone bundles are summed up as the map entropy H(x) of a candidate point x; Step 10.2: Calculate the utility of each candidate point x. The calculation formula is as follows: Select the candidate point with the highest utility as the target point of the unmanned vehicle; The path from the current position of the unmanned vehicle to the candidate point with the highest utility is used as the planned path of the unmanned vehicle.

9. The method for air-ground collaborative life detection and mapping for earthquake relief and disaster relief according to claim 8 is characterized by: In step 14, infrared thermal imaging detection of unmanned vehicles and drones is performed on the combined map; the specific process is as follows: Step 14: Install infrared thermal imaging sensors on the drone and the unmanned vehicle, and receive thermal imaging information sent back by the infrared thermal imaging sensors; Step 142: Determine whether there is an area representing human body temperature in the thermal image information returned by the infrared thermal imaging sensor. If so, execute step 143; if not, execute step 141; The human body temperature is 30°C≤T≤38°C, where T is the human body temperature; Step 14: Binarize the thermal image of the area with human body temperature according to the set threshold, cluster it after binarization, and eliminate the class with small number of pixels according to the preset threshold, and calculate the centroid of each remaining class; Step 14: Select an unselected centroid from the centroids of each remaining class; Step 14: Set the depth value of the center of gravity selected in step 14:4 in the camera coordinate system to Z; Step 14: Convert the selected center of gravity from the camera coordinate system to the world coordinate system; Step 147: Determine whether there is a point cloud within a radius d around the selected centroid. If so, there is an obstacle and proceed to step 148. If there is no point cloud, set Z = Z + s, where s is the step size, and proceed to steps 14 and 15. Step 148: Mark the point cloud within the range of the selected centroid radius d on the merged map, and record the thermal image information at this time; Step 149: Determine whether all centers of gravity have been traversed. If so, execute step 140; if not, execute step 144; Step 1410: Determine whether the exploration is completed. If so, end; if not, execute step 141.

10. The method for air-ground collaborative life detection and mapping in earthquake relief scenarios according to claim 9 is characterized by: In step 146, the selected center of gravity is converted from the camera coordinate system to the world coordinate system; the specific process is: Pixel coordinates [u,v] T The coordinates [X, Y, Z] in the camera coordinate system T Relationship: where f x 、f y 、c x and c y is the camera internal parameter, the unit is pixel; Z is the Z axis in the camera coordinate system, that is, the depth value of the center of gravity in the camera coordinate system is Z; P uv It is the homogeneous coordinate form in the pixel coordinate system; It is defined as K is the camera intrinsic parameter matrix and P is the coordinate in the camera coordinate system; The superscript T means to find the transpose; Since [X,Y,Z] T In the camera coordinate system, the camera coordinate system needs to be converted to the world coordinate system. The calculation formula is: in is the homogeneous transformation matrix of the camera coordinate system relative to the world coordinate system, P W is the coordinate of the target point in the world coordinate system; The homogeneous transformation matrix of the camera coordinate system relative to the world coordinate system is the relative position of the unmanned vehicle relative to the origin W of the mapping world coordinate system, is the relative position of the camera relative to the unmanned vehicle; The unmanned vehicle is a drone or an unmanned vehicle; Given any Z value and pixel coordinates [u,v] T , calculate the coordinates in the world coordinate system, the calculation formula is as follows:

Images