Mountain three-dimensional map construction method based on unmanned aerial vehicle cluster

Abstract

The invention discloses a mountain three-dimensional map construction method based on an unmanned aerial vehicle cluster, and the method comprises the following steps: 1, a rear-end main control end carries out the division of a flight region of the unmanned aerial vehicle cluster through a Voronoi graph segmentation method, and then completes the path planning of the unmanned aerial vehicle cluster through a particle swarm optimization algorithm; step 2, the unmanned aerial vehicle cluster collects images in a designated area and estimates an initial pose of an image frame; 3, the unmanned aerial vehicle cluster performs three-dimensional map construction on the local area by using the micro-host in the flight process; and step 4, the back-end master control end carries out splicing fusion on the returned local three-dimensional map through the ORB-SLAM3 algorithm, and construction of a global three-dimensional map is completed. The unmanned aerial vehicle cluster transmits the constructed local three-dimensional map back to the rear-end master control terminal, and the rear-end master control terminal performs data fusion and global map splicing, so that efficient and real-time mountain environment three-dimensional map construction is realized, and the requirements of various applications such as ecological detection, resource management and environment planning are met.

Description

Technical Field

[0001] The invention belongs to the field of geographic environment information technology, and in particular relates to a method for constructing a mountain three-dimensional map based on an unmanned aerial vehicle cluster. Background Art

[0002] In the process of 3D modeling of mountainous environments, rapid perception and real-time updating of terrain information are the key to accurate modeling. Mountainous terrain often has complex topography and abundant obstacle distribution, which requires technicians to obtain detailed information of the mountainous environment in real time in order to carry out scientific and reasonable modeling. Traditional terrain acquisition methods, such as manual measurement and fixed photography equipment, cannot meet the needs of the rapidly changing mountainous environment. Whether it is ecological detection, land planning or environmental assessment, real-time 3D maps are an important basis for making effective analysis and decisions.

[0003] Currently, there are significant limitations in constructing three-dimensional maps using a single drone. The coverage of a single drone is limited by the flight altitude, endurance time, and transmission distance, making it difficult to obtain a wide range of mountain environment information in a short period of time. The data processing capabilities of a single drone are limited, especially in high-load mountain environments, where processing large-scale image data and complex pose calculations requires high-performance computing resources. In addition, the computing power of handheld terminals is generally limited, and a single handheld terminal is difficult to process a large range of geographic information data and quickly build a high-precision three-dimensional map, which further limits the application effect of a single drone; at the same time, although the back-end system can continuously optimize local maps, in the long process of mapping, due to the inaccuracy of position estimation and the time accumulation effect, it often leads to significant errors and drift, which in turn makes the constructed local three-dimensional map inconsistent with the actual scene. Summary of the invention

[0004] The present invention aims to provide a method for constructing a three-dimensional mountain map based on a swarm of unmanned aerial vehicles, thereby achieving efficient and real-time three-dimensional map construction of a mountain environment and improving the accuracy, authenticity and reliability of local three-dimensional map construction.

[0005] In order to achieve the purpose of the present invention, the present invention provides a method for constructing a mountain three-dimensional map based on a drone cluster, comprising the following steps:

[0006] Step 1: The back-end main control end divides the flight area of ​​the drone cluster by the Voronoi diagram segmentation method, and then uses the particle swarm optimization algorithm to complete the path planning of the drone cluster;

[0007] Step 2: The drone cluster collects images in a specified area and estimates the initial pose of the image frame;

[0008] Step 3: The drone cluster uses the micro-host to construct a three-dimensional map of the local area during flight;

[0009] Step 4: The back-end master control terminal uses the ORB-SLAM3 algorithm to stitch and fuse the returned local three-dimensional maps to complete the construction of the global three-dimensional map.

[0010] Compared with the prior art, the significant improvements of the present invention are: (1) The present invention uses a loop detection mechanism to accurately re-identify the traversed scenes, significantly improving the accuracy of local three-dimensional map construction, preventing cumulative errors and drift, and thus ensuring the authenticity and reliability of the mapping results; (2) The present invention uses an intelligent algorithm to allocate flight areas to drone clusters, so that each drone can focus on information collection and processing in a specific area; (3) The distributed architecture of the drone cluster of the present invention can not only greatly improve the coverage range, but also share data processing tasks through distributed computing, thereby solving the shortcomings of a single drone in data processing capabilities.

[0011] In order to more clearly illustrate the functional characteristics and structural parameters of the present invention, further description is given below in conjunction with the accompanying drawings and specific implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0013] Figure 1 is a flow chart of the steps of the present invention;

[0014] Figure 2 It is a schematic diagram of flight area allocation of the drone cluster of the present invention;

[0015] Figure 3 It is a flowchart of constructing a local three-dimensional map of a drone cluster of the present invention;

[0016] Figure 4 It is a flow chart of the back-end master control terminal splicing the global three-dimensional map of the present invention;

[0017] Figure 5 It is a schematic diagram of a three-dimensional map constructed by the present invention. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0019] The present invention provides a method for constructing a mountain three-dimensional map based on a drone cluster, combined with Figure 1 , including the following steps:

[0020] Step 1: The back-end main control end divides the flight area of ​​the drone cluster by the Voronoi diagram segmentation method, and then uses the particle swarm optimization algorithm to complete the path planning of the drone cluster;

[0021] 1-1. The back-end main control end divides the flight area of ​​the drone cluster by the Voronoi diagram segmentation method: the drone cluster is dispersed in the mountain area, and the mountain area is set as a rectangular area. The Voronoi diagram segmentation method is used to divide the operation area. Figure 2 , is a Voronoi diagram formed by 6 points. The allocation principle is that the operating area of ​​each drone is as evenly distributed as possible;

[0022] The Voronoi diagram, also known as Thiessen polygons, is a mathematical structure used to divide a plane. A set of discrete points is given to it, which are called generating points or seed points. The Voronoi diagram divides the plane into several regions. The definition of each region is that the Euclidean distance of all points in the region to the corresponding generating point is closer than the distance to any other generating point. These regions are called Voronoi regions or Voronoi cells. On the plane, the boundary of the Voronoi diagram is composed of the perpendicular bisectors between the generating points. The boundary constitutes the shape of the Voronoi diagram region, making each region a possible convex polygon.

[0023] The Voronoi diagram is constructed from any position in the region, and the results obtained are consistent, and the complexity in terms of calculation time is low, and it is only related to the Voronoi edge; the present invention aims at the problem of collaborative image information collection of multiple drones, and uses the seed point in the Voronoi diagram as the starting point of each drone, and the Voronoi polygon where it is located is the divided task area, and each drone is responsible for an area to perform full coverage image collection on the target area from the starting point.

[0024] 1-2. The back-end main control end plans the flight path of the drone cluster using a particle swarm optimization algorithm: after dividing the mountain area into multiple sub-areas, the particle swarm optimization algorithm is used to plan the flight path of the drone cluster to obtain the shortest flight distance;

[0025] The drone cluster carries a monocular camera and a micro host during flight, and the total flight range is limited. Then:

[0026] l=l g +l w+l c ;

[0027] l<l max ;

[0028] Among them, l is the total flight range of a single UAV; l g is the distance of a single UAV from the starting point to the starting point of the image acquisition mission; l w is the range of a single UAV for image acquisition mission; l c is the return flight distance after a single UAV completes its mission, l max The sailing distance is the maximum range limit.

[0029] The Particle Swarm Optimization (PSO) algorithm is an optimization algorithm that simulates the behavior of natural groups. It is inspired by the collective movement and information sharing of groups such as bird flocks and fish schools in the process of searching for food. It finds the global optimal solution to the optimization problem by iteratively moving a set of candidate solutions, called particles, in the search space. Each particle represents a possible solution in the problem space and has a speed and position; the particle gradually updates its position by following the information of the optimal particle, thereby seeking the optimal solution in the problem space.

[0030] The particle swarm optimization algorithm comprises the following steps:

[0031] 1-2-1. The particle has an initial randomly generated position and velocity in the solution space; the individual best position (pBest) of the particle is initialized to its current position, and the global best position (gBest) is usually set to the position with the best performance among the initial positions of all particles.

[0032] 1-2-2. Update the speed of the particle:

[0033] v i (t+1)=w·v i (t)+c1·r1·(p i -x i (t))+c2·r2·(gx i (t));

[0034] Where w is the inertia weight, c1 and c2 are acceleration constants, r1 and r2 are random numbers, and p i is the individual optimal position of the particle, g is the global optimal position, and v i is the velocity of particle i, x i is the position of particle i, t is the time;

[0035] 1-2-3. Update the position of the particle:

[0036] xi (t+1)=x i (t)+v i (t+1);

[0037] 1-2-4. Update the best position: Check whether the current solution of each particle is better than its individual best position. If so, update the individual best position; at the same time, update the global best position.

[0038] 1-2-5. Repeat steps 1-2-2 to 1-2-4 until a termination condition is reached, wherein the termination condition is a maximum number of iterations or a satisfactory error range. In the present invention, the maximum number of iterations is generally between 100 and 500 times, and the error threshold range is generally set between 1% and 5% of the global optimal value.

[0039] Step 2: The drone cluster collects images in a specified area and estimates the initial pose of the image frame;

[0040] 2-1. The drone cluster completes image acquisition through a monocular camera: according to the area divided by the back-end main control end to each drone in the drone cluster in step 1-1, the drone cluster is responsible for image acquisition and posture estimation; in the present invention, a monocular camera is used as the visual sensor of the drone. The monocular camera has been widely used in many visual sensors due to its small size, economical cost and simple calibration process. The visual sensor can effectively convert three-dimensional coordinate points into two-dimensional coordinates to meet practical application needs.

[0041] 2-2. The micro-host of the drone cluster uses the ORB-SLAM3 algorithm to complete image feature point extraction and matching as well as image frame pose estimation;

[0042] The collected image information will be used to estimate the pose between adjacent image frames through visual odometer technology, thereby achieving accurate positioning and map construction;

[0043] The ORB-SLAM3 algorithm is used to complete feature extraction and matching: the micro-host of the drone cluster grayscales the input image, and uses the FAST algorithm to check the corner points in the detection image. The corner points determine their directions by determining the intensity difference of the pixels around them, and then generate ORB descriptors; the matching process is to compare the feature point descriptors in the new image frame with the existing frame, use the Hamming distance to measure the similarity, and screen the matching pairs through cross-checking and ratio measurement to reduce matching errors, and then use the RANSAC algorithm to eliminate outliers to ensure that the matching point pairs can accurately reflect the correspondence between images.

[0044] The initial pose estimation is determined by the feature points, and then the geometric relationship between the matching point pairs is determined by the ORB-SLAM3 algorithm to estimate the pose change of the camera movement.

[0045] The monocular camera uses the ORB-SLAM3 algorithm to determine the geometric relationship between the matching point pairs to estimate the pose change of the camera movement, starting from the same-name points observed from two perspectives, and then recovering the spatial positions of the same-name points through the principle of triangulation, and determining the position and orientation of the monocular camera in three-dimensional space.

[0046] Step 3: Combine Figure 3 ,UAV swarms use micro-hosts to construct three-dimensional maps of local areas during flight;

[0047] 3-1. The drone cluster uses BA optimization to complete the update of posture and map points. After the drone cluster collects new image frames using a monocular camera as a visual sensor, it determines the key frame based on the number of feature point matches and posture changes, and the system performs local RA (Bundle Adjustment) optimization on it. The RA optimization is an iterative process that aims to minimize the reprojection error, that is, optimize the camera posture and the three-dimensional coordinates of the map points so that the projection of the map points on the image is as close as possible to the actually observed feature points.

[0048] Triangulate the matching feature points between the key frames to estimate the three-dimensional position; add the triangulated map points to the local map for subsequent optimization, and then update the pose of the key frames and the coordinates of the map points through the BA optimization;

[0049] During the local map construction process, the system periodically performs global RA optimization at a regular interval of several seconds or minutes to optimize the key frames and the map points. During the optimization process, map points with poor quality will be screened out, such as points with few observations or large projection errors.

[0050] 3-2. The micro host of the drone cluster performs loop detection on the key frames to correct the accumulated errors, thereby improving the accuracy of constructing the three-dimensional map;

[0051] The process of loop detection involves analyzing the image information of the current key frame, aiming to determine whether the key frame is repeated with the image information previously collected. This detection mechanism is based on the principle of the visual word bag model, which converts the feature points in the image into "words" in the word bag, and generates a word vector related to the current key frame through rapid conversion. When the drone flies over a previously captured scene again, the system will use the word vector generated by the current key frame to index in the visual word bag library to find a matching key frame; when the matching key frame is found, other key frames that have a co-viewing relationship with this key frame are further searched to form a candidate key frame group. By comparing with these candidate key frames, the information is retrieved to obtain several overlapping points, thereby confirming the success of the loop detection.

[0052] Step 4: The back-end main control end uses the ORB-SLAM3 algorithm to stitch and fuse the returned local three-dimensional maps to form a global three-dimensional map.

[0053] Constructing a rigid body motion model of the drone: Determine the relative position relationship between the drones through rotation and translation transformation:

[0054] u2=(R2,t2)u1;

[0055] u3=(R3,t3)u1;

[0056] Among them, ui is the coordinate system of rigid body i, i=1, 2, 3, Ri and ti are rotation and translation transformations respectively.

[0057] Combination Figure 4 , the drone cluster is positioned and mapped in real time, and the communication module of the drone cluster sends the key frame and the local map data to the back-end master control end; the back-end master control end uses the loop detection module and the visual word bag (DBOW) method to complete the matching of the existing image frame with the current key frame features; when the scenes overlap, the coordinates of the two local maps are converted, and the different local maps are fused using the SIM3 transformation, and then the map drift caused by the cumulative error is eliminated through the global RA optimization, and finally an enhanced global three-dimensional map is formed.

[0058] Combination Figure 5 , a schematic diagram of the data collected by the drone cluster and the generation of a three-dimensional point cloud map by the ORB-SLAM3 algorithm at the back-end main control end. The map shows the outline of the map in the form of a dense dot matrix.

[0059] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0060] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for constructing a mountain three-dimensional map based on a drone cluster, characterized in that: The following steps are involved: Step 1: The back-end main control end divides the flight area of ​​the drone cluster by the Voronoi diagram segmentation method, and then uses the particle swarm optimization algorithm to complete the path planning of the drone cluster; Step 2: The drone cluster collects images in a specified area and estimates the initial pose of the image frame; Step 3: The drone cluster uses the micro-host to construct a three-dimensional map of the local area during flight; Step 4: The back-end master control terminal uses the ORB-SLAM3 algorithm to stitch and fuse the returned local three-dimensional maps to complete the construction of the global three-dimensional map.

2. The method for constructing a mountain three-dimensional map based on a drone cluster according to claim 1, characterized in that: The step 1 is specifically as follows: 1-1. The back-end main control end divides the flight area of ​​the drone cluster by the Voronoi diagram segmentation method: the drone cluster is dispersed in the mountain area, and the mountain area is set as a rectangular area, and the Voronoi diagram segmentation method is used to divide the operation area; The seed point in the Voronoi diagram is used as the starting point of the drone cluster, and the Voronoi polygon where it is located is the divided task area. The drone cluster is responsible for a region to collect full coverage images of the target area from the starting point. 1-2. The back-end main control end plans the flight path of the drone cluster using a particle swarm optimization algorithm: after dividing the mountain area into multiple sub-areas, the particle swarm optimization algorithm is used to plan the flight path of the drone cluster to obtain the shortest flight range.

3. The method for constructing a mountain three-dimensional map based on a drone cluster according to claim 2, characterized in that: The particle swarm optimization algorithm of step 1-2 comprises the following steps: 1-2-1. The particle has an initial randomly generated position and velocity in the solution space; 1-2-2. Update the speed of the particle: v i (t+1)=w·v i (t)+c1·r1·(p i -x i (t))+c2·r2·(g-x i (t)); Where w is the inertia weight, c1 and c2 are acceleration constants, r1 and r2 are random numbers, and p i is the individual optimal position of the particle, g is the global optimal position, and v i is the velocity of particle i, x i is the position of particle i, t is the time; 1-2-3. Update the position of the particle: x i (t+1)=x i (t)+v i (t+1); 1-2-4. Update the best position: Check whether the current solution of each particle is better than its individual best position. If so, update the individual best position; at the same time, update the global best position. 1-2-5. Repeat steps 1-2-2 to 1-2-4 until the termination condition is reached.

4. The method for constructing a mountain three-dimensional map based on a drone cluster according to claim 1, characterized in that: The step 2 is specifically as follows: 2-1. The drone cluster completes image acquisition through a monocular camera: According to the area divided by the back-end main control end for each drone in the drone cluster in step 1-1, the drone cluster is responsible for image acquisition and pose estimation; 2-2. The micro host of the drone cluster uses the ORB-SLAM3 algorithm to complete image feature point extraction and matching as well as image frame pose estimation.

5. The method for constructing a mountain three-dimensional map based on a drone cluster according to claim 1, characterized in that: The step 3 is specifically as follows: 3-1. The drone cluster uses BA optimization to complete the update of posture and map points. After the drone cluster collects new image frames through a monocular camera as a visual sensor, it will select key frames based on feature point matching and posture estimation, and the system will perform local RA optimization on them. During the construction of the local map, the system performs loop detection on the key frames to identify the scenes that the camera has visited before, eliminate the accumulated positioning error, and then regularly performs global BA optimization to optimize and update the key frames and the map points; 3-2. The drone cluster performs loop detection on the key frames to correct the accumulated errors.

6. The method for constructing a mountain three-dimensional map based on a drone cluster according to claim 1, characterized in that: The step 4 is specifically as follows: The drone cluster is positioned and mapped in real time, and the communication module of the drone cluster sends the key frame and the local map data to the back-end main control end; the back-end main control end uses the loop detection module and the visual word bag method to complete the matching of the existing image frame with the current key frame features; when the scenes overlap, the coordinates of the two local maps are converted, the coordinate conversion is performed using the SIM3 transformation, the different local maps are fused, and finally the global RA optimization is used to complete the construction of the global three-dimensional map.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

8. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute any one of claims 1 to 6.

9. A computer program product comprising computer program instructions, characterized in that When the computer program instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

Images