A method and system for exploring a UAV based on a LiDAR sensor

Abstract

The application discloses a kind of unmanned plane exploration method and system based on LiDAR sensor, comprising: obtaining single frame point cloud data;Single frame point cloud data is registered into global point cloud map by SLAM module;Point cloud map and three-dimensional occupancy grid map are obtained by SLAM module to carry out point cloud density analysis and boundary detection;Wherein, point cloud map is used to carry out point cloud density analysis;Three-dimensional occupancy grid map is used to carry out boundary detection;The result of point cloud density analysis and boundary detection is included in information gain to guide the generation of candidate viewpoint;Global path planning is carried out according to the generated candidate viewpoint, and local path is determined;Local path is optimized, and a time-constrained smooth trajectory is generated to guide the movement of unmanned plane.The real-time of the embodiment of the application is high, can generate efficient global path and safe, completely covered local observation path, and can be widely applied in the field of unmanned plane control technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle control, and particularly relates to an unmanned aerial vehicle exploration method and system based on a LiDAR sensor. BACKGROUND

[0002] Unmanned aerial vehicles, especially quadrotors, have attracted great attention and been widely used in various fields such as terrain mapping, structure inspection and three-dimensional modeling due to their flexibility and high maneuverability. An unmanned aerial vehicle acquires a large amount of scene observation data by using sensors such as cameras and laser radars, and a high-resolution, continuous three-dimensional map can be created by fusing and registering the observation data to represent the geometric structure and surface features of the scene. These applications use high-quality maps to obtain real and detailed environmental information of the scene, and therefore it is necessary to develop a key technology capable of quickly and autonomously exploring unknown environments and generating high-quality maps.

[0003] In recent years, as autonomous exploration tasks have attracted more and more attention, many autonomous exploration methods have emerged, such as sampling-based exploration and boundary-based exploration methods. The sampling-based method generates a series of candidate viewpoints in the known space by a method similar to the rapid exploration random tree (RRT), and selects the next best viewpoint by evaluating the expected information gain (EIG), and the unmanned aerial vehicle flies from the current position to the next best viewpoint position for observation. The boundary-based method first generates several boundary clusters, acquires a viewpoint for each boundary cluster, and determines the next best viewpoint with the maximum expected information gain, and guides the unmanned aerial vehicle to perform observation. However, most methods focus on efficiently covering unknown areas (using a cubic grid to represent the space), and ignore the quality of the scanned map, and do not fully consider surface information acquisition. This rough method of representing the space environment is not satisfactory for many complex practical applications.

[0004] In the existing autonomous exploration method of the unmanned aerial vehicle, the following problems exist:

[0005] Most existing methods focus on efficiently covering unknown areas, and only represent the space environment in a rough way (grid), which can cause the explored map to have sparse areas and hollow areas.

[0006] Existing planners plan exploration paths in a greedy way, such as maximizing the immediate information gain or navigating to the nearest unknown area. This greedy strategy ignores global optimality, thus resulting in low overall efficiency. In addition, most methods generate quite conservative movements in previously unknown environments to ensure information acquisition and safety at the same time.

[0007] Finally, many methods have high computational overhead and cannot quickly and frequently respond to environmental changes, resulting in inefficient exploration when new environmental information appears. SUMMARY

[0008] The main purpose of the embodiment of the present application is to provide a real-time and efficient unmanned aerial vehicle exploration method and system based on a LiDAR sensor, which can generate an efficient global path and a safe and completely covered local observation path.

[0009] To achieve the above-mentioned purpose, one aspect of the embodiment of the present application provides an unmanned aerial vehicle exploration method based on a LiDAR sensor, comprising the following steps:

[0010] The unmanned aerial vehicle acquires single-frame point cloud data by using an airborne laser radar sensor;

[0011] The acquired single-frame point cloud data is registered into a global point cloud map by a SLAM module;

[0012] According to the global point cloud map, a point cloud map and a three-dimensional occupancy grid map are obtained by a SLAM module for point cloud density analysis and boundary detection; wherein the point cloud map is used for point cloud density analysis; and the three-dimensional occupancy grid map is used for boundary detection;

[0013] The results of the point cloud density analysis and the boundary detection are included in the information gain to guide the generation of candidate viewpoints;

[0014] Global path planning is performed according to the generated candidate viewpoints, and a local path is determined;

[0015] The local path is optimized, and a time-constrained smooth trajectory is generated to guide the movement of the unmanned aerial vehicle.

[0016] In some embodiments, the process of point cloud density analysis comprises:

[0017] The density of each point cloud is calculated according to the average distance between each point cloud and a plurality of nearest neighbor points, wherein the calculation formula of the point cloud density is:

[0018]

[0019] Wherein, K represents the number of nearest neighbor points; p i represents the i-th point; τ p represents the threshold value of the point cloud density;

[0020] A voxel representation structure is created; the voxel representation structure comprises a point cloud layer, a high-resolution voxel layer, and a grid layer;

[0021] In the point cloud layer, the eligibility of the point cloud in the visible area captured by the viewpoint is evaluated according to the calculation formula of the point cloud density;

[0022] In the high-resolution voxel layer, the discrete point cloud is divided into continuous voxels according to their positions, and the eligibility of the voxel is determined by the ratio of the number of eligible points to the number of ineligible points in the voxel, and if the ratio exceeds a threshold value, the voxel is considered eligible;

[0023] In the grid layer, the voxels are placed into larger grids, and if the ratio of the number of eligible voxels to the number of ineligible voxels contained in the grid exceeds a certain threshold value, the grid is considered eligible.

[0024] In some embodiments, the results of the point cloud density analysis and the boundary detection are included in the information gain to guide the generation of candidate viewpoints, including:

[0025] A set of candidate points is uniformly sampled from the local planning range, and the points on the boundary of the local planning range are defined as boundary points;

[0026] According to the sampled candidate points, candidate viewpoints are generated to completely cover the target area of the ineligible point cloud and the boundary;

[0027] For each point of the candidate points, a plurality of yaw angles are determined as the maximum coverage range of a plurality of independent sensors to the target area, and the information gain of each yaw angle that meets the sensor model and is not blocked by obstacles is evaluated, and then the candidate viewpoints are generated.

[0028] In some embodiments, the calculation formula of the information gain is:

[0029]

[0030] where Gain v represents the information gain of mpoint; S v represents the number of ineligible grids observed from mpoint; F v represents the number of boundaries observed from mpoint; g max represents the number of ineligible voxels in each s v ; λ f is the weight coefficient of F v ; represents the number of ineligible voxels in each s i .

[0031] In some embodiments, the determination of the local path includes:

[0032] The total gain in the local planning range is calculated by calculating the sum of the target areas of all candidate viewpoints;

[0033] Sort the information gain of all mpoint, and randomly select one from the top n mpoint to add to the selected viewpoint set;

[0034] Recalculate the information gain of the candidate viewpoint, add the current position coordinates of the UAV and the coordinates of the boundary point to the viewpoint set in advance, and return to execute the step of sorting the information gain of all mpoint, and randomly selecting one from the top n mpoint to add to the selected viewpoint set, until a set of viewpoint subset capable of covering the total gain is obtained;

[0035] According to the solution strategy of the traveling salesman problem, an open loop path starting from the current position of the UAV and passing through all other points in the viewpoint set is generated to obtain a rough local path.

[0036] In some embodiments, the optimization of the local path includes:

[0037] Uniformly sampling points from the target path segment of the rough local path;

[0038] Generating a range capable of covering the target area according to the sampling points;

[0039] Replacing one end point in the target path segment to obtain a new target path segment, and returning to execute the step of generating a range capable of covering the target area according to the sampling points until the optimization of each path segment in the local path is completed.

[0040] In some embodiments, the global path planning according to the generated candidate viewpoint includes:

[0041] Defining a global planning range, and dividing the space outside the local planning range into uniform cubic subspaces;

[0042] Formalizing the global path planning into a traveling salesman problem, and calculating a closed loop path starting from the current position of the UAV, a starting boundary point, the centroid of the subspace in the exploration state, and an ending boundary point;

[0043] Two boundary points are determined by constructing a distance matrix containing the shortest path length.

[0044] In some embodiments, the generation of a time-constrained smooth trajectory includes:

[0045] Using a uniform B-spline curve to smooth the rotation, and setting a time constraint term to control the rotation time of the rotation angle;

[0046] Where the rotation angle ξ, the transition time t b and the calculation formula of the time constraint term J t are respectively:

[0047]

[0048] J t = max {(N s -k) x At, t b}

[0049] wherein, denotes the position reached at time t i , and and are the maximum values of the constraints of velocity and yaw rate, respectively, N s and k are the number of control points and the dimension of the uniform B-spline curve, respectively, At denotes the time interval, and the distance and angle vectors of each observation point in the given observation path T H are defined as D and Y, respectively.

[0050] For a given observation path, a smooth trajectory subject to time constraints is generated using a uniform B-spline curve.

[0051] Another aspect of the embodiments of the present application also provides a LiDAR sensor-based unmanned aerial vehicle exploration system, comprising:

[0052] A first module for the unmanned aerial vehicle to acquire single-frame point cloud data using an on-board LiDAR sensor;

[0053] A second module for registering the acquired single-frame point cloud data into a global point cloud map through a SLAM module;

[0054] A third module for obtaining a point cloud map and a three-dimensional occupancy grid map through a SLAM module according to the global point cloud map, for point cloud density analysis and boundary detection; wherein the point cloud map is used for point cloud density analysis; and the three-dimensional occupancy grid map is used for boundary detection;

[0055] A fourth module for incorporating the results of the point cloud density analysis and the boundary detection into information gain to guide the generation of candidate viewpoints;

[0056] A fifth module for global path planning according to the generated candidate viewpoints and determining a local path;

[0057] A sixth module for optimizing the local path and generating a smooth trajectory subject to time constraints to guide the movement of the unmanned aerial vehicle.

[0058] To achieve the above object, another aspect of the embodiment of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method described above when executing the computer program.

[0059] To achieve the above object, another aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method described above.

[0060] The embodiment of the present application also discloses a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to make the computer device execute the method described above.

[0061] The embodiment of the present application at least has the following beneficial effects: the present application provides a UAV exploration method and system based on a LiDAR sensor, the UAV uses an airborne LiDAR sensor to obtain single-frame point cloud data; a SLAM module is used to register the obtained single-frame point cloud data into a global point cloud map; according to the global point cloud map, a point cloud map and a three-dimensional occupancy grid map are obtained by the SLAM module to perform point cloud density analysis and boundary detection; the point cloud map is used for point cloud density analysis; the three-dimensional occupancy grid map is used for boundary detection; the results of the point cloud density analysis and the boundary detection are included in information gain to guide the generation of candidate viewpoints; global path planning is performed according to the generated candidate viewpoints, and a local path is determined; the local path is optimized, and a time-constrained smooth trajectory is generated to guide the movement of the UAV. The embodiment of the present application has high real-time performance, can generate an efficient global path and a safe and completely covered local observation path. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 is an implementation environment schematic diagram provided by the embodiment of the present application;

[0063] Figure 2 is a flowchart of the overall steps provided by the embodiment of the present application;

[0064] Figure 3 is a framework schematic diagram of the implementation process provided by the embodiment of the present application;

[0065] Figure 4 is a point cloud density evaluation structure schematic diagram provided by the embodiment of the present application;

[0066] Figure 5is a schematic diagram of local path generation provided by an embodiment of the present application;

[0067] Figure 6 is a schematic diagram of local path optimization provided by an embodiment of the present application;

[0068] Figure 7 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the accompanying drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0070] It can be understood that the terms "first", "second", and the like used in the present application can be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "when" or "in response to determining".

[0071] The terms "at least one", "multiple", "each", "any" and the like used in the present application include one, two or more than two, multiple includes two or more than two, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0072] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by a person skilled in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0073] Before the embodiments of the present application are described in detail, first, some related technologies involved in the embodiments of the present application are described as follows:

[0074] Fast Randomized Search Trees (FRST): A data structure and algorithm for approximate nearest neighbor search. It is an improvement and extension of balanced binary trees. FRST aims to solve the efficiency problem of nearest neighbor search in high-dimensional data space. Nearest neighbor search in high-dimensional data space is a complex problem because the distance calculation and search complexity between data points grow exponentially with the increase of dimensions. FRST provides an efficient approximate nearest neighbor search method through randomization and some optimization techniques.

[0075] Occupancy Grid Map: A commonly used method for modeling the environment around a robot. It divides the environment into a three-dimensional grid, with each grid cell called a grid. Each grid can be represented as one of three states: occupied, free, and unknown, with the grid adjacent to the free area and the unknown area defined as the frontier. In the occupancy grid map, each grid records whether the position is occupied by an obstacle. When the robot's sensor obtains environmental information, the state of the grid map can be updated according to the sensor data. If the sensor observes that a certain position has an obstacle, the corresponding grid is marked as occupied; if the sensor observes that a certain position is free, the corresponding grid is marked as free; if the sensor cannot determine the state of a certain position, the corresponding grid is marked as unknown.

[0076] Traveling Salesman Problem (TSP): A classic combinatorial optimization problem. In the TSP problem, it is assumed that a traveling salesman wants to travel between multiple cities, and the problem requires determining a shortest path so that the traveling salesman can visit each city exactly once and finally return to the starting city.

[0077] A* Pathfinding Algorithm (A-star): A commonly used heuristic search algorithm for finding the shortest path or optimal path in a graph or graph network. It combines breadth-first search and heuristic evaluation function to efficiently search for the target node. The basic idea of A* algorithm is to maintain an open list and a closed list during the search process. The open list stores the nodes to be explored, and the closed list stores the nodes that have been explored. By constantly selecting the optimal node from the open list for expansion, the target node is found or the open list is empty.

[0078] Uniform B-spline Curve: A mathematical curve representation method used for interpolating and approximating curves. It is a special form of B-spline curve with uniform node spacing. B-spline curve is a curve defined by a set of control points and knot vectors. Knot vectors contain a series of node values that determine the position of the curve in the parameter space. For uniform B-spline curve, the node values of knot vectors are equally distributed, i.e., the distance between nodes is equal.

[0079] The unmanned aerial vehicle exploration method and system based on the LiDAR sensor provided by the embodiments of the present application relate to the technical field of unmanned aerial vehicle control. The unmanned aerial vehicle exploration method based on the LiDAR sensor provided by the embodiments of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in the terminal or the server. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, and the like, but is not limited thereto; the server end can be configured as a standalone physical server, can also be configured as a server cluster or a distributed system composed of multiple physical servers, can also be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform, and the server can also be a node server in a blockchain network; the software can be an application that implements the unmanned aerial vehicle exploration method based on the LiDAR sensor, and the like, but is not limited to the above forms.

[0080] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as a program module. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0081] As shown in Figure 1 , it is an implementation environment schematic diagram provided by the embodiments of the present application. Referring to Figure 1The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected via a network, either wirelessly or via a wired connection, to complete data transmission and exchange.

[0082] Server 101 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0083] Additionally, server 101 can also be a node server in a blockchain network. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms.

[0084] Terminal 102 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc. It can also be a vehicle-mounted terminal of the various device types described above, but is not limited to these. Terminal 102 and server 101 can be directly or indirectly connected via wired or wireless communication, and this embodiment of the invention does not impose any limitations.

[0085] Exemplary based on Figure 1 The implementation environment shown in this embodiment of the invention provides a drone exploration method based on a LiDAR sensor. The following description uses the application of this drone exploration method based on a LiDAR sensor in server 101 as an example. It can be understood that this method can also be applied to terminal 102.

[0086] In relevant applications, rapid autonomous exploration while ensuring dense scanning to obtain high-quality maps is crucial. Such an exploration system must meet two basic requirements: (1) the ability to quickly analyze and identify areas requiring dense scanning to obtain high-quality maps during the exploration planning process; and (2) the ability of the planner to create a more efficient and reasonable observation path after incorporating the analysis results, thereby improving coverage efficiency, reducing redundant observations, and ensuring efficient exploration and dense mapping. To meet these two requirements, this invention proposes an efficient UAV exploration method using a dense sensing online scanning method with a LiDAR sensor.

[0087] To address the problems existing in the prior art, the purpose of this invention is to provide a dense sensing online scanning method using LiDAR sensors to achieve efficient UAV exploration. This technology has the following characteristics:

[0088] 1. During flight, it can efficiently assess the density of the current point cloud map in real time and can achieve high-frequency updates to guide exploration planning.

[0089] 2. It can generate efficient global paths and safe, fully covered local observation paths. By combining global and local methods, the efficiency of exploration can be further improved.

[0090] 3. A novel trajectory optimization method based on map density assessment results generates smooth trajectories with time constraints to ensure the integrity of the scan.

[0091] Reference Figure 2 , Figure 2 This is a flowchart illustrating a LiDAR sensor-based UAV exploration method applied to a server, provided as an embodiment of the present invention. The execution subject of this method can be any of the aforementioned computer devices (including a server or a terminal). (Refer to...) Figure 2 The method may include the following steps:

[0092] The drone uses an onboard lidar sensor to acquire single-frame point cloud data.

[0093] The acquired single-frame point cloud data is registered into the global point cloud map using the SLAM module;

[0094] Based on the global point cloud map, a point cloud map and a 3D occupancy grid map are obtained through the SLAM module for point cloud density analysis and boundary detection; wherein, the point cloud map is used for point cloud density analysis; and the 3D occupancy grid map is used for boundary detection.

[0095] The results of the point cloud density analysis and the boundary detection are incorporated into the information gain to guide the generation of candidate viewpoints;

[0096] Global path planning is performed based on the generated candidate viewpoints, and local paths are determined.

[0097] The local path is optimized, and a time-constrained smooth trajectory is generated to guide the movement of the UAV.

[0098] The specific implementation process of the method of the present invention will be described in detail below with reference to the accompanying drawings:

[0099] like Figure 3 As shown, to achieve efficient autonomous exploration by unmanned aerial vehicles (UAVs), this invention proposes a dense sensing online scanning method using a LiDAR sensor. The proposed method framework is as follows: Figure 3The UAV acquires single frame point cloud data using on-board LiDAR sensors, which are then registered into a global point cloud map by a SLAM module. The results of the SLAM provide a point cloud map and a 3D occupancy grid map, which are used for point cloud density analysis and boundary detection, respectively. The results of the point cloud analysis and boundary detection are incorporated into the information gain to guide the generation of candidate viewpoints, followed by global path planning and local path determination. To improve exploration efficiency, the local path is optimized. Finally, a time-constrained smooth trajectory is generated to guide the motion of the UAV. The above process is repeated until no information gain is observed in the space, indicating that autonomous exploration is complete.

[0100] The implementation process of the real-time point cloud density evaluation method is described in detail below:

[0101] The density of the point defined by the embodiment of the application is calculated by the average distance between the point p∈R 3 and its K nearest neighbors. The specific calculation is shown in the following formula:

[0102]

[0103] If the point cloud density exceeds the threshold τ p , the point is considered to be a qualified point. Since the number of discrete point clouds is large, in order to cope with the requirements of rapid changes in the environment and continuous surface information and high update rate, the embodiment of the application designs a voxel representation structure composed of three levels to solve this problem. As shown in Figure 4 , the first layer of the voxel representation structure is the point cloud layer, which evaluates the eligibility of the point cloud in the visible area captured by the viewpoint by the method of calculating the point cloud density described above. The second layer is the high-resolution voxel layer, which divides the discrete point cloud into continuous voxels according to their positions, and the eligibility of the voxel is determined by the ratio of the number of qualified points to the number of unqualified points in the voxel. If the ratio exceeds the threshold τ v , it is considered to be qualified. The third layer is the grid layer, which places the voxels into larger grids, defined as S={s1,...,s n , to speed up the calculation and analysis speed. If the ratio of the number of qualified voxels to the number of unqualified voxels contained in the grid exceeds a certain threshold τ s , the grid s i is considered to be qualified.

[0104] The implementation process of the exploration path planning is described in detail below:

[0105] The exploration path planning aims to generate a local observation path and a global path for efficient dense scanning. The global path only uses limited computational resources to achieve faster processing speed, while the local observation path utilizes detailed environmental information to achieve precise scanning of local areas. To support these functions, embodiments of the present invention propose a novel planning method to obtain the local observation path, which is divided into three steps: candidate viewpoint generation, local path determination, and local path optimization. Finally, embodiments of the present invention generate a global path based on candidate viewpoints.

[0106] As shown in Figure 5 , the first step is candidate viewpoint generation: the goal of this part is to generate a set of candidate viewpoints to cover the boundary and unqualified point cloud area. In order to facilitate representation, embodiments of the present invention define the boundary and unqualified point cloud area as FAB, which is identified by point cloud density analysis. As shown in Figure 5 (top), a set of candidate points CP = {p1,...,p 3} is first uniformly sampled from the local planning range H ∈ R n , and the points on the boundary of H are defined as boundary points BP. The sampling points must meet the following requirements: (1) there is no obstacle within a distance d centered on the sampling point, and it is within the known area. (2) The current position of the UAV can be observed by all sampling points, which ensures that the generated sampling points are safe. Based on CP, embodiments of the present invention generate candidate viewpoints VP = {v1,...,v n} to completely cover FAB, where v i = {(p i , ξ i,1 ),..., (p i , ξ i,m )}, and (p i , ξ i,j ) is defined as an mpoint. For each point in CP, multiple yaw angles ξ i,j are determined as the maximum coverage of multiple independent sensors on FAB, and the information gain of each yaw angle that meets the sensor model and is not blocked by obstacles is evaluated:

[0107]

[0108] where Gain v represents the information gain of the mpoint. S v represents the number of unqualified grids observed from the mpoint, and F v represents the number of boundaries observed from the mpoint. g max represents the number of high-resolution voxels in S v , and λf is the weight coefficient of F v . These values take into account the observed unqualified grids and boundaries for calculating the information gain of candidate viewpoints.

[0109] Second step, local path determination: After generating candidate viewpoints, a local path is determined through a set of candidate viewpoints, so that the path length is shorter and can completely cover the FAB of the local area. The detailed steps of determining the local path are as follows:

[0110] (1) Calculate the total gain in H by calculating the sum of FAB of all candidate viewpoints in VP

[0111] (2) Sort the Gain of all mpoint v , and randomly select one from the top n mpoint to add to the selected viewpoint set Q.

[0112] (3) Recalculate the Gain v of VP, taking into account the influence of overlapping fields of view, which may reduce the Gain v of the remaining candidate viewpoints. (4) By repeating steps (2) and (3), a subset of viewpoints is selected from VP that can cover e. In addition, the current position coordinates of the UAV and the coordinates of the BP are pre-added to Q.

[0113] (5) This problem is considered as a variant of the Traveling Salesman Problem (TSP), and an open-loop path is generated for a set of points starting from the current position of the UAV and passing through all other points in Q. TSP can be quickly solved by the Lin-Kernighan heuristic algorithm, which is designed with an appropriate overhead matrix M tsp . The cost matrix Mtsp corresponding to an n+1 dimensional square matrix can be calculated as follows:

[0114] M tsp = l(PATH(v i ,v j ))i,j∈1,2,...,n

[0115] Where l is the length of the path PATH(v i ,v j ), which is obtained using the A* search algorithm from v i to v j . By repeating steps (2)-(5) N times, the shortest path length can be determined as a rough local path

[0116] Third step, local path optimization (such as Figure 6shown): Given that a finite number of random choices cannot guarantee finding the most efficient local path, embodiments of the present invention employ the following method to optimize the coarse path and obtain a refined local path, as shown in FIG. 4. Figure 6 First, embodiments of the present invention uniformly sample points p in the path segment of the coarse path, e.g., PATH(v3, v4). Based on these points, a range that can cover the FAB observed by v4 is generated. Then, the point v'4 closest to v3 is selected and replaces v4. It is worth noting that after replacing v4, the next path segment is PATH(v'4, v5). By repeating these steps for each segment of the path, embodiments of the present invention optimize the local path and obtain the final observation path, denoted as T H .

[0117] The fourth step, also the last step, is global path planning: Embodiments of the present invention define the global planning range as G ∈ R 3 and divide the space outside H into uniform cubic subspaces, while H is composed of multiple subspaces. The goal of global path planning T G is to determine the exploration order of the cubic subspaces, taking into account the limited computing resources available. Embodiments of the present invention define three states for each subspace as follows: Unexplored: The current subspace has not been explored. Exploring: The current subspace has been explored, but there are still viewpoints with non-zero information gain within the subspace. Explored: The current subspace has been fully explored and does not require further exploration. Then, embodiments of the present invention formalize the global path planning as a Traveling Salesman Problem (TSP), which calculates a closed-loop path starting from the current location of the UAV, the starting boundary point, the centroid of the subspace in the Exploring state, and an ending boundary point. In addition, the determination of the two boundary points is achieved by constructing a distance matrix containing the shortest path length. Therefore, after completing the exploration in H, the UAV can transition to another subspace in the Exploring state to continue the exploration according to T G .

[0118] The specific implementation process of time-constrained smooth trajectory generation is described in detail as follows:

[0119] Given the observation path T H , a continuous trajectory needs to be generated to achieve smooth navigation. However, blindly pursuing fast movement and fast rotation can lead to insufficient point cloud scanning, resulting in redundant exploration and low coverage efficiency of previously explored areas. Therefore, the method of embodiments of the present invention combines the optimization of rotation smoothness and the time constraints of movement and rotation to generate a smooth motion trajectory.

[0120] First, embodiments of the present invention use a uniform B-spline curve to smooth the rotation and set a time constraint term to control the rotation time of the rotation angle. The rotation angle ξ, the transition time t b , and the time constraint term Jt The calculation formula is as follows:

[0121]

[0122] J t =max{(N s -k)×Δt,t b}

[0123] in, Indicates time t i The location reached, and These are the maximum values ​​of the constraints on speed and yaw angle angular velocity, respectively, N. s and k are the number of control points and the dimension of the uniform B-spline curve, respectively, and Δt represents the time interval.

[0124] T H The distance and angle vectors for each observation point are defined as D and Y, respectively. Therefore, the optimization equation for rotation is:

[0125]

[0126] J s It is a smoothness penalty term, J t Q is the time constraint term, Q is the set of control points, and δs and δt are the weights of each penalty and constraint term.

[0127] Meanwhile, trajectory T H Uniform B-spline curves were also used for optimization. The time constraint t′ was also used. b The optimization problem of the trajectory can be formulated as follows:

[0128]

[0129] J t =max{(N s -k)×Δt,t b}

[0130]

[0131] in, J is a constant related to the data update rate of the LiDAR sensor. s It is a smoothness penalty term, J c It's a collision penalty item, J d J′ represents the feasibility penalty term. t These are time constraints, and δs, δc, δd, and δ't are the weights of each penalty term.

[0132] In summary, compared with the prior art, the present invention has the following advantages:

[0133] 1. High completeness of the generated map: the exploration framework uses a point cloud density evaluation algorithm to quickly calculate unqualified areas of the map to guide the exploration of the UAV while the environment is being updated at a high speed.

[0134] 2. High efficiency of exploration: a method combining global and local exploration is introduced to generate efficient exploration decisions, and the local path is optimized and smoothed to further improve flight efficiency.

[0135] The embodiment of the present application also provides an unmanned aerial vehicle exploration system based on a LiDAR sensor, comprising:

[0136] A first module is configured to acquire single-frame point cloud data by using an airborne LiDAR sensor.

[0137] A second module is configured to register the acquired single-frame point cloud data into a global point cloud map by using a SLAM module.

[0138] A third module is configured to obtain a point cloud map and a three-dimensional occupancy grid map by using the SLAM module according to the global point cloud map, to perform point cloud density analysis and boundary detection; wherein the point cloud map is used for point cloud density analysis; and the three-dimensional occupancy grid map is used for boundary detection.

[0139] A fourth module is configured to include the results of the point cloud density analysis and the boundary detection into information gain to guide the generation of candidate viewpoints.

[0140] A fifth module is configured to perform global path planning according to the generated candidate viewpoints and determine a local path.

[0141] A sixth module is configured to optimize the local path and generate a time-constrained smooth trajectory to guide the movement of the unmanned aerial vehicle.

[0142] It can be understood that the contents in the above method embodiments are applicable to the system embodiments, the system embodiments specifically realize the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0143] The embodiment of the present application also provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor realizes the above unmanned aerial vehicle exploration method based on a LiDAR sensor when executing the computer program. The electronic device can be any intelligent terminal, such as a tablet computer or a vehicle-mounted computer.

[0144] It can be understood that the contents in the above method embodiments are all applicable to the present device embodiments, the present device embodiments specifically implement the functions same as those of the above method embodiments, and achieve the same beneficial effects as those of the above method embodiments.

[0145] Please refer to Figure 7 , Figure 7 The hardware structure of the electronic device of another embodiment is illustrated, and the electronic device comprises:

[0146] The processor 701 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is used to execute related programs to implement the technical solutions provided by the present application.

[0147] The memory 702 can be implemented in the form of a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory). The memory 702 can store an operating system and other application programs. When the technical solutions provided by the present application are implemented by software or firmware, the related program codes are stored in the memory 702 and are called and executed by the processor 701 to implement the LiDAR sensor-based unmanned aerial vehicle exploration method of the present application.

[0148] The input / output interface 703 is used to realize information input and output.

[0149] The communication interface 704 is used to realize the communication interaction between the present device and other devices. The communication can be realized by a wired manner (such as a USB, a network cable, etc.) or a wireless manner (such as a mobile network, WIFI, Bluetooth, etc.).

[0150] The bus 705 is used to transmit information between the components (such as the processor 701, the memory 702, the input / output interface 703, and the communication interface 704) of the device.

[0151] The processor 701, the memory 702, the input / output interface 703, and the communication interface 704 are connected to each other through the bus 705 to realize the communication connection between them inside the device.

[0152] The present application also provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned LiDAR sensor-based unmanned aerial vehicle exploration method is realized.

[0153] It can be understood that the contents in the above method embodiments are all applicable to the present storage medium embodiments, the present storage medium embodiments specifically implement the functions same as those of the above method embodiments, and achieve the beneficial effects same as those of the above method embodiments.

[0154] The memory, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0155] The embodiments described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0156] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than those shown in the figures, or combine certain steps, or different steps.

[0157] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment.

[0158] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the functional modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0159] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of this application, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of these terms herein is to be construed to cover a changeable order, sequence or arrangement, for example, unless otherwise stated or understood by the context of the description. Furthermore, the terms "comprising", "having", "including", and "containing" and any variations thereof used herein are intended to cover a non-exclusive inclusion, such that a process, method, system, product, or apparatus that comprises, has, includes or contains a list of elements is not necessarily limited to those elements, but can include other elements not expressly listed or inherent to such process, method, system, product or apparatus.

[0160] It should be understood that, in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0161] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the above-mentioned units is only a logical functional division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0162] The units described above as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0163] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0164] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in the form of a contribution to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0165] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not intended to limit the scope of the embodiments of the present application. Any modification, equivalent replacement, and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A drone exploration method based on a LiDAR sensor, characterized in that, Includes the following steps: The drone uses an onboard lidar sensor to acquire single-frame point cloud data. The acquired single-frame point cloud data is registered into the global point cloud map using the SLAM module; Based on the global point cloud map, a point cloud map and a 3D occupancy grid map are obtained through the SLAM module for point cloud density analysis and boundary detection; wherein, the point cloud map is used for point cloud density analysis; and the 3D occupancy grid map is used for boundary detection. The results of the point cloud density analysis and the boundary detection are incorporated into the information gain to guide the generation of candidate viewpoints; Global path planning is performed based on the generated candidate viewpoints, and local paths are determined. The local path is optimized, and a time-constrained smooth trajectory is generated to guide the movement of the UAV; the point cloud density analysis process includes: The density of each point cloud is calculated based on the average distance between each point cloud and its nearest neighbors. The formula for calculating the point cloud density is as follows: Where K represents the number of nearest neighbors; Represents the i-th point; The threshold representing the point cloud density; Create a voxel representation structure; the voxel representation structure includes a point cloud layer, a high-resolution voxel layer, and a mesh layer; In the point cloud layer, the qualification of the point cloud within the visible area captured by the viewpoint is evaluated according to the calculation formula of the point cloud density. In the high-resolution voxel layer, the discrete point cloud is divided into continuous voxels according to its position. The passability of a voxel is determined by the ratio of the number of passable points to the number of non-passable points within the voxel. If the ratio exceeds a threshold, it is considered passable. In the grid layer, voxels are placed within a larger grid. If the ratio of the number of qualified voxels to the number of unqualified voxels within the grid exceeds a certain threshold, the grid is considered qualified. The incorporation of the results of the point cloud density analysis and the boundary detection into the information gain to guide the generation of candidate viewpoints includes: A set of candidate points is uniformly sampled within the local planning range, and points on the boundary of the local planning range are defined as boundary points; Based on the sampled candidate points, candidate viewpoints are generated to fully cover the target area of ​​the boundary and the unqualified point cloud. For each of the candidate points, multiple yaw angles are determined as the coverage of the target area by the maximum coverage of multiple independent sensors. The information gain of each yaw angle that conforms to the sensor model and is not obstructed by obstacles is evaluated, thereby generating candidate viewpoints.

2. The UAV exploration method based on a LiDAR sensor according to claim 1, characterized in that, The formula for calculating the information gain is: in, The information gain represents the m-point. This indicates the number of defective meshes observed from mpoint; This represents the number of boundaries observed from mpoint; express The number of medium to high resolution voxels; yes Weighting coefficients; Represent each The number of substandard voxels.

3. The UAV exploration method based on a LiDAR sensor according to claim 2, characterized in that, The determination of the local path includes: The total gain within the local planning range is calculated by summing the target regions of all candidate viewpoints. Sort all mpoints by information gain and randomly select one from the top n mpoints to add to the selected viewpoint set; Recalculate the information gain in the candidate viewpoints, pre-add the current position coordinates of the UAV and the coordinates of the boundary points to the viewpoint set, and return to perform the step of sorting the information gain of all mpoints and randomly selecting one from the top n mpoints to add to the selected viewpoint set, until a subset of viewpoints that can cover the total gain is obtained. Based on the solution strategy for the Traveling Salesman Problem, an open-loop path is generated starting from the drone's current position and passing through all other points in the viewpoint set, resulting in a coarse local path.

4. The UAV exploration method based on a LiDAR sensor according to claim 3, characterized in that, The optimization of the local path includes: Uniformly sample points from the target path segment of the rough local path; Generate a range that can cover the target area based on the sampling points; Replace one endpoint of the target path segment to obtain a new target path segment, and return to execute the step of generating a range that can cover the target area based on the sampling points, until the optimization of each path segment in the local path is completed.

5. The UAV exploration method based on a LiDAR sensor according to claim 4, characterized in that, The step of performing global path planning based on the generated candidate viewpoints includes: Define the global planning scope and divide the space outside the local planning scope into uniform cubic subspaces; The global path planning is formalized as a traveling salesman problem, which calculates the closed-loop path starting from the drone's current position, the starting boundary point, the centroid of the subspace in the exploration state, and an ending boundary point. Two boundary points are determined by constructing a distance matrix that includes the shortest path length.

6. The UAV exploration method based on a LiDAR sensor according to claim 5, characterized in that, The generation of a time-constrained smooth trajectory includes: Use a uniform B-spline curve to smooth the rotation, and set a time constraint to control the rotation time of the rotation angle; Among them, rotation angle Transition time and time constraints The calculation formulas are as follows: in, Indicates time The location reached, and These are the maximum values ​​of the constraints on speed and yaw angle angular velocity, respectively. and k are the number of control points and the dimension of the uniform B-spline curve, respectively, and ∆t represents the time interval, given the observation path. The distance and angle vectors for each observation point are defined as D and Y, respectively; For a given observation path, optimization is performed using a uniform B-spline curve to generate a smooth trajectory subject to time constraints.

7. A system for implementing the LiDAR sensor-based unmanned aerial vehicle (UAV) exploration method as described in any one of claims 1-6, characterized in that, include: The first module is used by the UAV to acquire single-frame point cloud data using an airborne lidar sensor. The second module is used to register the acquired single-frame point cloud data into the global point cloud map through the SLAM module; The third module is used to obtain a point cloud map and a 3D occupancy grid map through the SLAM module based on the global point cloud map, so as to perform point cloud density analysis and boundary detection; wherein, the point cloud map is used for point cloud density analysis; and the 3D occupancy grid map is used for boundary detection. The fourth module is used to incorporate the results of the point cloud density analysis and the boundary detection into the information gain to guide the generation of candidate viewpoints; The fifth module is used to perform global path planning based on the generated candidate viewpoints and determine local paths; The sixth module is used to optimize the local path and generate a time-constrained smooth trajectory to guide the movement of the UAV.

8. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 6.

Images