Method for selecting a drone landing area, drone and medium

Abstract

The present application discloses a method for selecting a landing area for a drone, a drone and a medium. The method includes the following steps: obtaining a three-dimensional point set of the ground area in the drone coordinate system; wherein the drone coordinate system includes mutually perpendicular X-axis, Y-axis and Z-axis, and the three-dimensional point set includes multiple three-dimensional points; classifying the three-dimensional point set according to the average curvature of each three-dimensional point to obtain a first plane set including at least one plane point set; determining in the first plane set the area corresponding to the plane point set that meets the preset conditions as the landing area, the preset conditions including that the plane area of ​​the plane point set is greater than the area threshold, and the first angle between the normal vector of the plane point set and the Z axis is less than the first angle threshold. In the above manner, the present application can screen out a landing area with high safety.

Description

Technical Field

[0001] The present application relates to the field of drones, and in particular to a method for selecting a landing area for a drone, a drone, and a computer-readable storage medium. Background Art

[0002] In recent years, drones have been widely used in agriculture, public transportation, public security and other fields, and the number of drones has increased significantly. With the development of artificial intelligence, drone intelligent flight and autonomous flight have received more and more attention.

[0003] At present, most UAV missions are carried out by manually controlling the movement of UAVs. With the development of technology and the change of demand, the military, detection and other fields require UAVs to be able to move autonomously, including take-off, flight, landing and other actions.

[0004] Among all the stages of drone movement, the landing stage is the most critical and also carries the greatest risk of crashing. Currently, drones are mainly landed by manual control. As drone application scenarios become more and more extensive, autonomous landing will become the mainstream of drones. However, in related technologies, autonomous landing of drones has problems such as poor adaptability to complex environments and poor safety of the selected landing plane. Summary of the invention

[0005] The present application provides a method for selecting a landing area for a drone, a drone, and a computer-readable storage medium to solve the problems of poor adaptability and poor safety of autonomous landing of drones in related technologies.

[0006] In order to solve the above technical problems, the present application provides a method for selecting a landing area for a drone. The method includes: obtaining a three-dimensional point set of a ground area in a drone coordinate system; wherein the drone coordinate system includes mutually perpendicular X-axis, Y-axis and Z-axis, and the three-dimensional point set includes multiple three-dimensional points; classifying the three-dimensional point set according to the average curvature of each three-dimensional point to obtain a first plane set including at least one plane point set; determining in the first plane set an area corresponding to a plane point set that meets preset conditions as a landing area, wherein the preset conditions include that the plane area of ​​the plane point set is greater than an area threshold, and that the first angle between the normal vector of the plane point set and the Z axis is less than a first angle threshold.

[0007] To solve the above technical problems, the present application provides a drone. The drone includes a depth camera, a processor and a memory, wherein the processor is coupled to the depth camera and the memory, and executes instructions when working to cooperate with the depth camera and the memory to implement the above method for selecting a drone landing area.

[0008] To solve the above technical problems, the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, which can be executed by a processor to implement the above method for selecting a landing area for a drone.

[0009] The present application can adapt to complex scenarios by classifying multiple three-dimensional points corresponding to the ground area where the drone is estimated to land into at least one plane point set. Compared with two-dimensional image recognition of the landing environment, three-dimensional points can more accurately and effectively extract the landing area. Furthermore, the present application selects the area corresponding to the plane point set whose plane area is greater than the area threshold and whose first angle with the Z axis is less than the first angle threshold from the first plane set as the landing area, so that the selected landing area is close to the current posture of the drone and the corresponding plane area is sufficient for the drone to land, so that the drone can land in the landing area more safely and securely. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a structural schematic diagram of an embodiment of a drone provided by the present application;

[0011] Figure 2 This is a flow chart of an embodiment of a method for selecting a landing area for a drone provided by the present application;

[0012] Figure 3 is a flow chart of an implementation method of classifying a set of three-dimensional points according to the average curvature of each three-dimensional point provided by the present application;

[0013] Figure 4 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0014] In order to enable those skilled in the art to better understand the technical solution of the present invention, the method for selecting a drone landing area, the drone and the computer-readable storage medium provided in the present application are further described in detail below in combination with the accompanying drawings and specific implementation methods.

[0015] See also Figure 1 , Figure 1 1 is a schematic diagram of a structure of an embodiment of a drone provided by the present application. In this embodiment, the drone 10 may be a multi-rotor drone, including a depth camera 11, a processor (not shown), a memory (not shown) and a plurality of rotors 12.

[0016] The number of rotors 12 may be 3, 4, 6, 8, etc.

[0017] Compared with traditional cameras, the depth camera 11 adds a depth measurement function, so that it is more convenient and accurate to perceive the surrounding environment and changes. In this embodiment, the depth camera 11 can be a TOF (Time of flight). The depth camera 11 based on the TOF principle can measure targets at a long distance range and is not affected by changes in lighting and object texture. Therefore, it can adapt to complex landing environments and collect more accurate three-dimensional point cloud data.

[0018] When the processor receives a landing signal during the flight of the drone 10, it controls the depth camera 11 to collect three-dimensional point cloud data of the ground area corresponding to the current position, and stores the three-dimensional point cloud data in the memory.

[0019] The processor extracts the three-dimensional point cloud data from the memory, performs filtering on the three-dimensional point cloud data, and obtains a three-dimensional point set. The three-dimensional point set includes three-dimensional points in multiple drone coordinate systems, and the drone coordinate system includes mutually perpendicular X-axis, Y-axis, and Z-axis.

[0020] The processor classifies the three-dimensional point set according to the average curvature of each three-dimensional point to obtain a first plane set including at least one plane point set. Each plane set corresponds to a plane area in the ground area. By using three-dimensional points to segment the ground area, it is not necessary to rely on a preset landing target area, so that it can adapt to various application scenarios and is not affected by the geographical environment and the location.

[0021] The processor determines the area corresponding to the plane point set that meets the preset conditions in the first plane set as the landing area. The preset conditions include that the plane area of ​​the plane point set is greater than the area threshold, and the first angle between the normal vector of the plane point set and the Z axis is less than the first angle threshold. Among them, the area threshold is the safe area where the drone 10 can land, such as the area of ​​the area enclosed by the landing gear (not shown) of the drone 10. The area enclosed by the landing gear is smaller than the orthographic projection area of ​​the drone 10, which can reduce the constraints on the landing area while ensuring the landing safety. In addition, the first angle between the normal vector of the plane point set and the Z axis is less than the first angle threshold, that is, the plane angle between the plane corresponding to the plane point set and the plane where the drone 10 is located in the current posture (the plane where the X axis and the Y axis are located) is less than the first angle threshold, thereby ensuring that the drone 10 lands smoothly and safely in the landing area corresponding to the plane point set.

[0022] Based on the above-mentioned drone 10, the following embodiment of the method for selecting a drone landing area is implemented:

[0023] See also Figure 2 , Figure 2 It is a flowchart of an embodiment of a method for selecting a landing area for a drone provided in the present application.

[0024] S110: Obtain a three-dimensional point set of the ground area in the drone coordinate system.

[0025] The three-dimensional point set includes multiple three-dimensional points.

[0026] The three-dimensional points can be collected using a depth camera carried by a drone. Specifically, the depth camera can continuously emit light pulses to the ground area, and then receive the light pulses reflected from the ground. The distance of the ground area relative to the depth camera is calculated by calculating the flight time from the emitted to the received light pulses, thereby obtaining the three-dimensional point cloud data of the ground area in the drone coordinate system. The drone coordinate system is fixedly connected to the drone, with the origin at the center of gravity of the drone, the X-axis pointing to the nose of the drone (the direction of the nose is related to the layout of multiple rotors), and the Z-axis is perpendicular to the X-axis and downward in the symmetry plane of the drone. The Y-axis perpendicular to the X-axis and Z-axis is determined according to the right-hand rule.

[0027] The 3D points in the 3D point set can be obtained after filtering the 3D point cloud data. Specifically, when the depth camera collects 3D point cloud data of the ground area, the 3D point cloud data contains noise due to the influence of factors such as multiple reflections, scattered light and ambient light. Therefore, before selecting the landing area based on the 3D points, the 3D point cloud data can also be filtered to remove the noise in the 3D point cloud data to obtain the 3D points in the 3D point set. The filtering method can be bilateral filtering, voxel filtering or statistical filtering. Filtering the 3D point cloud data can improve the accuracy of the 3D points, thereby better performing subsequent processing such as surface reconstruction and plane extraction.

[0028] S120: Classifying the three-dimensional point set according to the average curvature of each three-dimensional point to obtain a first plane set including at least one plane point set.

[0029] Specifically, the average curvature of each 3D point is calculated, and based on the average curvature of each 3D point, the 3D points in the 3D point set are classified into corresponding plane point sets to obtain a first plane set including at least one plane point set. The plane set indicates that the 3D points in the plane set can form a continuous relatively flat area.

[0030] The 3D points in the 3D point set can exist in the form of discrete points. Therefore, before calculating the average curvature, the discrete 3D points are first continuous, that is, fitted to form a surface. There are many algorithms for generating surfaces from discrete 3D points, such as interpolation algorithms or least squares methods. Any point on the surface (a 3D point on the surface that is fitted to form the surface) has countless curvature radii, among which there is a maximum curvature radius and a minimum curvature radius corresponding to the point. The average curvature is the average of the maximum curvature and the minimum curvature of the point. The formula for the average curvature is as follows:

[0031] or

[0032] H=k1+k2

[0033] Wherein, H represents the average curvature, k1 represents the maximum curvature radius, and k2 represents the minimum curvature radius.

[0034] Mean curvature is an "extrinsic" measure of curvature that describes locally the curvature of a surface embedded in the surrounding space (e.g. a two-dimensional surface embedded in three-dimensional Euclidean space).

[0035] In this embodiment, since addition is used to calculate the average curvature of the three-dimensional points, the amount of calculation when the drone classifies the three-dimensional points can be reduced.

[0036] After the three-dimensional points in the three-dimensional point set are fitted to form a surface, the point normal corresponding to each three-dimensional point can also be calculated. The method for calculating the point normal of a certain three-dimensional point on the surface can refer to the existing technology, so it will not be repeated here.

[0037] After calculating the average curvature of each 3D point in the 3D point set, the process of classifying the 3D points based on the average curvature of each 3D point is as follows:

[0038] See also Figure 3 , Figure 3 It is a flowchart diagram of an implementation method of classifying a set of three-dimensional points according to the average curvature of each three-dimensional point provided by the present application.

[0039] S121: Determine the seed point of the plane point set in the three-dimensional point set according to the average curvature.

[0040] A seed is a starting point for the growth of a region. In this embodiment, which three-dimensional points can be used as seed points is determined according to the average curvature of the three-dimensional points.

[0041] Specifically, the 3D points in the 3D point set are sorted from small to large according to the average curvature, and the 3D point with the smallest average curvature is determined as the seed point. The smaller the average curvature of the 3D point, the smaller the curvature of the area where the 3D point is located in any direction. Based on this 3D point, other 3D points that can form a relatively flat area can be quickly screened out in the 3D point set.

[0042] After the seed point is screened, the seed point is added to an empty plane point set, and the 3D point corresponding to the seed point in the 3D point set is deleted.

[0043] S122: Filtering three-dimensional points to be added to the plane point set from the three-dimensional point set based on the seed point.

[0044] The seed point is compared with other 3D points in the 3D point set, 3D points with similar properties are added to the plane point set where the seed point is located, and the corresponding 3D points in the 3D point set are deleted.

[0045] Specifically, the neighborhood points of the seed point are obtained in the three-dimensional point set. In this embodiment, Radius-NN (Radius-NearestNeighbor) can be used to obtain the neighborhood points of the seed point. The Radius-NN algorithm presets a radius range and selects three-dimensional points within the radius range of the seed point as neighborhood points. Since the search range is pre-defined, the number of three-dimensional points to be searched can be reduced.

[0046] The first point normal of the seed point and the second point normal of each neighborhood point are obtained, and the second angle between the first point normal and each second point normal is calculated respectively.

[0047] Compare the second angle with the second angle threshold, so as to determine whether the seed point and the domain point are in the same relatively flat area. If the second angle is less than the second angle threshold, the corresponding 3D point is added to the plane point set, and the corresponding 3D point is deleted from the 3D point set; if the second angle is greater than or equal to the second angle threshold, the corresponding 3D point is added to the 3D point to be assigned, and the corresponding 3D point is deleted from the 3D point set.

[0048] The first angle threshold may be 1°, 2°, 5°, 8°, 10° or 20°, etc.

[0049] S123: After the screening is completed, the three-dimensional points to be allocated that are not added to the plane point set are assigned to the three-dimensional point set, and the step of determining the seed point of the plane point set according to the average curvature is performed until all the three-dimensional points to be allocated are allocated to the corresponding plane point set to obtain the first plane set.

[0050] S130: Determine in the first plane set an area corresponding to a plane point set that meets a preset condition as a landing area.

[0051] The preset conditions include that the plane area of ​​the plane point set is greater than an area threshold, and that a first angle between the normal vector of the plane point set and the Z axis is less than a first angle threshold.

[0052] Determining whether a plane point set meets a preset condition specifically includes: calculating the plane area corresponding to each plane point set. Eliminating plane point sets whose plane areas are not greater than an area threshold to obtain a second plane set. Calculating a first angle between a first normal vector and a second normal vector of each plane point set in the second plane set, wherein the second normal vector is parallel to the Z axis. The second normal vector is, for example, (0, 0, 1). Determining an area where a plane point set whose first angle is less than a first angle threshold is located is a landing area.

[0053] There may be multiple plane point sets that meet the area and angle conditions, so it is necessary to screen out the most suitable landing area from them. Optionally, the preset condition may also include that the first angle corresponding to the plane point set is the smallest in the second plane set.

[0054] Specifically, the plane point set with the smallest first angle in the second plane set is selected, and the smallest first angle is compared to see whether it is less than the angle threshold. If the smallest first angle is less than the angle threshold, the area corresponding to the plane point set corresponding to the smallest first angle can be used as the landing area.

[0055] In some implementations, a sorting algorithm may be used to screen the plane point set with the smallest first angle, such as bubble sort, quick sort, insertion sort, or recursive sort.

[0056] In another embodiment, in order to improve the screening efficiency and reduce the amount of calculation of the drone, the screening can be performed through the following steps:

[0057] Take the first plane point set from the second plane set as the candidate plane, and delete the first plane point set in the second plane set; obtain the second plane point set from the second plane set; compare whether the third angle of the candidate plane is smaller than the first angle of the second plane point set; if the third angle is smaller than the first angle of the second plane point set, delete the second plane point set in the second plane set, and return to the step of obtaining the second plane point set from the second plane set; if the third angle is not smaller than the first angle of the second plane point set, replace the first plane point set with the second plane point set as the candidate plane, and return to the step of obtaining the second plane point set from the second plane set; until the second plane set is empty, and the plane point set corresponding to the candidate plane is the plane point set with the smallest first angle in the second plane set.

[0058] The first angle between the normal vector of the plane point set and the Z axis is the smallest and less than the first angle threshold, indicating that the inclination angle of the area corresponding to the plane point set is more consistent with the current posture of the drone. When landing, the drone can land more smoothly in this area, thereby improving the safety of the drone's landing.

[0059] The method of the above embodiment may exist in the form of a computer program, so the present application proposes a computer readable storage medium, see Figure 4 , Figure 4 The computer-readable storage medium 400 of this embodiment stores a computer program 401, which can be executed to implement the method in the above embodiment.

[0060] The computer-readable storage medium 400 of this embodiment may be a medium that can store program instructions, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or may be a server that stores the program instructions. The server may send the stored program instructions to other devices for execution, or may execute the stored program instructions by itself.

[0061] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0062] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0063] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0064] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0065] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for selecting a landing area for a drone, characterized in that: The selection method includes: Acquire a three-dimensional point set of the ground area in the drone coordinate system; wherein the drone coordinate system includes an X-axis, a Y-axis, and a Z-axis that are perpendicular to each other, the three-dimensional point set includes a plurality of three-dimensional points, and the three-dimensional point set includes a point normal of each three-dimensional point; Determine at least one seed point of a plane point set in the three-dimensional point set according to the average curvature, wherein the average curvature is an average value of a maximum curvature and a minimum curvature corresponding to the three-dimensional point; Based on a first point normal of the seed point and a second point normal of a neighborhood point of the seed point, selecting three-dimensional points to be added to the plane point set from the three-dimensional point set; After the screening is completed, the three-dimensional points to be allocated that are not added to the plane point set are assigned to the three-dimensional point set, and the step of determining the seed point of the plane point set according to the average curvature is performed until all the three-dimensional points to be allocated are allocated to the corresponding plane point set to obtain a first plane set; An area corresponding to a plane point set that meets preset conditions is determined in the first plane set as a landing area, wherein the preset conditions include that a plane area of ​​the plane point set is greater than an area threshold, and a first angle between a normal vector of the plane point set and the Z axis is less than a first angle threshold.

2. The selection method according to claim 1, characterized in that: The step of determining, in the first plane set, an area corresponding to a plane point set that meets a preset condition as a landing area includes: Calculate the plane area corresponding to each of the plane point sets; Eliminate the plane point set whose plane area is not greater than the area threshold to obtain a second plane set; Calculate a first angle between a first normal vector and a second normal vector of each plane point set in the second plane set, wherein the second normal vector is parallel to the Z axis; Determine that an area where a set of plane points where the first angle is smaller than a first angle threshold is located is a landing area.

3. The selection method according to claim 1, characterized in that: The three-dimensional point set includes a point normal of each three-dimensional point, and the three-dimensional points to be added to the plane point set are screened in the three-dimensional point set based on the first point normal of the seed point and the second point normal of the region point of the seed point, including: Acquire the neighborhood points of the seed point in the three-dimensional point set; Obtaining a first point normal of the seed point and a second point normal of each of the neighborhood points; respectively calculating a second angle between the normal line of the first point and each normal line of the second point; comparing the second angle with a second angle threshold; If the second angle is less than the second angle threshold, adding the corresponding three-dimensional point to the plane point set, and deleting the corresponding three-dimensional point from the three-dimensional point set; If the second angle is greater than or equal to the second angle threshold, the corresponding three-dimensional point is added to the three-dimensional points to be allocated, and the corresponding three-dimensional point is deleted from the three-dimensional point set.

4. The selection method according to claim 1, characterized in that: The step of determining at least one seed point of a plane point set in the three-dimensional point set according to the average curvature comprises: sorting the three-dimensional points in the three-dimensional point set from small to large according to the average curvature; The three-dimensional point with the smallest average curvature is determined as the seed point.

5. The selection method according to claim 2, characterized in that: The preset condition also includes that the first angle corresponding to the plane point set is the smallest in the second plane set; Before determining that the area where the plane point set whose first angle is less than the first angle threshold is located is a landing area, the method includes: Filtering a plane point set with the smallest first angle in the second plane set; Compare whether the smallest first angle is smaller than the angle threshold.

6. The selection method according to claim 5, characterized in that: The step of screening a plane point set with the smallest first angle in the second plane set includes: Taking the first plane point set from the second plane set as a candidate plane, and deleting the first plane point set in the second plane set; Acquire a second plane point set from the second plane set; Comparing whether the third angle of the candidate plane is smaller than the first angle of the second plane point set; If the third angle is smaller than the first angle of the second plane point set, deleting the second plane point set in the second plane set, and returning to the step of obtaining the second plane point set from the second plane set; If the third angle is not less than the first angle of the second plane point set, replacing the first plane point set with the second plane point set as the candidate plane, and returning to the step of obtaining the second plane point set from the second plane set; Until the second plane set is empty, the plane point set corresponding to the candidate plane is the plane point set with the smallest first angle in the second plane set.

7. The selection method according to claim 1, characterized in that: Before obtaining the three-dimensional point set of the ground area in the drone coordinate system, the method includes: Collecting three-dimensional point cloud data of the ground area using a depth camera; The three-dimensional point cloud data is filtered and the filtered three-dimensional point cloud data is used as the three-dimensional point set.

8. A drone, characterized in that: It includes a depth camera, a processor and a memory, wherein the processor is coupled to the depth camera and the memory, and executes instructions when working to cooperate with the depth camera and the memory to implement the method for selecting a drone landing area as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement the method for selecting a landing area for a drone according to any one of claims 1 to 7.

Images