Autonomous landing method, medium and electronic equipment for drones

Abstract

The present application relates to the field of drone control technology, and in particular to a drone autonomous landing method, medium, and electronic device. The method comprises: in response to receiving a target landing command, obtaining the current position information PM of the target drone; obtaining a first distance set D1 based on PM; determining a target preset landing area based on D1; obtaining a second distance set D2; determining a key preset sub-landing area based on D2; determining whether a point cloud collection point in the key preset sub-landing area is a safe point cloud collection point; if it is a safe point cloud collection point, obtaining a three-dimensional laser point cloud map; performing plane fitting on the key preset sub-landing area; and controlling the target drone to land in the key preset sub-landing area if the plane slope is less than a preset plane slope threshold. The drone autonomous landing method provided in the present application accurately determines the landing point for the drone in an unfamiliar environment, ensuring the drone's rapid, safe, and stable landing.

Description

Technical Field

[0001] The present application relates to the field of drone control technology, and in particular to a drone autonomous landing method, medium, and electronic equipment. Background Art

[0002] When operating in unfamiliar environments, drones may need to land in a designated area. Because the specific conditions in this area are unknown, existing technologies often use images captured by the drone above the designated area to select a specific landing point. However, since the drone is some distance above the ground, the captured images may not fully reflect the specific conditions of the landing point. There is a risk of obstacles damaging the drone, or uneven terrain preventing the drone from landing steadily. Therefore, there is an urgent need for a technology that can accurately determine the landing point for drones in unfamiliar environments, ensuring a fast, safe, and stable landing. Summary of the Invention

[0003] The technical problem to be solved by this application is: how to accurately determine the landing point for a drone in an unfamiliar environment and ensure the drone's rapid, safe and stable landing.

[0004] According to a first aspect of the present application, a method for autonomous landing of a drone is provided, comprising:

[0005] S100, in response to receiving the target landing instruction, obtain the current position information PM of the target UAV = (lng m ,lat m ); among them, lng m The current longitude information of the target drone; lat m The current latitude information of the target UAV.

[0006] S200, according to PM, obtain the first distance between the current position and each preset landing zone to obtain a first distance set D1 = (D11, D12, ..., D1a, ..., D1b); a = 1, 2, ..., b; where D1a is the first distance between the current position ranked a and the image acquisition point corresponding to the preset landing zone; D1a = 2arcsin(sin 2 (lat m -lat a ) / 2)+cos(lat m )*cos(lat a )*sin 2 ((lng m -lng a ) / 2)) 1 / 2 *dc;D1a<D1(a+1);lat aThe latitude information of the image acquisition point corresponding to the preset landing area corresponding to D1a; lng a is the longitude information of the image acquisition point corresponding to the preset landing zone corresponding to D1a; dc is the radius of the Earth's equator; b is the number of preset landing zones; the image acquisition point is the shooting position when the image acquisition device preset by the target UAV takes the image corresponding to the preset landing zone; the preset landing zone is the area within the target UAV's working area corresponding to the image containing the preset area type; the preset area type is the area type where the UAV can land.

[0007] S300: Determine the preset landing zone corresponding to the smallest first distance in D1 as the target preset landing zone.

[0008] S400: Obtain the second distance between each preset sub-landing area of the target preset landing area and the PM to obtain a second distance set D2 = (D21, D22, ..., D2c, ..., D2d); c = 1, 2, ..., d; D2c is the second distance between the geometric center point of the preset sub-landing area ranked c and the PM; D2c = 2arcsin(sin 2 (lat m -lat c ) / 2)+cos(lat m )*cos(lat c )*sin 2 ((lng m -lng c ) / 2)) 1 / 2 *dc; D2a<D2(a+1); where lat c lng is the latitude information of the geometric center point corresponding to the preset sub-landing area corresponding to D2c; c is the longitude information of the geometric center point corresponding to the preset sub-landing zone corresponding to D2c; d is the number of preset sub-landing zones in the target preset landing zone; the preset sub-landing zone is an area determined according to the preset area type in the preset landing zone.

[0009] S500: Determine the preset sub-landing zone corresponding to the smallest second distance in D2 as a key preset sub-landing zone.

[0010] S600: Control the target UAV to move to a point cloud collection point in a key preset sub-landing area, and determine whether the point cloud collection point in the key preset sub-landing area is a safe point cloud collection point based on a preset radar detection sensor; the above-mentioned point cloud collection point is a position above the key preset sub-landing area, with a true height from the ground where the key preset sub-landing area is located being a preset true height, and capable of obtaining complete point cloud information of the key preset sub-landing area; the true height of the point cloud collection point is less than the true height of the image acquisition point corresponding to the preset landing area; the above-mentioned safe point cloud collection point is a point cloud collection point with no obstacles within a distance with a preset safety distance as a radius and a circle centered at the point cloud collection point.

[0011] S700: If the point cloud collection point of the key preset sub-landing area is a safe point cloud collection point, a three-dimensional laser point cloud map of the key preset sub-landing area is obtained at the point cloud collection point.

[0012] S800: Obtain the slope of the plane corresponding to the key preset sub-landing area based on the three-dimensional laser point cloud map.

[0013] S900: If the slope of the plane corresponding to the key preset sub-landing area is less than the preset plane slope threshold, control the target UAV to land in the key preset sub-landing area.

[0014] According to a second aspect of the present application, a non-transitory computer-readable storage medium is provided, in which at least one instruction or at least one program is stored. The at least one instruction or at least one program is loaded and executed by a processor to implement the above-mentioned file detection method.

[0015] According to a third aspect of the present application, an electronic device is provided, comprising a processor and the above-mentioned non-transitory computer-readable storage medium.

[0016] This application has at least the following beneficial effects:

[0017] In order to accurately determine a landing point for a drone in an unfamiliar environment, the present application first obtains the current position information PM of the target drone upon receiving a target landing command. Then, a first distance between the current position and each preset landing zone is obtained, and each first distance is sorted in ascending order to obtain a first distance set D1. To enable the drone to quickly respond to the landing command, the preset landing zone corresponding to the smallest first distance in D1 is determined as the target preset landing zone. Here, a plurality of preset landing zones are obtained using an image acquisition device pre-installed on the target drone. A preset landing zone is an area corresponding to an image containing a preset area type, i.e., a preset area type (such as an open space or road) in which the drone can land. The preset landing zone closest to the current position of the target drone is selected from the plurality of preset landing zones as the target preset landing zone. After the target preset landing zone is initially determined, since not all locations in the target preset landing zone are suitable for landing, in order to enable the drone to land quickly, the preset sub-landing zone within the target preset landing zone with the smallest distance from PM (i.e., closest to the current position of the target drone) is selected as the key preset sub-landing zone. However, since the acquired image is planar and the shooting altitude is high, the image accuracy is low, and it is impossible to accurately determine the actual situation of the key preset sub-landing area near the ground. Therefore, in order to further determine whether the actual situation of the key preset sub-landing area near the ground is suitable for drone landing, this application needs to reach a point cloud collection point above the key preset sub-landing area and at a preset height from the ground where the preset sub-landing area is located to obtain point cloud information of the key preset sub-landing area, and obtain a three-dimensional laser point cloud map based on the point cloud information for further judgment. However, there may be obstacles near the preset point cloud collection point, which will prevent the drone from obtaining point cloud information or even damage the drone. Therefore, First, the target drone is controlled to move to a point cloud collection point in a key pre-set sub-landing zone. Using a pre-installed radar detection sensor, the point cloud collection point in the key pre-set sub-landing zone is determined to be a safe point cloud collection point. Specifically, the point cloud collection point is detected to determine if there are no obstacles within a circle centered at the point cloud collection point and with a preset safety distance as its radius. If there are no obstacles, the point cloud collection point is considered a safe point cloud collection point. After the point cloud collection point is determined to be a safe point cloud collection point, a three-dimensional laser point cloud map of the key pre-set sub-landing zone is acquired at the point cloud collection point. This identification of the point cloud collection point as a safe point cloud collection point ensures the safety of the target drone during operation and, in the absence of obstructions, provides a more accurate three-dimensional laser point cloud map. Based on this three-dimensional laser point cloud map, a plane fitting is performed on the key pre-set sub-landing zone. The ground slope of the key pre-set sub-landing zone is determined. If the slope is less than a preset slope threshold, the ground slope in the key pre-set sub-landing zone is relatively flat and suitable for landing. The target drone is then controlled to land in the key pre-set sub-landing zone.

[0018] In order to accurately determine the landing point of a drone when autonomously landing in an unfamiliar environment, the present application first selects the closest preset landing area from images collected by the drone at a relatively high altitude as the target preset landing area. Then, within the target preset landing area, a key preset sub-landing area closest to the drone's current position is determined. This reduces the distance the target drone needs to travel, allows it to get closer to the target command issuance point, and quickly prepare for landing. Furthermore, to avoid inaccurate judgments about the actual conditions of the key preset sub-landing area due to the low image clarity corresponding to the target preset landing area and the image's inability to determine the actual scene, the present application obtains a three-dimensional point cloud map around the key preset sub-landing area by arriving at a point cloud collection point, and uses the three-dimensional point cloud information to determine the actual conditions near the ground in the key preset sub-landing area. Before obtaining the three-dimensional point cloud map, it is necessary to first determine whether the point cloud collection point is a safe point cloud collection point. This is to ensure that the three-dimensional point cloud information obtained by the target drone at the point cloud collection point is relatively accurate and comprehensive, and to ensure the safety of the target drone. Finally, after determining that the point cloud collection point is a safe point cloud collection point, the obtained three-dimensional point cloud map is used to determine whether the plane slope corresponding to the key preset sub-landing area is suitable for drone landing. By using a three-dimensional point cloud map generated from a point cloud acquisition point at a lower altitude than the image acquisition point corresponding to the preset landing area, it is possible to more accurately determine the plane slope corresponding to the key preset sub-landing area, and thus more accurately determine whether the key preset sub-landing area is suitable for drone landing. In summary, the autonomous drone landing method provided by this application accurately determines the landing point for the drone in an unfamiliar environment, ensuring the drone's rapid, safe, and stable landing. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A flowchart of a method for autonomous landing of a drone provided in an embodiment of the present application;

[0021] Figure 2 A flowchart for obtaining a three-dimensional point cloud map provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0023] like Figure 1 A method for autonomous landing of a drone according to one embodiment of the present application is shown, comprising the following steps:

[0024] S100, in response to receiving the target landing instruction, obtain the current position information PM of the target UAV = (lng m ,lat m ); among them, lng m The current longitude information of the target drone; lat m The current latitude information of the target UAV.

[0025] Specifically, the target landing instruction is an instruction instructing the target UAV to land.

[0026] S200, according to PM, obtain the first distance between the current position and each preset landing zone to obtain a first distance set D1 = (D11, D12, ..., D1a, ..., D1b); a = 1, 2, ..., b; where D1a is the first distance between the current position ranked a and the image acquisition point corresponding to the preset landing zone; D1a = 2arcsin(sin 2 (lat m -lat a ) / 2)+cos(lat m )*cos(lat a )*sin 2 ((lng m -lng a ) / 2)) 1 / 2 *dc;D1a<D1(a+1);lat a The latitude information of the image acquisition point corresponding to the preset landing area corresponding to D1a; lng a is the longitude information of the image acquisition point corresponding to the preset landing zone corresponding to D1a; dc is the radius of the Earth's equator; b is the number of preset landing zones; the image acquisition point is the shooting position when the image acquisition device preset by the target UAV takes the image corresponding to the preset landing zone; the preset landing zone is the area within the target UAV's working area corresponding to the image containing the preset area type; the preset area type is the area type where the UAV can land.

[0027] Specifically, the above-mentioned preset landing zone is obtained through the following steps:

[0028] S201, at each set image acquisition time point, an initial image set S = (S1, S2, ..., Sx, ..., Sy) of the target UAV when flying in a preset flight state on the target flight path is acquired; x = 1, 2, ..., y; y is the number of initial images acquired when the target UAV flies in the preset flight state on the target flight path; Sx is the xth initial image acquired when the target UAV flies in the preset flight state on the target flight path; the image coverage area corresponding to Sx is a rectangle; the area STx of the image coverage area corresponding to Sx meets the following condition: STx = ζ * (tan (fov) * hx) 2 ζ is the preset coverage parameter; hx is the true height of the target UAV when acquiring the xth initial image; fov is half of the field of view of the image acquisition device of the target UAV; the preset flight state is the flight state in which the true height of the target UAV remains unchanged or increases during the flight; any two consecutive initial images have at least one overlapping point; the target flight path is the flight path of the target UAV before receiving the target landing command; the initial image acquisition time interval Fx corresponding to Sx and S(x-1) meets the following conditions: Fx=(τ*tan(fov)*h (x-1) ) / v (x-1) ;v (x-1) is the flight speed of the target UAV when the x-1th initial image is obtained while the target UAV is flying in the preset flight state on the target flight path; τ is the preset time parameter;

[0029] S202 , screening S according to a preset image recognition algorithm, and determining an area corresponding to an image containing a preset area type as a preset landing area.

[0030] Here, the target drone's image acquisition device (such as a camera) is located directly below the target drone. This device has a fixed field of view (FOV). H is the target drone's current true altitude, which is the vertical distance from the target drone to the area directly below it. The target drone's current true altitude can be determined by using a laser radar (LIDAR) to emit a downward beam and record the return time. The image captured by the target drone's image acquisition device is rectangular.

[0031] As an example, when the imaging ratio of the target drone's image acquisition device (camera, etc.) is 3:4,

[0032] Since tan((fov)=(QM / 2) / h x ;

[0033] So QM=tan((fov)*h x *2;

[0034] Furthermore, STx=PQ*PM=(12 / 25)*QM 2 =(12 / 25)*(tan((fov)*h x *2) 2 =(48 / 25)*(tan((fov)*h x ) 2 .

[0035] Wherein, PQ is the length of the image coverage area (parallel to the velocity direction when the image coverage area is acquired); PM is the width of the image coverage area; QM is the diagonal of the image coverage area;

[0036] Fx=PQ / v=((4 / 5)*QM) / v=(8*tan(fov)*h (x-1) ) / 5v (x-1) =(8 / 5)*tan(fov)*h (x-1) ) / v (x-1) ;

[0037] Therefore, Fx and h (x-1) is positively correlated; f and v (x-1) Is negatively correlated.

[0038] Here, Fx changes with the current flight speed and true altitude of the target drone, so any two consecutive initial images have at least one overlapping point, thus ensuring a high degree of continuity in the acquired images. For example, if the target drone maintains a constant altitude, speed, and direction, several consecutive initial images will be acquired, i.e., the wide edges of two adjacent initial images will overlap.

[0039] It should be noted that in this application, when the drone accelerates downward, the true altitude decreases and the speed increases, and the acquired images may be discontinuous. In order to ensure the continuity of the acquired images, this application does not acquire the initial image when the drone flies downward.

[0040] S203: Screen S according to a preset image recognition algorithm, and determine an area corresponding to an image containing a preset area type as a preset landing area.

[0041] Here, the preset area type can be a road area, an open space area, or other area suitable for drone landing. Images containing the preset area type are screened from the initial images using a preset image recognition algorithm, and the corresponding captured area is used as the preset landing zone. The preset image recognition algorithm is determined by those skilled in the art and is capable of screening the initial images containing the preset area type from the initial images.

[0042] In this embodiment, in order to ensure a high degree of continuity in the images acquired by the target UAV on the target flight path, an appropriate frequency is set to ensure that when flying in a preset flight state, any two consecutively acquired initial images have at least one overlapping point, so that the number of subsequently acquired preset landing areas is larger, thereby facilitating the provision of multiple landing areas for the target UAV.

[0043] S300: Determine the preset landing zone corresponding to the smallest first distance in D1 as the target preset landing zone.

[0044] Specifically, in order to reduce the moving distance of the target UAV and enable it to land quickly, a preset landing area closest to the current position of the target UAV is selected from among several preset landing areas as the target preset landing area.

[0045] S400: Obtain the second distance between each preset sub-landing area of the target preset landing area and the PM to obtain a second distance set D2 = (D21, D22, ..., D2c, ..., D2d); c = 1, 2, ..., d; D2c is the second distance between the geometric center point of the preset sub-landing area ranked c and the PM; D2c = 2arcsin(sin 2 (lat m -lat c ) / 2)+cos(lat m )*cos(lat c )*sin 2 ((lng m -lng c ) / 2)) 1 / 2 *dc; D2a<D2(a+1); where lat c lng is the latitude information of the geometric center point corresponding to the preset sub-landing area corresponding to D2c; c is the longitude information of the geometric center point of the preset sub-landing zone corresponding to D2c; d is the number of preset sub-landing zones in the target preset landing zone; a preset sub-landing zone is an area determined according to the preset area type in the preset landing zone;

[0046] Specifically, after the target area is determined, each target area also includes a number of preset sub-landing areas, which are obtained through the following steps:

[0047] S401 , pre-processing the image corresponding to the preset landing zone of the target and inputting it into a preset target detection model to obtain a plurality of preset sub-landing zones; wherein each preset sub-landing zone has the same area and shape; and no two preset sub-landing zones overlap.

[0048] Here, the preset target detection model can be a deep reinforcement learning network model based on a target detection algorithm. The input of the preset target detection model is a preprocessed image, and the output is an image with predefined sub-landing zone labeled boxes. Furthermore, the preset target detection model can be trained for classification based on the type of the target predefined landing zone. For example, the target predefined landing zone can be a forest, a plain, or a river. Anchor box labeling pre-training is performed for different types of predefined sub-landing zones. The loss function of the preset target detection model can be a Focal Loss loss function, which can alleviate the problem of class imbalance in target detection and improve model performance.

[0049] Here, the training and deployment process of the preset target detection model is as follows:

[0050] (1) Data preparation: Collect and prepare pre-processed images and annotated datasets for training and testing (the images include: aerial drone images of different scenes such as forests, streets, roads, grasslands, and rivers. The datasets need to be diverse and large in number). The annotated datasets should include the category labels of landable targets (streets, open spaces, etc.) and bounding box information (preset sub-landing areas, non-preset sub-landing areas);

[0051] (2) Model training: The model is trained using the prepared dataset. The training process includes inputting the preprocessed images into the model, calculating the loss function, and performing backpropagation to optimize the model parameters.

[0052] (3) Model evaluation: Use an independent test dataset to evaluate the performance of the trained model. Evaluation metrics typically include precision, recall, and mean average precision (mAP).

[0053] (4) Model deployment: The trained model is deployed on the computing device of the target UAV and transmitted back in real time using the cloud server to ensure that the computing device has sufficient computing power to perform real-time target detection.

[0054] The workflow of the preset object detection model is as follows: the input pre-processed image is implicitly divided into S×S grid cells. Each grid is responsible for detecting objects whose center points fall within the grid. The position coordinates and confidence of the bounding box are then predicted. Finally, the non-maximum suppression algorithm is used to select the optimal bounding box. The optimal bounding box selection process is as follows:

[0055] (1) Sort the confidence scores of all predicted bounding boxes and select the box with the highest confidence score and its corresponding box;

[0056] (2) Traverse the remaining bounding boxes. If the overlapping area (IOU) between a certain bounding box and the current bounding box with the highest confidence is greater than the preset IOU threshold (the commonly used value is around 0.5), the bounding box is deleted.

[0057] (3) Select the one with the highest confidence from the unprocessed boxes and repeat (1)-(2) above.

[0058] Here, IOU is calculated based on the maximum value of the horizontal and vertical coordinates of the left top corners of the two bounding boxes, and the minimum value of the horizontal and vertical coordinates.

[0059] The image preprocessing includes the following steps:

[0060] S4010, adjusting the pixel values of the image corresponding to the target preset landing zone to preset pixel values to obtain a first processed image;

[0061] Here, as an example, the preset pixel value may be 416*416 pixels; the specific processing process is as follows:

[0062] (1) Assuming that the size of the original image is (W, H), the size of the target image is (W', H'), and the coordinates of the target pixel are (x', y'), calculate the value of the target pixel as follows:

[0063] (2) Calculate the position of the target pixel in the original image:

[0064] x=x'*(W-1) / (W'-1)

[0065] y=y'*(H-1) / (H'-1)

[0066] (3) Determine the coordinates of the four neighboring pixels around the target pixel:

[0067] (x1,y1)=(floor(x),floor(y))#upper left corner pixel

[0068] (x2,y2)=(ceil(x),floor(y))#upper right corner pixel

[0069] (x3,y3)=(floor(x),ceil(y))#lower left corner pixel

[0070] (x4,y4)=(ceil(x),ceil(y))#lower right corner pixel

[0071] (4) Calculate the distance between the target pixel and the four neighboring pixels:

[0072] dx=x-floor(x)

[0073] dy=y-floor(y)

[0074] (5) Perform weighted average of the values of four adjacent pixels:

[0075] value=(1-dx)(1-dy)I(x1,y1)+dx(1-dy)I(x2,y2)+(1-dx)dyI(x3,y3)+dxdyI(x4,

[0076] y4)

[0077] Where I(x,y) represents the pixel value at coordinate (x,y) in the original image.

[0078] (6) Adjust the original image according to value to obtain the size corresponding to the target image.

[0079] S4011: Perform image enhancement on the first processed image to obtain a second processed image.

[0080] Here, we can define a 3x3 sliding window and place it over each pixel in the image to obtain the pixel values within the window. We then sort the pixel values within the window from smallest to largest or from largest to smallest. We then find the middle value (the median) of the sorted pixel values and use it as the new value for the current pixel, replacing the original pixel value to complete the image enhancement.

[0081] S4012: Standardize the second processed image to complete preprocessing.

[0082] Here, the specific process of standardization is as follows:

[0083] (1) Calculate the mean of each pixel channel:

[0084] meanR=sum(R) / n

[0085] meanG=sum(G) / n

[0086] meanB=sum(B) / n

[0087] Where sum(R), sum(G), and sum(B) represent the sets of red, green, and blue channel pixel values in the image, respectively, and n is the total number of pixel values;

[0088] (2) Calculate the standard deviation of each channel:

[0089] std R=sqrt(sum((R-mearR)2) / n)

[0090] std G=sqrt(sum((G-mearG)2) / n)

[0091] std B=sqrt(sum((B-mearB)2) / n)

[0092] (3) Normalize each channel of each pixel:

[0093] normalized R=(R-mearR) / std R

[0094] normalized G=(R-mearG) / std G

[0095] normalized B=(R-mearB) / std B

[0096] Among them, normalized R, normalized G, and normalized B are the normalized pixel values of the corresponding channels;

[0097] (4) Calculate the mean and standard deviation of each channel, and subtract the mean of the corresponding channel from each pixel value and divide it by the standard deviation of the corresponding channel to achieve the normalization operation of the RGB three-channel image. The processed image will have a distribution with a mean of 0 and a standard deviation of 1 in each channel.

[0098] S500: Determine the preset sub-landing zone corresponding to the smallest second distance in D2 as a key preset sub-landing zone.

[0099] Specifically, in order to reduce the moving distance of the target UAV and enable it to land quickly, the preset sub-landing area closest to the current position of the target UAV is selected as the key preset sub-landing area from the multiple preset sub-landing areas included in the target preset landing area.

[0100] S600: Control the target UAV to move to a point cloud collection point in a key preset sub-landing area, and determine whether the point cloud collection point in the key preset sub-landing area is a safe point cloud collection point based on a preset radar detection sensor; the above-mentioned point cloud collection point is a position above the key preset sub-landing area, with a true height from the ground where the key preset sub-landing area is located being a preset true height, and capable of obtaining complete point cloud information of the key preset sub-landing area; the true height of the point cloud collection point is less than the true height of the image acquisition point corresponding to the preset landing area; the above-mentioned safe point cloud collection point is a point cloud collection point with no obstacles within a distance with a preset safety distance as a radius and a circle centered at the point cloud collection point.

[0101] Specifically, because the images acquired by the aforementioned method are planar and the image accuracy is low due to the high altitude, it is impossible to accurately determine the actual ground conditions near the key predetermined sub-landing zone. Therefore, to further determine whether the ground conditions near the key predetermined sub-landing zone are suitable for drone landing, it is necessary to reach a point cloud collection point above the key predetermined sub-landing zone and at a preset height from the ground where the predetermined sub-landing zone is located to obtain point cloud information for the key predetermined sub-landing zone. A 3D laser point cloud map is then generated from this point cloud information for further determination. However, obstacles may exist near this preset point cloud collection point, which could hinder the drone's acquisition of point cloud information or even damage the drone. Therefore, the target drone is first controlled to move to the point cloud collection point in the key predetermined sub-landing zone. The preset radar detection sensor then determines whether the point cloud collection point in the key predetermined sub-landing zone is a safe point cloud collection point. Specifically, the radar sensor detects whether there are no obstacles within a radius of a preset safety distance centered on the point cloud collection point. If there are no obstacles, the point cloud collection point is considered a safe point cloud collection point. Here, the preset safety distance is 3-5 times the radius of the drone body.

[0102] S700: If the point cloud collection point of the key preset sub-landing area is a safe point cloud collection point, a three-dimensional laser point cloud map of the key preset sub-landing area is obtained at the point cloud collection point.

[0103] Specifically, the process of obtaining the 3D point cloud map is shown in Figure 2 .

[0104] Point cloud information is obtained based on the laser radar and inertial measurement unit (IMU), and then the process of dedistorting the feature point cloud is performed. Laser mapping is performed based on the laser odometry and GPS, and finally a three-dimensional laser point cloud map is obtained.

[0105] Here, the three-dimensional point cloud map obtained by the point cloud acquisition point at a lower altitude than the image acquisition point corresponding to the preset landing area is more accurate because the acquisition point is lower. In addition, the three-dimensional point cloud map is three-dimensional compared to the two-dimensional image and can better reflect the undulation and flatness of the ground corresponding to the key preset sub-landing area.

[0106] S800: Obtain the slope of the plane corresponding to the key preset sub-landing area based on the three-dimensional laser point cloud map.

[0107] Specifically, the plane slope α corresponding to the key preset sub-landing area meets the following conditions:

[0108] α=arccos(v T up n)

[0109] Among them, v up is the vector of the z-axis in the three-dimensional rectangular coordinate system; v up=(0,0,-1); T is the vector transpose; n is the vector of the plane normal of the plane corresponding to the key preset sub-landing area.

[0110] Here, plane fitting can be performed from the point cloud based on the LOAM algorithm; the specific process is:

[0111] (1) Determine the data set for plane fitting (3 points)

[0112] (2) Perform plane fitting based on the data set.

[0113] S900: If the slope of the plane corresponding to the key preset sub-landing area is less than the preset plane slope threshold, control the target UAV to land in the key preset sub-landing area.

[0114] Specifically, if the plane slope corresponding to the key preset sub-landing area obtained by plane fitting is less than the preset plane slope threshold, it means that the ground slope of the key preset sub-landing area is small and relatively flat, and the drone can land. The target drone is then controlled to land in the key preset sub-landing area.

[0115] It should be noted that in order to improve the landing stability of the target drone, the landing process of the target drone is as follows:

[0116] The target drone has four adjustable legs mounted on the bottom of the quadcopter, capable of rotating and extending at various angles. Upon receiving a tripod-opening signal, the legs are commanded to unfold in the direction of gravity. The target drone's acceleration and altitude sensors are detected, and shock-absorbing pads are installed on the outside of the support frame. Current position information is obtained based on the fitted plane. The diagonal legs are deployed accordingly, while the other two legs remain stationary. Landing is considered complete when both the acceleration and altitude sensors are at zero, and the target drone's level indicator indicates the angle at which it would roll over. The target drone is equipped with a landing detection sensor to detect contact with the ground. Once the target drone successfully lands, the detection system sends a feedback signal to the control system, confirming landing completion. The coordinates of the landing location are then fed back to the control terminal.

[0117] In an exemplary embodiment of the present application, after step S600, the method further includes:

[0118] S710. If the point cloud collection point of the key preset sub-landing zone is a non-safe point cloud collection point, obtain the third distance between the key preset sub-landing zone and each preset sub-landing zone in the target preset landing zone except the key preset sub-landing zone, and obtain a third distance set D3 = (D31, D32, ..., D3g, ..., D3z); g = 1, 2, ..., z; D3g is the third distance between the geometric center point of the preset sub-landing zone ranked at the gth position and the geometric center point of the key preset sub-landing zone; D3g = 2arcsin(sin 2 (lat m -lat g ) / 2)+cos(lat m )*cos(lat g )*sin 2 ((lng m -lng g ) / 2)) 1 / 2 *dc; D3g<D3(g+1); where lat g lng is the latitude information of the geometric center point corresponding to the preset sub-landing area corresponding to D3g; g is the longitude information of the geometric center point of the preset sub-landing zone corresponding to D3g; z is the number of preset sub-landing zones in the target preset landing zone excluding the key preset sub-landing zones;

[0119] S710: Update the preset sub-landing zone corresponding to the smallest third distance in D3 as the key preset sub-landing zone, and proceed to step S600.

[0120] Specifically, if the point cloud collection point of the key preset sub-landing zone is a non-safe point cloud collection point, to ensure the safe landing of the target UAV, it will not land there, but will change to another preset sub-landing zone for landing. At this time, the target UAV has moved above the geometric center of the key preset sub-landing zone, so the third distance between the key preset sub-landing zone and each preset sub-landing zone in the target preset landing zone except the key preset sub-landing zone is taken, and the preset sub-landing zone corresponding to the smallest third distance is updated as the new key preset sub-landing zone (that is, the preset sub-landing zone closest to the current position of the target UAV is selected); then jump to step S600 to further determine whether the updated key preset sub-landing zone can be landed.

[0121] In this embodiment, when the point cloud collection point of the key preset sub-landing area is a non-safe point cloud collection point, other preset sub-landing areas closest to the current position are selected for the target drone to screen the landing conditions and determine whether it can land, thereby ensuring the safety of the target drone.

[0122] In an exemplary embodiment of the present application, after step S800, the method further includes:

[0123] At step S910, if the plane slope corresponding to the key preset sub-landing zone obtained by plane fitting is greater than or equal to the preset plane slope threshold, then obtain the third distance between the key preset sub-landing zone and each preset sub-landing zone in the target preset landing zone except the key preset sub-landing zone, and sort each third distance in ascending order to obtain a third distance set D3 = (D31, D32, ..., D3g, ..., D3z), and proceed to step S710.

[0124] Here, if the plane slope corresponding to the key preset sub-landing area obtained by plane fitting is greater than or equal to the preset plane slope threshold, it means that the ground slope of the key preset sub-landing area is too steep and not flat enough, and the target drone cannot land stably. Therefore, it will not land there and needs to change to another preset sub-landing area for landing. At this time, the target drone has moved above the geometric center of the key preset sub-landing area. Therefore, the third distance between the key preset sub-landing area and each preset sub-landing area in the target preset landing area except the key preset sub-landing area is calculated, and the process proceeds to step S710: the preset sub-landing area corresponding to the smallest third distance is updated as the new key preset sub-landing area (i.e., the preset sub-landing area closest to the current position of the target drone is selected); then the process jumps to step S600 to further determine whether the updated key preset sub-landing area is suitable for landing.

[0125] It should be noted that the first distance set, the second distance set, and the third distance set can all be sorted using a quick sorting algorithm, and the sorting process is as follows:

[0126] (1) First, set a dividing value to divide the array into two parts, left and right.

[0127] (2) The data that is greater than or equal to the cutoff value is concentrated on the right side of the array, and the data that is less than the cutoff value is concentrated on the left side of the array. At this time, all elements in the left part are less than the cutoff value, and all elements in the right part are greater than or equal to the cutoff value.

[0128] (3) Then, the data on the left and right can be sorted independently. For the array data on the left, a demarcation value can be used to divide the data into two parts, with the smaller value on the left and the larger value on the right. The array data on the right can also be processed similarly.

[0129] (4) Repeating the above process, we can see that this is a recursive definition. After recursively sorting the left part, we recursively sort the right part. Once the data in the left and right parts are sorted, the sorting of the entire array is complete.

[0130] In this embodiment, when the plane slope corresponding to the key preset sub-landing area is greater than or equal to the preset plane slope threshold, the target drone is selected from other preset sub-landing areas closest to the current location to screen landing conditions and determine whether it can land, thereby ensuring the safety of the target drone.

[0131] An embodiment of the present application further provides a computer program product, which includes program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the method according to various exemplary embodiments of the present application described above in this specification.

[0132] Furthermore, although the steps of the method of the present disclosure are described in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in this particular order, or that all steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0133] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0134] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0135] Those skilled in the art will appreciate that various aspects of the present application can be implemented as systems, methods, or program products. Therefore, various aspects of the present application can be specifically implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, which may be collectively referred to herein as a "circuit," "module," or "system."

[0136] The electronic device according to this embodiment of the present application is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0137] The electronic device is implemented as a general-purpose computing device. Components of the electronic device may include, but are not limited to, the aforementioned at least one processor, the aforementioned at least one storage, and a bus connecting different system components (including the storage and the processor).

[0138] The storage stores program codes, which can be executed by the processor, so that the processor executes the steps described in the above “Exemplary Method” section of this specification according to various exemplary embodiments of the present application.

[0139] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read only memory (ROM).

[0140] The storage may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0141] The bus may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0142] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter. As shown, the network adapter communicates with other modules of the electronic device via a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0143] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0144] In exemplary embodiments of the present disclosure, a computer-readable storage medium is also provided, on which is stored a program product capable of implementing the aforementioned methods of this specification. In some possible implementations, various aspects of this application may also be implemented in the form of a program product, which includes program code. When the program product is executed on a terminal device, the program code is used to cause the terminal device to execute the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present application.

[0145] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0146] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0147] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0148] The program code for performing the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, and the like, as well as conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0149] Furthermore, the above-mentioned figures are merely illustrative of the processes included in the methods according to exemplary embodiments of the present application and are not intended to be limiting. It is readily understood that the processes illustrated in the above-mentioned figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0150] It should be noted that although several modules or units of the device for action execution are mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0151] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for autonomous landing of a drone, characterized in that: include: S100, in response to receiving the target landing instruction, obtain the current position information PM of the target UAV = (lng m ,lat m ); where lng m The current longitude information of the target drone; lat m The current latitude information of the target UAV; S200, according to PM, obtain the first distance between the current position and each preset landing zone to obtain a first distance set D1 = (D11, D12, ..., D1a, ..., D1b); a = 1, 2, ..., b; where D1a is the first distance between the current position ranked a and the image acquisition point corresponding to the preset landing zone; D1a = 2arcsin(sin 2 (lat m -lat a ) / 2)+cos(lat m )×cos(lat a )×sin 2 ((lng m -lng a ) / 2)) 1 / 2 ×dc; D1a<D1 (a+1); lat a The latitude information of the image acquisition point corresponding to the preset landing area corresponding to D1a; lng a is the longitude of the image acquisition point corresponding to the preset landing zone corresponding to D1a; dc is the radius of the Earth's equator; b is the number of preset landing zones; the image acquisition point is the location where the image acquisition device preset by the target UAV captures the image corresponding to the preset landing zone; the preset landing zone is the area within the target UAV's operating area corresponding to the image containing the preset area type; the preset area type is the type of area where the UAV can land; S300, determining the preset landing zone corresponding to the smallest first distance in D1 as the target preset landing zone; S400: Obtain the second distance between each preset sub-landing area of the target preset landing area and the PM to obtain a second distance set D2 = (D21, D22, ..., D2c, ..., D2d); c = 1, 2, ..., d; D2c is the second distance between the geometric center point of the preset sub-landing area ranked c and the PM; D2c = 2arcsin(sin 2 (lat m -lat c ) / 2)+cos(lat m ) ×cos(lat c ) × sin 2 ((lng m -lng c ) / 2)) 1 / 2 ×dc; D2a<D2(a+1); where lat c lng is the latitude information of the geometric center point corresponding to the preset sub-landing area corresponding to D2c; c is the longitude information of the geometric center point of the preset sub-landing zone corresponding to D2c; d is the number of preset sub-landing zones in the target preset landing zone; a preset sub-landing zone is an area determined according to the preset area type in the preset landing zone; S500, determining the preset sub-landing zone corresponding to the smallest second distance in D2 as the key preset sub-landing zone; S600: Control the target drone to move to a point cloud collection point in a key preset sub-landing zone, and determine, using a preset radar detection sensor, whether the point cloud collection point in the key preset sub-landing zone is a safe point cloud collection point. The point cloud collection point is a location above the key preset sub-landing zone, at a preset true height from the ground where the key preset sub-landing zone is located, and capable of acquiring complete point cloud information for the key preset sub-landing zone. The true height of the point cloud collection point is less than the true height of the image acquisition point corresponding to the preset landing zone. The safe point cloud collection point is a point cloud collection point free of obstacles within a circle centered at the point cloud collection point and with a preset safety distance as a radius. S700: If the point cloud collection point of the key preset sub-landing area is a safe point cloud collection point, obtain a three-dimensional laser point cloud map of the key preset sub-landing area at the point cloud collection point; S800: Obtaining the slope of a plane corresponding to a key preset sub-landing area based on the three-dimensional laser point cloud map; S900: If the slope of the plane corresponding to the key preset sub-landing area is less than the preset plane slope threshold, control the target UAV to land in the key preset sub-landing area.

2. The autonomous landing method of a UAV according to claim 1, characterized in that: The preset landing zone is obtained by the following steps: S201, every time the set image acquisition time point is reached, an initial image set S=(S1, S2, ..., Sx, ..., Sy) of the target UAV when it flies in the preset flight state on the target flight path is acquired; x=1, 2, ..., y; y is the number of initial images acquired when the target UAV flies in the preset flight state on the target flight path; Sx is the x-th initial image acquired when the target UAV flies in the preset flight state on the target flight path; the image coverage area corresponding to Sx is a rectangle; the area STx of the image coverage area corresponding to Sx meets the following conditions: STx=ζ×(tan(fov) ×h x ) 2 ζ is the preset coverage parameter; h x is the true height of the target UAV when acquiring the xth initial image; fov is half of the field of view of the image acquisition device of the target UAV; the preset flight state is the flight state in which the true height of the target UAV remains unchanged or increases during the flight; any two consecutive initial images have at least one overlapping point; the target flight path is the flight path of the target UAV before receiving the target landing command; the initial image acquisition time interval Fx corresponding to Sx and S(x-1) meets the following conditions: Fx=(τ×tan(fov)×h (x-1) ) / v (x-1) ;v (x-1) is the flight speed of the target UAV when the x-1th initial image is obtained while the target UAV is flying in the preset flight state on the target flight path; τ is the preset time parameter; S202 , screening S according to a preset image recognition algorithm, and determining an area corresponding to an image containing a preset area type as a preset landing area.

3. The autonomous landing method of a UAV according to claim 1, characterized in that: The preset sub-landing area is obtained by the following steps: S401 , pre-processing the image corresponding to the preset landing zone of the target and inputting it into a preset target detection model to obtain a plurality of preset sub-landing zones; wherein each preset sub-landing zone has the same area and shape; and no two preset sub-landing zones overlap.

4. The autonomous landing method for a UAV according to claim 3, characterized in that: The pretreatment comprises the following steps: S4010, adjusting the pixel values of the image corresponding to the target preset landing zone to preset pixel values to obtain a first processed image; S4011, performing image enhancement on the first processed image to obtain a second processed image; S4012: Standardize the second processed image to complete preprocessing.

5. The autonomous landing method for a UAV according to claim 1, characterized in that: The preset safety distance is 3-5 times the radius of the drone body.

6. The autonomous landing method for a UAV according to claim 1, characterized in that: The plane slope α corresponding to the key preset sub-landing area meets the following conditions: α=arccos(v) T up n) Among them, v up is the vector of the z-axis in the three-dimensional rectangular coordinate system; v up =(0,0,-1); T is the vector transpose; n is the vector of the plane normal of the plane corresponding to the key preset sub-landing area.

7. The autonomous landing method for a UAV according to claim 1, characterized in that: After step S600, the method further includes: S710. If the point cloud collection point of the key preset sub-landing zone is a non-safe point cloud collection point, obtain the third distance between the key preset sub-landing zone and each preset sub-landing zone in the target preset landing zone except the key preset sub-landing zone, and obtain a third distance set D3=(D31, D32, ..., D3g, ..., D3z); g=1, 2, ..., z; D3g is the third distance between the geometric center point of the preset sub-landing zone ranked at the gth position and the geometric center point of the key preset sub-landing zone; D3g=2arcsin(sin 2 (lat m -lat g ) / 2)+cos(lat m )×cos(lat g ) × sin 2 ((lng m -lng g ) / 2)) 1 / 2 ×dc; D3g<D3(g+1); where lat g lng is the latitude information of the geometric center point corresponding to the preset sub-landing area corresponding to D3g; g is the longitude information of the geometric center point of the preset sub-landing zone corresponding to D3g; z is the number of preset sub-landing zones in the target preset landing zone excluding the key preset sub-landing zones; S720: Update the preset sub-landing zone corresponding to the smallest third distance in D3 as the key preset sub-landing zone, and proceed to step S600.

8. The autonomous landing method for a UAV according to claim 7, characterized in that: After step S800, the method further includes: At step S910, if the plane slope corresponding to the key preset sub-landing zone obtained by plane fitting is greater than or equal to the preset plane slope threshold, then obtain the third distance between the key preset sub-landing zone and each preset sub-landing zone in the target preset landing zone except the key preset sub-landing zone, obtaining a third distance set D3 = (D31, D32, ..., D3g, ..., D3z), and proceed to step S720.

9. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the storage medium, characterized in that: The at least one instruction or the at least one program is loaded and executed by a processor to implement the method according to any one of claims 1 to 8.

10. An electronic device, characterized in that: The device comprises a processor and the non-transitory computer-readable storage medium as claimed in claim 9.

Images