Method for autonomously selecting landing area for unmanned aerial vehicle, unmanned aerial vehicle, and computer-readable storage medium

Abstract

A method for autonomously selecting a landing area for a drone, a drone, and a computer-readable storage medium. The method comprises: obtaining a first ground image captured by a drone's camera, inputting the first ground image into a pre-trained image segmentation network to obtain a second ground image including a pixel region corresponding to a landing area; obtaining point cloud data collected by a laser radar, and projecting each point in the point cloud data onto the first ground image; obtaining a point cloud image of the landing area based on target points located in the pixel region of the second ground image after projection; determining a target landing area based on the point cloud image; and controlling the drone to land in the target landing area. This method can increase the accuracy and controllability of the drone's autonomous selection of a target landing area.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of drone technology, and in particular to a method for a drone to autonomously select a landing area, a drone, and a computer-readable storage medium. Background Art

[0002] During flight, drones may need to make emergency landings in many situations, such as low battery or adverse flight conditions. These situations often require landing in unplanned locations, often in uncharted and unstructured areas. When selecting a landing area, the drone must autonomously locate a sufficiently large, flat, and suitable landing surface. Generally, flat, hard surfaces, such as concrete or asphalt roads, are suitable, while rough or soft surfaces, such as mountains or water, are not suitable. Failure to select a suitable landing site can result in a failed landing and subsequent damage to the drone.

[0003] In the existing technology, drones are usually required to identify markers set in the landing area in order to control the landing of the drone. This relies on the markers to be identified. When the drone encounters an emergency or an unstructured environment and needs to land in an environment without markers, its practicality is limited. Summary of the Invention

[0004] The embodiments of the present application provide a method for a drone to autonomously select a landing area, a drone, and a computer-readable storage medium. The method does not require any markers and enables the drone to autonomously select a landing area based on camera images and laser radar point cloud data.

[0005] An embodiment of the present application provides a method for a drone to autonomously select a landing area, including:

[0006] Obtaining a first ground image captured by a camera of a drone, and inputting the first ground image into a pre-trained image segmentation network to obtain a second ground image including a pixel region corresponding to the landing area;

[0007] Acquire point cloud data collected by a laser radar, and project each point in the point cloud data into the first ground image;

[0008] Obtaining a point cloud image of the landing area according to the target point located in the pixel area of ​​the second ground image after projection;

[0009] A target landing area is determined according to the point cloud image of the landing area, and the UAV is controlled to land in the target landing area.

[0010] In one aspect, an embodiment of the present application further provides a drone, comprising:

[0011] An acquisition module, configured to acquire a first ground image captured by a camera of the UAV;

[0012] a processing module, configured to input the first ground image into a pre-trained image segmentation network to obtain a second ground image including a pixel area corresponding to a landing area;

[0013] The acquisition module is further used to acquire point cloud data collected by the laser radar;

[0014] The processing module is further configured to project each point in the point cloud data into the first ground image, and obtain a point cloud image of the landing area based on target points located in a pixel area in the second ground image after projection;

[0015] The processing module is further configured to determine a target landing area based on the point cloud image of the landing area, and control the UAV to land in the target landing area.

[0016] In one aspect, an embodiment of the present application further provides a drone, comprising:

[0017] memory and processor;

[0018] The memory stores an executable computer program;

[0019] The processor coupled to the memory calls the executable computer program stored in the memory to execute the steps in the above-mentioned method for autonomously selecting a landing area for a drone.

[0020] On the one hand, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for autonomously selecting a landing area for a drone as provided in the above embodiment is implemented.

[0021] It can be seen from the above-mentioned embodiments of the present application that a first ground image taken by a camera of a drone is obtained, and the first ground image is input into a pre-trained image segmentation network to obtain a second ground image including a pixel area corresponding to a landing area, point cloud data collected by a lidar is obtained, and each point in the point cloud data is projected into the first ground image, and a point cloud image of the landing area is obtained based on the target points in the pixel area in the second ground image after projection, and a target landing area is determined based on the point cloud image of the landing area, and the drone is controlled to land in the target landing area. The accuracy and controllability of determining the target landing area can be increased by semantically distinguishing the landing area from the non-landing area in the ground image taken by the camera, so that the drone can autonomously select the target landing area. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] Figure 1 A flowchart of a method for autonomously selecting a landing area for a drone provided in an embodiment of the present application;

[0024] Figure 2 A schematic diagram of a first ground image captured by a camera of a drone provided in an embodiment of the present application;

[0025] Figure 3 for Figure 2 A schematic diagram of a labeled image of the first ground image shown;

[0026] Figure 4 Another schematic diagram of a first ground image captured by a camera of a drone provided in an embodiment of the present application;

[0027] Figure 5 for Figure 4 A schematic diagram of a second image output after a first ground image is input into an image segmentation network;

[0028] Figure 6 A schematic diagram of the structure of a drone provided in one embodiment of the present application;

[0029] Figure 7 A schematic diagram of the hardware structure of a drone provided in one embodiment of the present application. DETAILED DESCRIPTION

[0030] To make the purpose, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0031] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "axial", "radial", "circumferential", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application.

[0032] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integrated connection; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to the specific circumstances. The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0033] See also Figure 1 An embodiment of the present application provides a method for a drone to autonomously select a landing area. The execution subject is the drone, specifically the processor of the drone.

[0034] The UAV is also equipped with a camera, a lidar and an integrated navigation module (specifically, an IMU-RTK module), wherein the camera is used to capture images, the processor can obtain the ground image captured by the camera, and perform semantic segmentation on the ground image to find pixels of the ground type suitable for landing in the ground image, thereby determining the landing area; the lidar is used to provide point cloud data, the points in the point cloud data are points on the ground scanned by the lidar, the processor projects each point in the point cloud data onto the ground image, and if the pixel projected by a point is a ground type suitable for landing, the point is retained; the integrated navigation module is used to provide UAV posture data, which may include: coordinates (lon, lat, alt) in the latitude and longitude (lla) coordinate system and coordinates (roll, pitch, yaw) in the northeast celestial (enu) coordinate system, the processor can convert the coordinates of each retained point from the lidar coordinate system to the earth-centered earth-fixed (ecef) coordinate system, and determine the target landing area of ​​the UAV in the landing area based on the points in the ecef coordinate system corresponding to multiple frames of continuous ground images, and control the UAV to land in the target landing area.

[0035] The method comprises the following steps:

[0036] S101, obtaining a first ground image captured by a camera of a drone, and inputting the first ground image into a pre-trained image segmentation network to obtain a second ground image including a pixel region corresponding to a landing area;

[0037] Before step S101, the following steps are included:

[0038] The camera intrinsic parameters and distortion parameters of the drone, the first extrinsic parameter between the camera and the lidar, and the second extrinsic parameter between the camera and the integrated navigation module are calibrated, and the third extrinsic parameter between the lidar and the integrated navigation module is calculated based on the first extrinsic parameter and the second extrinsic parameter.

[0039] Specifically, the Zhang Zhengyou calibration method is used to collect chessboard images at different angles with the camera, and the OpenCV library is used to solve the problem to obtain the camera intrinsic parameters and distortion parameters f x 、f y 、c x 、c y , k1, k2, p1, p2 and k3;

[0040] Using a fixed calibration plate, the open source software library opencalib is used to calibrate the first external parameter between the camera and the lidar, which is recorded as the rotation matrix R between the camera coordinate system and the world coordinate system. cl and the translation vector t cl ;

[0041] The open source software library kalibr is used to calibrate the second external parameter between the camera and the integrated navigation module: R ic and t ic ;

[0042] Calculate the third external parameter between the lidar and the integrated navigation module:

[0043] R il =R ic R cl

[0044] t il =t ic +R ic t cl

[0045] After acquiring a first ground image captured by the drone's camera, the first ground image is dedistorted using the calibrated camera intrinsic parameters and distortion parameters. Based on the preset landing zone parameters, the image regions corresponding to the landing zone are marked in each frame of the dedistorted first ground image to produce a marked image. If K frames of first ground images are collected, each frame is marked to obtain K "ground image-marked region" data pairs.

[0046] Among them, the landing area parameters may include:

[0047] A hard surface that reaches the preset hardness, i.e. it cannot be woodland, water surface, etc.;

[0048] Reaching a flat ground with a slope less than a preset slope, the preset slope may be, for example, 10 degrees;

[0049] The width, length or area of ​​the ground must meet the required landing conditions for the drone. For example, the minimum width of the landing area must be greater than three times the wingspan of the drone.

[0050] The distance from the surrounding buildings is greater than a preset first distance, and the preset distance is, for example, greater than three times the wingspan of the drone;

[0051] The distance from the high-voltage line device is greater than a preset second distance, that is, far away from the high-voltage line device. The high-voltage line device may include a high-voltage line tower and a high-voltage cable.

[0052] Figure 2 The first ground image is taken by the drone, and may include various areas such as mountains, roads, lakes, and residential areas; Figure 3 for Figure 2 The corresponding marked image is marked with the landing area.

[0053] Furthermore, the marked image marked with the landing area is input into the trained image segmentation network, which can specifically be a U-net neural network. The marked image is used as the input of the U-net neural network. The image output by the U-net neural network is compared with the image of the marked area in the marked image by pixel comparison, and the U-net neural network is trained by backpropagation. When the image output by the network after training is consistent with the image of the marked area, a trained image segmentation network is obtained. Figure 4 Another schematic diagram of the first ground image taken by the camera of the UAV. After the first ground image is input into the image segmentation network, the output is Figure 5 The second ground image is shown with pixel areas corresponding to the landing area marked.

[0054] The pixel area corresponding to the landing area in the second ground image is denoted as {I}

[0055] S102: Acquire point cloud data collected by a laser radar, and project each point in the point cloud data into the first ground image;

[0056] Specifically, the kth point Pk in each frame point cloud l (x kl ,y kl ,z kl ) is projected onto the first ground image captured by the camera, or onto the image obtained by dedistorting the first ground image using the calibrated camera intrinsic parameters and distortion parameters, and the pixel coordinates of the point (u k ,v k );

[0057] P kc (x kc ,y kc ,z kc )=R cl P kl +t cl

[0058]

[0059]

[0060] Among them, P kl is the three-dimensional coordinate of the point in the lidar coordinate system; P kc It is the three-dimensional coordinate of the point in the camera coordinate system.

[0061] S103: Obtain a point cloud image of the landing area based on the target point in the pixel area of ​​the second ground image;

[0062] If the pixel coordinates of the kth point (u k ,v k ) In the pixel area {I} of the second ground image obtained in step 101, the kth point is retained, otherwise the kth point is discarded. After processing the points of a frame of point cloud, several target points {P l},{P l The points in} are the points in the laser radar coordinate system.

[0063] The target point is a point located in the second ground image after each point in the point cloud data is projected onto the first ground image.

[0064] The coordinates of each target point are converted from the coordinates in the laser radar coordinate system to the coordinates in the earth-centered earth-fixed coordinate system, and the point cloud image of the landing area is obtained according to the coordinates in the earth-centered earth-fixed coordinate system.

[0065] Specifically, the coordinates of the target point are converted from the coordinates in the laser radar coordinate system to the coordinates in the integrated navigation module coordinate system;

[0066] Get the latitude and longitude coordinates and northeast celestial coordinates of the integrated navigation module;

[0067] According to the northeast celestial coordinates of the integrated navigation module, the coordinates of the target point in the integrated navigation module coordinate system are converted to the coordinates in the northeast celestial coordinate system;

[0068] The relative position of the target point converted to the coordinates of the Northeast Celestial Coordinate System and the fuselage of the UAV is converted to the coordinates of the Earth-centered Earth-fixed Coordinate System to obtain a coordinate increment;

[0069] Obtain the longitude and latitude coordinates of the UAV, and convert the longitude and latitude coordinates of the UAV into coordinates in the Earth-centered Earth-fixed coordinate system. Add the coordinate increment to the converted Earth-centered Earth-fixed coordinates to obtain the coordinates of the target point in the Earth-centered Earth-fixed coordinate system.

[0070] Specifically, the coordinates P of each target point in the laser radar coordinate system are l (x l ,y l ,z l ) is converted to the combined navigation module coordinate system P b (x b ,y b ,z b )=R il P l +t il ;

[0071] Get the combined navigation module data, that is, get the combined navigation module's latitude and longitude coordinates (lon, lat, alt) and roll, pitch, yaw in the enu coordinate system;

[0072] The coordinates P of the midpoint of the integrated navigation rtk module coordinate system b (x b ,y b ,z b )Convert to the enu coordinate system:

[0073]

[0074] P enu =RP b

[0075] Convert the relative position of the point in the enu coordinate system and the drone's body to the ecef coordinate system to obtain the coordinate increment:

[0076] That is, Δx, Δy, and Δz are obtained as follows:

[0077]

[0078] Among them, Penu represents the relative position of the point in the enu coordinate system and the fuselage of the drone, and this relative position is a known value.

[0079] Convert the latitude and longitude coordinates (lon, lat, alt) of the fuselage itself to the ecef coordinate system, and add the coordinate increment to obtain the coordinate P of the point in the ecef coordinate system ecef :

[0080] X=(N+alt)cos(lat)cos(lon)+Δx

[0081] Y=(N+alt)cos(lat)sin(lon)+Δy

[0082] Z=(N(1+e 2 )+alt)sin(lat)+Δz

[0083] The RTK module is installed on the drone's fuselage. The RTK output, which converts latitude, longitude, and altitude into the ecef coordinates, represents the drone's ecef coordinates. However, the points in the point cloud data are on the ground. Therefore, the distance between the point and the drone must first be converted to the ecef coordinate system, obtaining the coordinate increments Δx, Δy, and Δz mentioned above. These increments are then added to the drone's ecef coordinates to obtain the ecef coordinates of each point.

[0084] Continuously collect m frames (such as 10 frames) of data and obtain {P ecef}1…{P ecef} m , put these points together to get a point cloud as the local landing area.

[0085] S104: Determine a target landing area based on the point cloud image of the landing area, and control the UAV to land in the target landing area.

[0086] The target landing area is determined in the point cloud image of the landing area according to the preset rules for selecting the target landing area. The rule may be to select the landing area with the largest area among multiple landing areas; or the landing area farthest from the building, etc.

[0087] In this embodiment, a first ground image taken by a camera of a drone is obtained, and the ground image is input into a pre-trained image segmentation network to obtain a second ground image including a pixel area corresponding to a landing area. Point cloud data collected by a lidar is obtained, and each point in the point cloud data is projected into the first ground image. Based on each target point located in the pixel area in the second ground image after projection, a point cloud image of the landing area is obtained. Based on the point cloud image of the landing area, a target landing area is determined, and the drone is controlled to land in the target landing area. By semantically distinguishing the landing area from the non-landing area through the ground image taken by the camera, the accuracy and controllability of determining the target landing area are increased, and the drone can autonomously select the target landing area.

[0088] See also Figure 6 , a schematic structural diagram of a drone provided in one embodiment of the present application, the drone comprises:

[0089] An acquisition module 301 is configured to acquire a first ground image captured by a camera of a UAV;

[0090] The processing module 302 is configured to input the first ground image into a pre-trained image segmentation network to obtain a second ground image including a pixel area corresponding to the landing area;

[0091] The acquisition module 301 is also used to acquire point cloud data collected by the laser radar;

[0092] The processing module 302 is further configured to project each point in the point cloud data into the first ground image;

[0093] Obtaining a point cloud image of the landing area according to each target point located in a pixel area of ​​the second ground image after projection;

[0094] The target landing area is determined based on the point cloud image of the landing area, and the UAV is controlled to land in the target landing area.

[0095] Furthermore, the processing module 302 is further configured to convert the coordinates of each target point from the coordinates in the laser radar coordinate system to the coordinates in the Earth-centered Earth-fixed coordinate system;

[0096] A point cloud image of the landing area is obtained according to the coordinates in the Earth-centered Earth-fixed coordinate system.

[0097] Furthermore, the processing module 302 is further configured to convert the coordinates of the target point from the coordinates in the laser radar coordinate system to the coordinates in the integrated navigation module coordinate system;

[0098] Obtaining the latitude and longitude coordinates and northeast celestial coordinates of the integrated navigation module;

[0099] According to the northeast celestial coordinates of the integrated navigation module, the coordinates of each target point in the integrated navigation module coordinate system are converted into coordinates in the northeast celestial coordinate system;

[0100] Converting the relative position of the target point converted to the coordinates in the northeast celestial coordinate system and the fuselage of the UAV into the coordinates in the Earth-centered Earth-fixed coordinate system to obtain a coordinate increment;

[0101] Obtain the longitude and latitude coordinates of the UAV, and convert the longitude and latitude coordinates of the UAV into coordinates in the Earth-centered Earth-fixed coordinate system, add the coordinate increment to the converted Earth-centered Earth-fixed coordinates, and obtain the coordinates of the target point in the Earth-centered Earth-fixed coordinate system.

[0102] Furthermore, the UAV also includes: a calibration module ( Figure 7 not shown);

[0103] The calibration module is used to calibrate the camera's intrinsic parameters and distortion parameters, calibrate the first extrinsic parameter between the camera and the lidar, and calibrate the second extrinsic parameter between the camera and the integrated navigation module;

[0104] The processing module 302 is further configured to obtain a third external parameter between the laser radar and the integrated navigation module based on the first external parameter and the second external parameter.

[0105] The processing module 302 is further configured to perform dedistortion processing on the first ground image using the camera intrinsic parameters and the distortion parameters, and mark an image area corresponding to the landing area in each frame of the dedistorted first ground image according to a preset landing area parameter to obtain a marked image;

[0106] The labeled image is input into the image segmentation network, and a second ground image including pixel areas corresponding to the landable area is output.

[0107] In this embodiment, a first ground image taken by a camera of a drone is obtained, and the ground image is input into a pre-trained image segmentation network to obtain a second ground image including a pixel area corresponding to a landing area. Point cloud data collected by a lidar is obtained, and each point in the point cloud data is projected into the first ground image. Based on each point in the pixel area in the second ground image after projection, a point cloud image of the landing area is obtained. Based on the point cloud image of the landing area, a target landing area is determined, and the drone is controlled to land in the target landing area. By semantically distinguishing the landing area from the non-landing area through the ground image taken by the camera, the accuracy and controllability of determining the target landing area are increased, and the drone can autonomously select the target landing area.

[0108] See also Figure 7, a schematic diagram of the hardware structure of a drone provided in one embodiment of the present application. Figure 6 As shown, the drone includes: a memory 401 and a processor 402.

[0109] The memory 401 stores an executable computer program 403. The processor 402 coupled to the memory 401 calls the executable computer program 403 stored in the memory to execute the method for autonomously selecting a landing area for a UAV provided in the above embodiment.

[0110] Exemplarily, the computer program 403 may be divided into one or more modules / units, which are stored in the memory 401 and executed by the processor 402 to implement the present invention. The one or more modules / units may include the various modules in the drone described in the above embodiments, such as the acquisition module 301 and the processing module 302.

[0111] Furthermore, the drone also includes:

[0112] At least one input device and at least one output device.

[0113] The processor 402 , memory 401 , input device, and output device may be connected via a bus.

[0114] The input device may specifically be a camera, a touch panel, a physical button, a mouse, etc. The output device may specifically be a display screen.

[0115] Furthermore, the drone may include more components than shown in the figure, or a combination of certain components, or different components, such as network access equipment, sensors, etc.

[0116] The processor 402 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0117] The memory 401 can be, for example, a hard disk drive memory, a non-volatile memory (such as a flash memory or other electronically programmable, erasable memory used to form a solid-state drive), a volatile memory (such as a static or dynamic random access memory), etc., and is not limited in the embodiments of the present application. Specifically, the memory 401 can be an internal storage unit of the drone, such as the hard disk or memory of the drone. The memory 401 can also be an external storage device of the drone, such as a plug-in hard disk equipped on the drone, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Furthermore, the memory 401 can also include both the internal storage unit of the electronic device and an external storage device. The memory 401 is used to store computer programs and other programs and data required by the terminal. The memory 401 can also be used to temporarily store data that has been output or is to be output.

[0118] Furthermore, the embodiment of the present application also provides a computer-readable storage medium, which can be set in the drone in each of the above embodiments. The computer-readable storage medium can be the aforementioned Figure 7 Memory 401 in the illustrated embodiment. This computer-readable storage medium stores a computer program that, when executed by a processor, implements the method for autonomously selecting a landing area for a drone as described in the aforementioned embodiments. Furthermore, the computer-storable medium may be a USB flash drive, a mobile hard drive, a read-only memory (ROM), RAM, a magnetic disk, or an optical disk, among other media capable of storing program code.

[0119] In the several embodiments provided herein, it should be understood that the disclosed drones and methods can be implemented in other ways. For example, the drone embodiments described above are merely illustrative. For example, the module division is merely a logical functional division. In actual implementation, other division methods may be used, such as combining or integrating multiple modules or components into another system, or omitting or not implementing certain features. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device, or module, which may be electrical, mechanical, or other forms.

[0120] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected to achieve the purpose of the present embodiment according to actual needs.

[0121] In addition, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The above-mentioned integrated modules may be implemented in the form of hardware or software functional modules.

[0122] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a readable storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned readable storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0123] It should be noted that for the aforementioned method embodiments, for ease of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0124] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0125] In the present application, unless otherwise clearly specified and limited, a first feature being “on” or “under” a second feature may mean that the first feature and the second feature are in direct contact, or the first feature and the second feature are in indirect contact through an intermediate medium.

[0126] Furthermore, when a first feature is “above,” “above,” or “above” a second feature, it may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. When a first feature is “below,” “below,” or “below” a second feature, it may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0127] In the description of this specification, reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

[0129] The above is a description of the method for autonomously selecting a landing area for a drone, the drone, and the computer-readable storage medium provided in this application. For those skilled in the art, based on the ideas of the embodiments of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for autonomously selecting a landing area for a drone, characterized in that: include: Obtaining a first ground image captured by a camera of a drone, and inputting the first ground image into a pre-trained image segmentation network to obtain a second ground image output by the image segmentation network, with pixel areas corresponding to landing areas marked thereon; Acquire point cloud data collected by a laser radar, and project each point in the point cloud data into the first ground image; Acquire a target point located in a pixel area of ​​the second ground image after projection, and obtain a point cloud image of the landing area based on the target point, wherein the target point is a point located in the second ground image after each point in the point cloud data is projected onto the first ground image; A target landing area is determined according to the point cloud image of the landing area, and the UAV is controlled to land in the target landing area.

2. The method according to claim 1, characterized in that Obtaining the point cloud image of the landing area according to the projected target point located in the pixel area of ​​the second ground image includes: Converting the coordinates of the target point from the laser radar coordinate system to the Earth-centered Earth-fixed coordinate system; A point cloud image of the landing area is obtained according to the coordinates in the Earth-centered Earth-fixed coordinate system.

3. The method according to claim 2, characterized in that The step of converting the coordinates of the target point from the laser radar coordinate system to the Earth-centered Earth-fixed coordinate system comprises: Convert the coordinates of the target point from the laser radar coordinate system to the integrated navigation module coordinate system; Obtaining the latitude and longitude coordinates and northeast celestial coordinates of the integrated navigation module; According to the northeast celestial coordinates of the integrated navigation module, the coordinates of the target point in the integrated navigation module coordinate system are converted into coordinates in the northeast celestial coordinate system; Converting the relative position of the target point converted to the coordinates in the Northeast Celestial Coordinate System and the fuselage of the UAV into coordinates in the Earth-centered Earth-fixed Coordinate System to obtain a coordinate increment; The longitude and latitude coordinates of the UAV are obtained, and the longitude and latitude coordinates of the UAV are converted into coordinates in an earth-centered earth-fixed coordinate system. The coordinates obtained after the conversion in the earth-centered earth-fixed coordinate system are added with the coordinate increment to obtain the coordinates of the target point in the earth-centered earth-fixed coordinate system.

4. The method according to claim 1, wherein The method of acquiring a first ground image captured by a camera of the UAV includes: Calibrate the camera's intrinsic parameters and distortion parameters, calibrate the first extrinsic parameter between the camera and the lidar, and calibrate the second extrinsic parameter between the camera and the integrated navigation module; A third extrinsic parameter between the laser radar and the integrated navigation module is obtained according to the first extrinsic parameter and the second extrinsic parameter.

5. The method according to claim 4, characterized in that Inputting the first ground image into a pre-trained image segmentation network to obtain a second ground image including a pixel area corresponding to the landing area includes: Dedistorting the first ground image using the camera intrinsic parameters and the distortion parameters, and marking an image area corresponding to a landing area in each frame of the dedistorted first ground image according to a preset landing area parameter to obtain a marked image; The labeled image is input into the image segmentation network, and a second ground image including pixel areas corresponding to the landable area is output.

6. A drone, characterized in that: include: An acquisition module, configured to acquire a first ground image captured by a camera of the UAV; a processing module, configured to input the ground image into a pre-trained image segmentation network to obtain a second ground image output by the image segmentation network, the second ground image being marked with pixel areas corresponding to landing areas; The acquisition module is further used to acquire point cloud data collected by the laser radar; The processing module is further configured to project each point in the point cloud data onto the first ground image, obtain a target point located in a pixel area in the second ground image after the projection, and obtain a point cloud image of the landing area based on the target point, wherein the target point is a point located in the second ground image after each point in the point cloud data is projected onto the first ground image; The processing module is further configured to determine a target landing area based on the point cloud image of the landing area, and control the UAV to land in the target landing area.

7. The drone according to claim 6, characterized in that: The processing module is further configured to convert the coordinates of the target point from the coordinates in the laser radar coordinate system to the coordinates in the Earth-centered Earth-fixed coordinate system; A point cloud image of the landing area is obtained according to the coordinates in the Earth-centered Earth-fixed coordinate system.

8. The drone according to claim 7, characterized in that: The processing module is further configured to convert the coordinates of the target point from the coordinates in the laser radar coordinate system to the coordinates in the integrated navigation module coordinate system; Obtaining the latitude and longitude coordinates and northeast celestial coordinates of the integrated navigation module; According to the northeast celestial coordinates of the integrated navigation module, the coordinates of the point in the integrated navigation module coordinate system are converted into the coordinates in the northeast celestial coordinate system; Converting the relative position of the target point converted to the coordinates in the Northeast Celestial Coordinate System and the fuselage of the UAV into coordinates in the Earth-centered Earth-fixed Coordinate System to obtain a coordinate increment; The longitude and latitude coordinates of the UAV are obtained, and the longitude and latitude coordinates of the UAV are converted into coordinates in the Earth-centered Earth-fixed coordinate system. The coordinate increment is added to the converted Earth-centered Earth-fixed coordinates to obtain the coordinates of the target point in the Earth-centered Earth-fixed coordinate system.

9. A drone, characterized in that: include: memory and processor; The memory stores an executable computer program; The processor coupled to the memory calls the executable computer program stored in the memory to execute the method for autonomously selecting a landing area for a drone as described in any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for autonomously selecting a landing area for a drone as described in any one of claims 1 to 5 is implemented.

Images