Unmanned aerial vehicle landing method, device and equipment and storage medium

Abstract

The invention provides an unmanned aerial vehicle landing method and device, equipment and a storage medium. The method comprises the following steps: acquiring a ground image in a visual range of the unmanned aerial vehicle, performing scene segmentation on the ground image, and acquiring a plurality of candidate landing preparation areas from the ground range corresponding to the ground image based on a scene segmentation result; and according to the current position of the unmanned aerial vehicle and the position of the candidate landing preparation area, planning a candidate flight path of the unmanned aerial vehicle landing to the candidate landing preparation area. And according to the candidate flight path, determining a target flight path and a target landing preparation area corresponding to the target flight path, and landing to the target landing preparation area according to the target flight path. According to the method, the intelligence, stability and safety of landing of the unmanned aerial vehicle are improved.

Description

Technical Field

[0001] The present application relates to the field of drone technology, and in particular to a drone landing method, device, equipment and storage medium. Background Art

[0002] Unmanned aircraft, also known as "drones", are unmanned aircraft that are controlled by radio remote control equipment and self-contained program control devices, or are operated completely or intermittently autonomously by onboard computers. With the development of drone technology, drones have been widely used in many fields such as aerial photography, agriculture, rescue, and plant protection. Among them, the landing of drones is an important part of the use of drones. The current drone landing methods have the problem of low intelligence.

[0003] Therefore, how to improve the intelligence of drone landing is an urgent problem to be solved. Summary of the invention

[0004] The present application provides a drone landing method, device, equipment and storage medium, which are used to solve the problem of low intelligence of drone landing in the prior art.

[0005] In a first aspect, the present application provides a method for landing a drone, comprising:

[0006] Acquire ground images within the visual range of the drone;

[0007] Performing scene segmentation on the ground image, and acquiring a plurality of candidate alternate landing areas from a ground range corresponding to the ground image based on the scene segmentation result;

[0008] Planning a candidate flight path for the drone to land at the candidate alternate landing area according to the current position of the drone and the position of the candidate alternate landing area;

[0009] Determining a target flight path and a target alternate landing area corresponding to the target flight path according to the candidate flight path;

[0010] Land at the target alternate landing area according to the target flight path.

[0011] Optionally, performing scene segmentation on the ground image, and acquiring a plurality of candidate alternate areas from a ground range corresponding to the ground image based on the scene segmentation result, includes:

[0012] Inputting the ground image into a trained semantic segmentation model based on the Farseg algorithm to obtain a segmentation result of the ground image; the semantic segmentation model is used to segment the ground image; the segmentation result is used to characterize the category of the area included in the ground range;

[0013] According to the category of the pre-equipped alternate landing area and the segmentation result, a candidate alternate landing area is obtained from a ground range corresponding to the ground image.

[0014] Optionally, determining a target flight path and a target alternate landing area corresponding to the target flight path according to the candidate flight path includes:

[0015] Obtaining the flight distance and / or estimated flight duration corresponding to the candidate flight path;

[0016] Selecting the target flight path from the candidate flight paths according to the flight distance and / or the estimated flight duration;

[0017] Determine a target alternate landing area corresponding to the target flight path.

[0018] Optionally, obtaining the estimated flight duration corresponding to the candidate flight path includes:

[0019] Acquire the flight parameters of the UAV and the environmental parameters of the area where the candidate flight path is located;

[0020] An estimated flight duration corresponding to the candidate flight path is determined according to the flight parameters, the environmental parameters, and the flight distance of the candidate flight path.

[0021] Optionally, selecting the target flight path from the candidate flight paths according to the flight distance and / or the estimated flight duration includes:

[0022] Obtaining the remaining flight endurance of the drone;

[0023] Selecting a target flight path to be determined whose flight distance is less than or equal to the remaining flight endurance from the candidate flight paths;

[0024] The target flight path is selected from the target flight paths to be determined according to the flight distance of the target flight paths to be determined and / or the estimated flight duration.

[0025] Optionally, planning a candidate flight path for the drone to land at the candidate alternate landing area according to the current position of the drone and the position of the candidate alternate landing area includes:

[0026] Detecting whether there are obstacles on the flight path of the UAV landing to the candidate alternate landing area;

[0027] If the obstacle exists, obtaining the obstacle avoidance strategy of the drone;

[0028] According to the obstacle avoidance strategy of the UAV, the position of the UAV, and the position of the candidate alternate landing area, a candidate flight path for the UAV to land in the candidate alternate landing area is planned.

[0029] Optionally, if the obstacle exists, the method further includes:

[0030] Acquire environmental parameters of an area where the flight path is located, wherein the environmental parameters include wind direction;

[0031] According to the wind direction, determining a direction around the obstacle where the wind direction is smaller;

[0032] Obstacle avoidance planning is performed in a direction where the wind direction is smaller around the obstacle.

[0033] In a second aspect, the present application provides a drone landing device, comprising:

[0034] An acquisition module, used to acquire ground images within the visual range of the drone;

[0035] a processing module, configured to perform scene segmentation on the ground image, and obtain a plurality of candidate alternate landing areas from a ground range corresponding to the ground image based on the scene segmentation result; plan a candidate flight path for the drone to land at the candidate alternate landing area according to the current position of the drone and the position of the candidate alternate landing area; and determine a target flight path and a target alternate landing area corresponding to the target flight path according to the candidate flight path;

[0036] An execution module is used to land at the target alternate landing area according to the target flight path.

[0037] In a third aspect, the present application provides an electronic device, comprising: a processor, a communication interface, and a memory; the processor is communicatively connected to the communication interface and the memory respectively;

[0038] The memory stores computer-executable instructions;

[0039] The communication interface communicates and interacts with external devices;

[0040] The processor executes the computer-executable instructions stored in the memory to implement the method as described in any one of the first aspects.

[0041] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the drone landing method as described in any one of the first aspects.

[0042] In a fifth aspect, the present application provides a computer program product, which, when executed by a processor, is used to implement the drone landing method as described in any one of the first aspects.

[0043] The drone landing method, device, equipment and storage medium provided by the present application obtain the ground image within the visual range of the drone, perform scene segmentation on the ground image, and obtain multiple candidate alternate landing areas from the ground range corresponding to the ground image based on the scene segmentation result. According to the current position of the drone and the position of the candidate alternate landing area, the candidate flight path for the drone to land to the candidate alternate landing area is planned, and the target flight path and the target alternate landing area corresponding to the target flight path are determined according to the candidate flight path, and the drone lands to the target alternate landing area according to the target flight path, so that the drone can automatically determine the alternate landing area that is convenient for landing, and automatically land in the alternate landing area, thereby improving the intelligence, stability, and safety of the drone landing. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0045] Figure 1 A schematic diagram of a process of landing a drone provided in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of another method for landing a drone provided in an embodiment of the present application;

[0047] Figure 3 A schematic diagram of a flow chart of another method for landing a drone provided in an embodiment of the present application;

[0048] Figure 4 A schematic diagram of a flow chart of another method for landing a drone provided in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of a scenario of obstacle avoidance planning provided in an embodiment of the present application;

[0050] Figure 6 A schematic diagram of the structure of a UAV landing device provided in an embodiment of the present application;

[0051] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0052] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0053] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0054] At present, the landing of drones is mainly achieved through two methods: manual control and automatic landing of drones. When it is achieved through manual control, the drone usually receives control instructions from the user's handheld remote control terminal, lowers the altitude of the drone according to the control instructions, and controls the flight direction of the drone, thereby achieving the function of landing the drone. This landing method has high requirements for user operation. If there is an operational error, the flight stability of the drone may be poor during the landing process, and the drone may bump into something, causing damage to the drone.

[0055] In order to provide a more stable and safe landing function, it is currently possible to control the drone to achieve the landing function by automatically landing the drone. In this implementation mode, the user can send an automatic landing command to the drone through a handheld remote control terminal, and the drone will automatically land with one click according to the automatic landing command. However, after the existing drone triggers the one-key landing, the drone will enter the automatic landing mode. At this time, the drone will no longer receive the flight operation command from the handheld remote control terminal, so the user cannot set the parameters for the horizontal dimension of the drone. When the user wants to perform a one-key landing operation on the drone at a certain moment, it is found that the current position of the drone is not suitable for landing. The user can only perform flight operations on the drone first, and then land the drone with one click when the drone flies to a relatively ideal position. Therefore, the current drone landing method has the problem of low intelligence.

[0056] In view of this, the present application provides a drone landing method, which automatically obtains candidate alternate landing areas where the drone can land by performing scene segmentation on ground images acquired by the drone, and selects a target alternate landing area that meets the landing strategy from the candidate alternate landing areas, and controls the drone to automatically land in the target alternate landing area according to the flight path to the target alternate landing area, thereby improving the intelligence of the drone landing method.

[0057] The execution subject of the data processing method provided in the present application may be a drone, or a processing chip of the drone, or software or program code for implementing the drone landing method. When the execution subject is a drone, the drone may be deployed with software or program code for running the drone landing method, and the landing of the drone may be controlled by the software or program code. The execution subject of the method may also be a remote control terminal of the drone, or may also be a cloud platform with drone control function. When the execution subject is a cloud platform, the control of the landing of the drone in the method may be performed in the cloud, and the cloud platform may be logically divided into multiple parts according to actual needs, each part having different functions. The various parts of the drone control platform may be deployed in any two or three of the electronic device (located on the user side), the edge environment, and the cloud environment. The edge environment is an environment including a collection of edge electronic devices that are close to the electronic device, and the edge electronic device includes: an edge server, an edge station with computing power, etc. The various parts of the drone control platform deployed in different environments or devices collaborate to realize the functions of the drone control platform. It should be understood that this application does not restrict the deployment of which parts of the drone control platform are deployed in what environment. In actual application, it can be adaptively deployed according to the computing power of the electronic equipment, the resource usage of the edge environment and the cloud environment, or the specific application requirements.

[0058] The following takes the drone that implements the method as an example, and describes in detail the technical solution of the present application and how the technical solution of the present application solves the above technical problems through specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0059] Figure 1 The following is a flow chart of a method for landing a drone provided in an embodiment of the present application. Figure 1 As shown, the method may include:

[0060] S101. Acquire ground images within the visual range of the UAV.

[0061] Among them, the drone is currently in a flying state, and the visual range of the drone may include the ground area around the drone, for example, the ground area below the drone, the ground area in front of the drone, the ground area behind the drone, etc., and this application does not impose any restrictions on this.

[0062] The ground image may be collected by the drone itself, for example, it may be obtained by an image acquisition device such as a camera or a still camera on the drone. The ground image may also be obtained by the drone from a device or network that is in communication connection with the drone, for example, it may be obtained from a cloud platform connected to the drone based on the location of the drone, or it may be a ground image within the visual range of the drone corresponding to the location obtained from the network.

[0063] The ground image may be a remote sensing image, for example, a high spatial resolution remote sensing image (referred to as a high-resolution image) to obtain richer ground information in the ground image and improve the accuracy of the ground image.

[0064] Optionally, the drone may start collecting ground images within the visual range of the drone after receiving the automatic landing instruction, or may periodically collect ground images within the visual range of the drone. Alternatively, the drone may automatically execute the automatic landing procedure and start acquiring ground images within the visual range of the drone when the remaining flight endurance is lower than a preset endurance threshold.

[0065] S102: Perform scene segmentation on the ground image, and obtain multiple candidate alternate landing areas from a ground range corresponding to the ground image based on the scene segmentation result.

[0066] The candidate alternate landing area may be, for example, a baseball field, a tennis court, a basketball court, a ground runway, a football field, or other areas suitable for drone landing. The method for scene segmentation of ground images may be any image segmentation method, such as a threshold-based segmentation method, a region-based segmentation method, an edge-based segmentation method, etc. Exemplarily, for example, Deeplab, Deeplabv2, Deeplabv3, DenseASPP, or other networks may be used to segment specific scenes.

[0067] Through the image segmentation method, the ground image is divided into non-intersecting areas, and each segmented area is marked to obtain a scene segmentation result. The scene segmentation result includes the type of each segmented area, for example, it can be the types of baseball field, tennis court, basketball court, etc. mentioned above. The present application does not limit the image segmentation method. The specific method of scene segmentation of the ground image through the image segmentation method can refer to the prior art and will not be repeated here. After obtaining the scene segmentation result, a ground area of ​​the same type as the candidate alternate landing area is determined from the scene segmentation result as the candidate alternate landing area.

[0068] S103: planning a candidate flight path for the drone to land at the candidate alternate landing area according to the current position of the drone and the position of the candidate alternate landing area.

[0069] The current position of the drone is obtained, for example, by a positioning device on the drone. The current position may include the position of the drone in the horizontal dimension, and the flight altitude of the drone, which may be represented by an altitude. The position of the candidate alternate area may be the center position of the candidate alternate area, or any position within the candidate alternate area. For example, it may be any position within the candidate alternate area within a preset distance from the edge of the candidate alternate area, and the present application does not limit this. The position may include the position in the horizontal dimension, and the altitude of the position.

[0070] A target location is selected from the candidate alternate landing area, and the location is used as the landing location. According to the current location of the UAV and the location of the candidate alternate landing area, a candidate flight path for the UAV to land in the candidate alternate landing area is planned. For example, the flight path with the shortest distance can be used as the candidate flight path, or the flight path with the least turns can be used as the candidate flight path, or the flight path with the least obstacles can be used as the candidate flight path, etc. The planning of the flight path can refer to any existing UAV flight path planning method, which will not be described in detail here.

[0071] S104. Determine a target flight path and a target alternate landing area corresponding to the target flight path according to the candidate flight paths.

[0072] In one possible implementation, the target flight path is determined according to the flight distances corresponding to the candidate flight paths. For example, the candidate flight path with the shortest flight distance may be used as the target flight path.

[0073] Another possible implementation manner is to determine the target flight path according to the estimated flight durations corresponding to the candidate flight paths. For example, the candidate flight path with the shortest estimated flight duration may be used as the target flight path.

[0074] Another possible implementation method is to determine the target flight path based on the flight distance and expected flight time corresponding to the candidate flight paths. For example, the score of each candidate flight path can be determined based on the flight distance, flight distance weight, expected flight time, and expected flight time weight, and the target flight path is determined based on the scoring results.

[0075] S105. Land at the target alternate landing area according to the target flight path.

[0076] In one possible implementation, the end point of the target flight path is above the landing position of the target alternate landing area. After the UAV flies along the target flight path to the landing position, it lowers its flight altitude until it lands in the target alternate landing area.

[0077] Another possible implementation is that the end point of the target flight path is the ground where the landing position of the target alternate landing area is located, and the UAV flies along the target flight path and lands in the target alternate landing area.

[0078] The drone landing method provided by the embodiment of the present application obtains a ground image within the visual range of the drone, performs scene segmentation on the ground image, and obtains multiple candidate alternate landing areas from the ground range corresponding to the ground image based on the scene segmentation result. According to the current position of the drone and the position of the candidate alternate landing area, a candidate flight path for the drone to land to the candidate alternate landing area is planned, and according to the candidate flight path, a target flight path and a target alternate landing area corresponding to the target flight path are determined, and the drone lands to the target alternate landing area according to the target flight path, so that the drone can automatically determine an alternate landing area that is convenient for landing, and automatically land in the alternate landing area, thereby improving the intelligence, stability, and safety of the drone landing.

[0079] Next, taking the ground image as a high-resolution image and improving the accuracy of the ground image as an example, a detailed description is given of how to perform scene segmentation on the ground image in step S102, and obtain multiple candidate alternate landing areas from the ground range corresponding to the ground image based on the scene segmentation result.

[0080] At present, high-resolution images always face problems such as large scale changes, large background intra-class variance, and unbalanced distribution between foreground and background, resulting in a small proportion of pixels in the UAV alternate landing area, inconsistency in edge information, and irregular distribution of feature deformation. The existing image segmentation algorithms are not sufficient to effectively deal with the above problems caused by the large background intra-class variance and unbalanced distribution between foreground and background, resulting in low accuracy and efficiency in scene segmentation of the above-mentioned ground images.

[0081] Figure 2 A flowchart of another method for landing a drone provided in an embodiment of the present application. Figure 2 As shown, the aforementioned step S102 may include:

[0082] S201, inputting a ground image into a trained semantic segmentation model based on the Farseg algorithm to obtain a segmentation result of the ground image.

[0083] Among them, the semantic segmentation model is used to segment the ground image, and the segmentation result is used to characterize the category of the area included in the ground range. The trained semantic segmentation model based on the Farseg algorithm can utilize the symbiotic relationship between geospatial scenes and geospatial objects, associate the context related to the foreground, and improve the model's recognition ability for foreground features. In addition, the semantic segmentation model includes a foreground-aware optimization function, which includes three steps: difficult sample estimation, dynamic weighting, and back propagation, thereby suppressing a large number of simple background samples, so that the entire network focuses on the foreground and difficult samples for balanced optimization, thereby reducing the contribution of a large number of simple samples to the gradient, so as to alleviate the problem of imbalance between foreground and background. The specific architecture and training method of the semantic segmentation model based on the Farseg algorithm can refer to the prior art and will not be repeated here.

[0084] The trained semantic segmentation model based on the Farseg algorithm can accurately segment the high-resolution image of the ground range corresponding to the ground image into different areas, and generate a segmentation result of the ground image.

[0085] S202: Acquire a candidate alternate landing area from a ground range corresponding to the ground image according to the category of the alternate landing area and the segmentation result.

[0086] According to the segmentation result, each area in the ground image data and the type of each area are obtained. According to the type of each area and the category of the pre-equipment landing area, the areas belonging to the category of the pre-equipment landing area in the ground range corresponding to the ground image are determined, and these areas are used as candidate alternate landing areas.

[0087] The method provided in the embodiment of the present application obtains the segmentation result of the ground image by inputting the ground image into a trained semantic segmentation model based on the Farseg algorithm, and obtains the candidate alternate landing area from the ground range corresponding to the ground image according to the pre-equipment landing area category and the segmentation result, thereby reducing the problems caused by the large background intra-class variance and unbalanced distribution of foreground and background in the high-resolution image, and improving the accuracy and efficiency of scene segmentation of the above-mentioned ground image.

[0088] The following is a detailed description of how to determine the target flight path and the target alternate landing area corresponding to the target flight path based on the candidate flight paths in the aforementioned step S104.

[0089] Figure 3 A flowchart of another method for landing a drone provided in an embodiment of the present application is shown below. Figure 3 As shown, the aforementioned step S104 may include:

[0090] S301. Obtain the flight distance and / or estimated flight duration corresponding to the candidate flight path.

[0091] One possible implementation method is to obtain the flight distance based on the path length of the flight path, and to obtain the estimated flight time of the candidate flight path based on the flight speed of the UAV. For example, the estimated flight time of the flight path can be obtained by dividing the flight distance by the flight speed.

[0092] Another possible implementation method is to determine the estimated flight time by considering the environmental parameters of the area where the candidate flight path is located. The environmental parameters may include the wind speed and wind direction in the area, and may also include the parameters of obstacles in the area. This implementation method may be implemented by the following steps:

[0093] S3011. Obtain the flight parameters of the UAV and the environmental parameters of the area where the candidate flight path is located.

[0094] The flight parameters of the UAV may include the flight speed, turning speed, etc. of the UAV, or other flight parameters that determine the flight speed and turning speed, such as the rotation speed and output power of the driving motor of the UAV.

[0095] When the environmental parameters include the wind speed and wind direction in the area, the current wind speed and wind direction in the area can be obtained from the network, or the wind speed and wind direction at the location of the drone can be used as the wind speed and wind direction in the area. When the environmental parameters include the parameters of obstacles in the area, the map data of the area can be obtained according to the location of the area, or the existence of obstacles on the candidate flight path and the size of the obstacles can be obtained from the map data. For example, if the obstacle is a high-rise building, the width, length, and height of the building can be obtained to characterize the size of the obstacle.

[0096] S3012. Determine the estimated flight duration corresponding to the candidate flight path based on the flight parameters, environmental parameters, and the flight distance of the candidate flight path.

[0097] When the flight parameter includes the flight speed and the environmental parameter includes the wind speed and the wind direction, the actual flight speed of the UAV is calculated according to the flight speed, the wind speed and the wind direction. The estimated flight duration is calculated according to the actual flight speed and the flight distance of the candidate flight path.

[0098] Furthermore, when the flight parameter also includes turning speed, the estimated flight time can be corrected according to the number of turns in the candidate flight path. In this implementation, the time calculated according to the turning speed when the drone turns is replaced by the time when the flight speed passes through these turns, so as to obtain a more accurate estimated flight time.

[0099] Furthermore, when the environmental parameters also include parameters of obstacles in the area, the actual time taken by the drone to execute the obstacle avoidance strategy to avoid the obstacle can be determined based on the drone's obstacle avoidance strategy, and the above-mentioned estimated flight time can be corrected according to the actual time to obtain a more accurate estimated flight time.

[0100] S302: Select a target flight path from candidate flight paths according to the flight distance and / or the estimated flight duration.

[0101] Implementation A: Determine the target flight path according to the flight distances corresponding to the candidate flight paths. For example, the candidate flight path with the shortest flight distance may be used as the target flight path.

[0102] Implementation method B: Determine the target flight path according to the estimated flight durations corresponding to the candidate flight paths. For example, the candidate flight path with the shortest estimated flight duration may be used as the target flight path.

[0103] Implementation method C: Determine the target flight path based on the flight distance and estimated flight time corresponding to the candidate flight path. For example, the score of each candidate flight path can be determined based on the flight distance, flight distance weight, estimated flight time, and estimated flight time weight, and the target flight path is determined based on the scoring results.

[0104] Optionally, in the above implementations AC, the remaining flight endurance of the UAV may be further considered to select a target flight path from the candidate flight paths. This implementation may be achieved by the following steps:

[0105] S3021. Obtain the remaining flight endurance of the drone.

[0106] The remaining flight endurance of the drone can be obtained by calculating the remaining energy of the drone, and the remaining flight endurance of the drone is determined by the remaining energy and the operating status of the drone. For example, for an electric drone, the remaining energy is the remaining power of the drone. Alternatively, the remaining flight endurance can also be obtained by calculating the remaining flight time of the drone. According to the remaining flight time of the drone and the flight speed of the drone, the distance that the drone can fly at the remaining flight time at the flight speed is calculated, and the distance is used as the remaining flight endurance of the drone.

[0107] S3022. Select a target flight path to be determined whose flight distance is less than or equal to the remaining flight endurance from the candidate flight paths.

[0108] One possible implementation is to use a candidate flight path whose flight distance is less than or equal to the remaining flight endurance as the target flight path to be determined.

[0109] Another possible implementation is to set a range protection margin, which is used to prevent the drone from running out of energy, that is, after landing, the remaining flight range of the drone is greater than or equal to the range protection margin, thereby improving the safety of the drone during landing. The candidate flight path whose flight distance is less than or equal to the remaining flight range minus the range protection margin is used as the target flight path to be determined.

[0110] S3023. Select a target flight path from the target flight paths to be determined according to the flight distance of the target flight paths to be determined and / or the estimated flight duration.

[0111] The implementation method of selecting the target flight path from the target flight path to be determined based on the flight distance and / or the expected flight duration of the target flight path to be determined is similar to the implementation method of selecting the target flight path from the candidate flight paths based on the flight distance and / or the expected flight duration in the aforementioned step S302, and will not be repeated here.

[0112] S303: Determine the target alternate landing area corresponding to the target flight path.

[0113] The target alternate landing area is determined according to the mapping relationship between the target flight path and the target alternate landing area, and the target flight path. Alternatively, the location of the endpoint of the target flight path can be determined, and according to the coordinates of the location of the endpoint, the candidate alternate landing area including the coordinates is determined as the target alternate landing area.

[0114] The method provided in the embodiment of the present application obtains the flight distance and / or the expected flight duration corresponding to the candidate flight path, and selects the target flight path and the target alternate landing area corresponding to the target flight path from the candidate flight paths according to the flight distance and / or the expected flight duration and the flight path selection strategy. In addition, the target alternate landing area is comprehensively determined by considering factors such as environmental parameters and remaining flight endurance, thereby improving the intelligence and efficiency of the landing of the drone.

[0115] Next, taking the consideration of whether there are obstacles on the flight path of the candidate alternate landing area as an example, a detailed introduction is given to how to plan the candidate flight path for the drone to land in the candidate alternate landing area based on the current position of the drone and the position of the candidate alternate landing area in the aforementioned step S103.

[0116] Figure 4 A flowchart of another method for landing a drone provided in an embodiment of the present application is shown below. Figure 4 As shown, the aforementioned step S103 may include:

[0117] S401, detecting whether there are obstacles on the flight path of the UAV landing to the candidate alternate landing area.

[0118] The obstacle may be, for example, a building, a tree, a rock, a mountain, or other obstacle that may hinder the flight of the drone. The obstacle may be monitored by an image acquisition device deployed on the drone, or may be acquired through map data.

[0119] For example, the map data includes volume information, size information, etc. of obstacles such as buildings or mountains that can be displayed in the map data. The drone obtains the map data through a network or communication connection, and obtains the volume information, size information, etc. of the obstacle from the map data.

[0120] S402: If there is an obstacle, obtain the obstacle avoidance strategy of the drone.

[0121] One possible implementation method is to bypass the obstacle through an existing obstacle avoidance strategy. For example, if the obstacle is on the line between the current position of the UAV and the position of the candidate alternate landing area, the obstacle avoidance strategy can be that when the UAV flies near the obstacle, it can bypass the obstacle to a position where there is no obstacle blocking it based on the volume information and size information of the obstacle, pass the obstacle, and return to the line after passing the obstacle.

[0122] Another possible implementation method is to perform obstacle avoidance planning by reducing the impact of wind direction on the flight of the UAV through the obstacle. This implementation method can be achieved by the following steps:

[0123] S4021. Obtain environmental parameters of the area where the flight path is located.

[0124] The environmental parameter includes wind direction. The method of obtaining the environmental parameter is the same as that in the aforementioned step S3011, and will not be repeated here.

[0125] S4022. Determine the direction of the obstacle with the smallest wind direction based on the wind direction.

[0126] The direction opposite to the wind direction is determined as the direction with the smaller wind direction around the obstacle. For example, if the wind direction is easterly, the west side of the obstacle is determined as the direction with the smaller wind direction around the obstacle.

[0127] S4023. Perform obstacle avoidance planning in the direction with smaller wind direction around the obstacle.

[0128] In a possible implementation, the obstacle avoidance plan is that when the UAV flies around the obstacle, it first flies to a direction where the wind direction around the obstacle is smaller, and then flies in this direction according to a flight trajectory around the obstacle.

[0129] Another possible implementation method is that when the UAV flies around the obstacle, it first flies to a position with a smaller wind direction around the obstacle, and determines the height difference between the current position of the UAV and the position of the candidate alternate landing area based on the current position of the UAV and the position of the candidate alternate landing area, and determines the height to be lowered based on the height difference. The height to be lowered is less than the height difference, and the height to be lowered can be preset or determined based on a preset target height. For example, if the height difference is 20 meters and the preset target height is 5 meters, the height to be lowered is 15 meters. At the position of the obstacle, after lowering the height to be lowered, the flight is then carried out according to the flight trajectory that bypasses the obstacle. Figure 5 A schematic diagram of a scenario of obstacle avoidance planning provided in an embodiment of the present application. Figure 5 As shown, the obstacle avoidance plan is the same as described in the implementation method.

[0130] S403: planning a candidate flight path for the drone to land at the candidate alternate landing area according to the obstacle avoidance strategy of the drone, the location of the drone, and the location of the candidate alternate landing area.

[0131] According to the obstacle avoidance strategy of the UAV, an obstacle avoidance flight path of the UAV is determined to bypass the obstacle. According to the position of the UAV and the position of the candidate alternate landing area, a connection line between the two is determined, and the connection line is divided into a first flight path before the starting point of the aforementioned obstacle avoidance flight path, and a second flight path after the end point of the aforementioned obstacle avoidance flight path. The first flight path, the obstacle avoidance flight path, and the second flight path are spliced ​​to obtain a candidate flight path for the UAV to land in the candidate alternate landing area.

[0132] The method provided in the embodiment of the present application detects whether there is an obstacle on the flight path of the drone landing to the candidate alternate landing area. If there is an obstacle, the obstacle avoidance strategy of the drone is obtained, and the candidate flight path of the drone landing to the candidate alternate landing area is planned according to the obstacle avoidance strategy of the drone, the position of the drone, and the position of the candidate alternate landing area, so that the drone can obtain a candidate flight path that bypasses the obstacle. In addition, the obstacle is used to resist the wind that may affect the stability of the drone, and obstacle avoidance planning is performed in the direction with smaller wind direction around the obstacle, thereby further improving the stability and safety of the landing of the drone.

[0133] Figure 6 This is a schematic diagram of the structure of a drone landing device provided in an embodiment of the present application. Figure 6 As shown, the UAV landing device may include: an acquisition module 11, a processing module 12, and an execution module 13.

[0134] The acquisition module 11 is used to acquire the ground image within the visual range of the UAV.

[0135] The processing module 12 is used to perform scene segmentation on the ground image, and obtain multiple candidate alternate landing areas from the ground range corresponding to the ground image based on the scene segmentation result. According to the current position of the UAV and the position of the candidate alternate landing area, a candidate flight path for the UAV to land in the candidate alternate landing area is planned. According to the candidate flight path, a target flight path and a target alternate landing area corresponding to the target flight path are determined.

[0136] The execution module 13 is used to land at the target alternate landing area according to the target flight path.

[0137] In a possible implementation, the processing module 12 is specifically used to input the ground image into a trained semantic segmentation model based on the Farseg algorithm to obtain a segmentation result of the ground image. According to the category of the pre-equipment landing area and the segmentation result, a candidate alternate landing area is obtained from the ground range corresponding to the ground image. The semantic segmentation model is used to segment the ground image. The segmentation result is used to characterize the category of the area included in the ground range.

[0138] In another possible implementation, the acquisition module 11 is specifically configured to acquire the flight distance and / or the estimated flight duration corresponding to the candidate flight path. The processing module 12 is specifically configured to select the target flight path from the candidate flight paths according to the flight distance and / or the estimated flight duration, and determine the target alternate landing area corresponding to the target flight path.

[0139] In this implementation, in a possible implementation, the acquisition module 11 is specifically used to acquire the flight parameters of the UAV and the environmental parameters of the area where the candidate flight path is located. The processing module 12 is specifically used to determine the estimated flight time corresponding to the candidate flight path according to the flight parameters, the environmental parameters, and the flight distance of the candidate flight path.

[0140] In another possible implementation, the acquisition module 11 is specifically configured to acquire the remaining flight endurance of the UAV. The processing module 12 is specifically configured to select a target flight path to be determined from the candidate flight paths, the flight distance of which is less than or equal to the remaining flight endurance. The target flight path is selected from the target flight paths to be determined based on the flight distance of the target flight path to be determined and / or the estimated flight duration.

[0141] In any of the above implementations, the processing module 12 is specifically used to detect whether there is an obstacle on the flight path of the drone landing to the candidate alternate landing area. If the obstacle exists, the obstacle avoidance strategy of the drone is obtained. According to the obstacle avoidance strategy of the drone, the position of the drone, and the position of the candidate alternate landing area, a candidate flight path for the drone to land to the candidate alternate landing area is planned.

[0142] Wherein, when there is an obstacle, optionally, the acquisition module 11 is further used to acquire environmental parameters of the area where the flight path is located, and the environmental parameters include wind direction. The processing module 12 is further used to determine the direction with smaller wind direction around the obstacle according to the wind direction. Obstacle avoidance planning is performed in the direction with smaller wind direction around the obstacle.

[0143] The drone landing device provided in the embodiment of the present application can execute the drone landing method in the above method embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0144] Figure 7 The structure diagram of an electronic device provided in an embodiment of the present application. The electronic device is used to execute the aforementioned drone landing method, for example, the aforementioned drone. Figure 7 As shown, the electronic device 700 may include: at least one processor 701 , a memory 702 , and a communication interface 703 .

[0145] The memory 702 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer operation instructions.

[0146] The memory 702 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0147] The processor 701 is used to execute the computer-executable instructions stored in the memory 702 to implement the method described in the above method embodiment. The processor 701 may be a CPU, or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0148] The processor 701 can communicate and interact with external devices through the communication interface 703. The external device can be, for example, the remote control terminal of the drone mentioned above. In a specific implementation, if the communication interface 703, the memory 702 and the processor 701 are implemented independently, the communication interface 703, the memory 702 and the processor 701 can be interconnected through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0149] Optionally, in a specific implementation, if the communication interface 703, the memory 702 and the processor 701 are integrated on a chip, the communication interface 703, the memory 702 and the processor 701 can communicate through an internal interface.

[0150] The present application also provides a computer-readable storage medium, which may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes. Specifically, the computer-readable storage medium stores program instructions, and the program instructions are used for the methods in the above embodiments.

[0151] The present application also provides a program product, which includes an execution instruction, which is stored in a readable storage medium. At least one processor of a computing device can read the execution instruction from the readable storage medium, and at least one processor executes the execution instruction so that the computing device implements the above-mentioned drone landing method.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for landing a drone, characterized in that: include: Acquire ground images within the visual range of the UAV; Performing scene segmentation on the ground image, and acquiring a plurality of candidate alternate landing areas from a ground range corresponding to the ground image based on the scene segmentation result; Planning a candidate flight path for the drone to land at the candidate alternate landing area according to the current position of the drone and the position of the candidate alternate landing area; Determining a target flight path and a target alternate landing area corresponding to the target flight path according to the candidate flight path; Land at the target alternate landing area according to the target flight path.

2. The method according to claim 1, characterized in that The performing scene segmentation on the ground image, and acquiring a plurality of candidate alternate areas from a ground range corresponding to the ground image based on the scene segmentation result, includes: Inputting the ground image into a trained semantic segmentation model based on the Farseg algorithm to obtain a segmentation result of the ground image; the semantic segmentation model is used to segment the ground image; the segmentation result is used to characterize the category of the area included in the ground range; According to the category of the pre-equipped alternate landing area and the segmentation result, a candidate alternate landing area is obtained from a ground range corresponding to the ground image.

3. The method according to claim 1, characterized in that The determining, according to the candidate flight path, a target flight path and a target alternate landing area corresponding to the target flight path includes: Obtaining the flight distance and / or estimated flight duration corresponding to the candidate flight path; Selecting the target flight path from the candidate flight paths according to the flight distance and / or the estimated flight duration; Determine a target alternate landing area corresponding to the target flight path.

4. The method according to claim 3, characterized in that The obtaining of the estimated flight duration corresponding to the candidate flight path includes: Acquire the flight parameters of the UAV and the environmental parameters of the area where the candidate flight path is located; An estimated flight duration corresponding to the candidate flight path is determined according to the flight parameters, the environmental parameters, and the flight distance of the candidate flight path.

5. The method according to claim 3, characterized in that: The step of selecting the target flight path from the candidate flight paths according to the flight distance and / or the estimated flight duration includes: Obtaining the remaining flight endurance of the drone; Selecting a target flight path to be determined whose flight distance is less than or equal to the remaining flight endurance from the candidate flight paths; The target flight path is selected from the target flight paths to be determined according to the flight distance of the target flight paths to be determined and / or the estimated flight duration.

6. The method according to any one of claims 1 to 5, characterized in that: The step of planning a candidate flight path for the drone to land at the candidate alternate landing area according to the current position of the drone and the position of the candidate alternate landing area includes: Detecting whether there are obstacles on the flight path of the UAV landing to the candidate alternate landing area; If the obstacle exists, obtaining the obstacle avoidance strategy of the drone; According to the obstacle avoidance strategy of the UAV, the position of the UAV, and the position of the candidate alternate landing area, a candidate flight path for the UAV to land in the candidate alternate landing area is planned.

7. The method according to claim 6, characterized in that If the obstacles mentioned above exist, it also includes: Acquire environmental parameters of an area where the flight path is located, wherein the environmental parameters include wind direction; According to the wind direction, determining a direction around the obstacle where the wind direction is smaller; Obstacle avoidance planning is performed in a direction where the wind direction is smaller around the obstacle.

8. A drone landing device, characterized in that: include: An acquisition module, used to acquire ground images within the visual range of the drone; a processing module, configured to perform scene segmentation on the ground image, and obtain a plurality of candidate alternate landing areas from a ground range corresponding to the ground image based on the scene segmentation result; plan a candidate flight path for the drone to land at the candidate alternate landing area according to the current position of the drone and the position of the candidate alternate landing area; and determine a target flight path and a target alternate landing area corresponding to the target flight path according to the candidate flight path; An execution module is used to land at the target alternate landing area according to the target flight path.

9. An electronic device, characterized in that: include: A processor, a communication interface and a memory, wherein the processor is communicatively connected to the communication interface and the memory respectively; The memory stores computer-executable instructions; The communication interface communicates and interacts with external devices; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the drone landing method according to any one of claims 1 to 7.

Images