Method, device and system for controlling landing of unmanned aerial vehicle

Abstract

The invention discloses a method, device and system for controlling landing of an unmanned aerial vehicle, and belongs to the field of unmanned aerial vehicles. The method comprises the steps of determining a real-time position and a return position of an unmanned aerial vehicle in response to a received landing instruction, and planning a first path based on the real-time position and the return position; in the process of controlling the unmanned aerial vehicle to fly based on the first path, if the first distance between the real-time position of the unmanned aerial vehicle and the return flight position is smaller than or equal to a first distance threshold value, the vehicle position of the vehicle is obtained; determining a reference position by using the vehicle position and the return position, and planning a second path based on the real-time position of the unmanned aerial vehicle and the reference position; and in the process of controlling the unmanned aerial vehicle to fly based on the second path, if a second distance between the real-time position of the unmanned aerial vehicle and the reference position in the vertical direction is smaller than or equal to a second distance threshold value, controlling the unmanned aerial vehicle to execute landing operation until the unmanned aerial vehicle lands to the target position. The accuracy and safety of landing of the unmanned aerial vehicle are improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of drones, and in particular to a method, device, and system for controlling the landing of a drone. Background Art

[0002] Vehicle-mounted drones, a key application area of ​​drones, are increasingly being used in aerial photography and mapping, emergency rescue, logistics, and other fields. However, high positioning accuracy is required for drones to return to their vehicles.

[0003] To meet the high-precision positioning requirements of drones, related technologies use real-time kinematic (RTK) technology to achieve centimeter-level positioning accuracy. For example, RTK modules are installed on drones and vehicles, and differential correction information sent by base stations is used to improve the positioning accuracy of vehicles and drones. However, installing RTK modules on drones and vehicles not only increases the overall hardware cost of vehicle-mounted drones, but also requires additional network RTK service fees, which limits the application and promotion of vehicle-mounted drones.

[0004] Therefore, how to control the landing of drones has become an urgent problem to be solved. Summary of the Invention

[0005] This application provides a method, device, and system for controlling the landing of a drone, which can improve the accuracy and safety of the determined drone landing process. The technical solution is as follows:

[0006] In one aspect, an embodiment of the present application provides a method for controlling the landing of a drone, the method comprising:

[0007] In response to receiving the landing instruction, determining the real-time position and the return position of the UAV, and planning a first path based on the real-time position and the return position, wherein the return position is the position of the UAV hangar;

[0008] During the process of controlling the drone to fly based on the first path, if a first distance between the real-time position of the drone and the return position is less than or equal to a first distance threshold, obtaining a vehicle position of a vehicle where the drone lands;

[0009] Determining a reference position using the vehicle position and the return position, and planning a second path based on the real-time position of the drone and the reference position, wherein the reference position is a reference position of the drone when preparing to land;

[0010] During the process of controlling the drone to fly based on the second path, if a second distance in the vertical direction between the real-time position of the drone and the reference position is less than or equal to a second distance threshold, the drone is controlled to perform a landing operation until the drone lands at the target position.

[0011] On the other hand, an embodiment of the present application provides a device for controlling the landing of a drone, the device comprising:

[0012] a determination module, configured to determine a real-time position and a return position of the UAV in response to receiving a landing instruction, and plan a first path based on the real-time position and the return position, wherein the return position is a position of a UAV hangar;

[0013] an acquisition module, configured to acquire a vehicle position of a vehicle on which the drone has landed if a first distance between the real-time position of the drone and the return position is less than or equal to a first distance threshold during the process of controlling the drone to fly based on the first path;

[0014] The determining module determines a reference position using the vehicle position and the return position, and plans a second path based on the real-time position of the UAV and the reference position, wherein the reference position is a reference position of the UAV when preparing to land;

[0015] The control module is configured to control the drone to perform a landing operation until the drone lands at a target location if a second distance in a vertical direction between the real-time position of the drone and the reference position is less than or equal to a second distance threshold during the process of controlling the drone to fly based on the second path.

[0016] In a possible implementation, the determining module is configured to determine a first reference path based on the real-time position and the return position, where the first reference path is a straight line path between the real-time position and the return position, and includes a plurality of first nodes.

[0017] determining first reference information, where the first reference information includes at least one of terrain information, no-fly zone information, weather information, or obstacle information;

[0018] The position of the first node in the first reference path is optimized based on the first reference information, and the first path is determined based on the optimized position of the first node.

[0019] In a possible implementation, the determining module is configured to determine a first offset vector of the vehicle position relative to the return position in the same coordinate system;

[0020] Obtaining a safety distance threshold of the UAV corresponding to the vehicle position, and correcting the first offset vector using the safety distance threshold to obtain a second offset vector;

[0021] Predicting the time it takes for the UAV to reach the vehicle location, and correcting the second offset vector using the predicted time and flight parameters of the UAV to obtain a third offset vector;

[0022] The reference position is determined using the return position and the third offset vector.

[0023] In one possible implementation, the determining module is configured to determine a second reference path based on the real-time position of the drone and the reference position, where the second reference path is a straight line path between the real-time position and the reference position, and includes a plurality of second nodes.

[0024] determining second reference information and a target optimization function, wherein the second reference information includes real-time motion parameters of the vehicle position, and the target optimization function is determined based on at least one of a length, energy consumption, or collision risk of the second reference path;

[0025] The position of the second node in the second reference path is optimized using the second reference information and the target optimization function, and the second path is determined based on the optimized position of the second node.

[0026] In a possible implementation, the target position has a landing mark, and the acquisition module is further configured to acquire a reference image captured by the drone in a vertical direction, wherein the reference image includes the landing mark of the target position;

[0027] The control module is further configured to extract features of the landing mark in the reference image to obtain a first feature;

[0028] The first feature is matched with the second feature to obtain a matching result, where the second feature is a landing identification feature template pre-stored in the drone, and is used to determine whether the landing identification captured in the reference image is the landing identification corresponding to the drone. The matching result is used to indicate whether to control the drone to perform a landing operation.

[0029] In one possible implementation, the control module is further configured to, when the matching result indicates that the landing mark of the first feature is the landing mark corresponding to the drone, determine the first pose information of the drone relative to the landing mark using the pixel coordinates of the landing mark, the actual size of the landing mark, and acquisition parameters of the reference image acquired by the drone;

[0030] A visual landing control instruction for controlling the UAV to perform a landing operation is generated based on the first posture information and the second posture information, wherein the second posture information is posture information corresponding to safely landing at the landing mark.

[0031] In one possible implementation, the device further includes a generation module, which is configured to generate the landing instruction based on the landing interaction operation when the control application of the drone receives the landing interaction operation; or automatically generate the landing instruction when the battery level of the drone is lower than a battery threshold.

[0032] In a possible implementation, the determining module is configured to generate a first signal using a global positioning system of the drone, and determine first location information of the drone based on the first signal;

[0033] generating a second signal using an inertial measurement unit of the drone, and determining second position information of the drone based on the second signal;

[0034] generating a third signal using a data acquisition module of the drone, and determining third position information of the drone based on the third signal;

[0035] determining, based on signal qualities of the first signal, the second signal, and the third signal, reference weights corresponding to the first location information, the second location information, and the third location information, respectively;

[0036] The real-time position of the UAV is determined using the first position information, the second position information, the third position information, and the reference weight.

[0037] On the other hand, an embodiment of the present application provides a system for landing a drone, the system comprising: a drone body and a drone control system; a drone control system for controlling the drone body to execute any of the above-mentioned methods for controlling drone landing.

[0038] The technical solution provided by this application brings at least the following beneficial effects:

[0039] After receiving a landing command, the present application plans a first path based on the drone's real-time position and return position. When the drone's real-time position and return position are less than or equal to a first distance threshold, the reference position is re-determined and a second path is planned based on the vehicle's position and return position. Through multiple path planning, the drone is dynamically adapted to the mobile landing point (vehicle position). In the vertical direction, when the drone's real-time position and reference position are less than or equal to a second distance threshold, the drone is controlled to perform a landing operation, avoiding the drone's reliance on high-precision positioning during the landing process and reducing the cost of the drone. Furthermore, by using phased path planning, the system can flexibly adapt to changes in the vehicle's position during the drone's return, improving the accuracy and safety of the drone's landing. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0041] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0042] Figure 2 This is a flow chart of a method for landing a drone provided in an embodiment of the present application;

[0043] Figure 3 This is a structural diagram of a device for controlling the landing of a drone provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0045] It should be noted that the terms "first," "second," and the like in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0046] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present application. Figure 1As shown, vehicle 101 includes a smart hangar 103, which provides a landing platform for drone 102. Smart hangar 103 also integrates functional modules such as charging and data transmission. When drone 102 lands on smart hangar 103, it can automatically recharge and quickly transmit mission data back, ensuring the drone's continued operation. Smart hangar 103 can be installed on the roof of vehicle 101. The location of smart hangar 103 is not limited to this, and this application does not impose any restrictions on the location of smart hangar 103.

[0047] Drone 102 can be any unmanned aerial vehicle (UA) or unmanned aircraft (UA) that is operated remotely or autonomously. Drone 102 includes, but is not limited to, multi-rotor drones, fixed-wing drones, and composite-wing drones. It can be used in entertainment scenarios (e.g., travel photography, outdoor sports photography), commercial scenarios (e.g., topographic mapping, power inspections, logistics distribution), and military scenarios (e.g., reconnaissance and surveillance, target strikes, and communications relay).

[0048] During the flight of drone 102, drone 102 proactively initiates a connection request to vehicle 101 using its onboard wireless communication module. After vehicle 101 completes identity verification, a stable communication link is established between the two parties. Drone 102 then continuously transmits flight status information, such as altitude, speed, attitude, and remaining battery life, as well as mission data collected during mission execution, such as topographic mapping data and captured images and videos, to vehicle 101 according to established protocols. Vehicle 101 can display the drone's status on its onboard display screen for the operator's convenience and can also store mission data. Furthermore, the operator can send control commands to drone 102 through vehicle 101's control terminal to adjust its flight direction, altitude, speed, or mission parameters. Drone 102 receives the control commands, executes the corresponding control operations, and feeds back the results to vehicle 101.

[0049] During the landing process of the drone 102, after the drone 102 receives the landing command, the control system carried by the drone 102 determines the real-time position of the drone and the intelligent hangar 103 (return position) on the vehicle 101 and plans a first path, controlling the drone 102 to fly along this path; when the first distance between the real-time position of the drone 102 and the return position is less than or equal to the first distance threshold, the position of the vehicle 101 is obtained through wireless communication, and the vehicle position and the return position are used to determine the preparatory landing reference position (reference position), and then the second path is planned based on the real-time position and reference position of the drone 102; when the second distance between the real-time position of the drone 102 and the reference position in the vertical direction is less than or equal to the second distance threshold, the control system of the drone 102 controls the drone 102 to perform the landing operation until it lands smoothly at the target position.

[0050] Based on the above Figure 1 In the implementation environment shown, an embodiment of the present application provides a method for landing a drone. Figure 2 This is a flow chart of a drone landing method provided by an embodiment of the present application. Figure 2 As shown, this method can be Figure 1 The vehicle 101 and the drone 102 interact and execute the method. The method may include steps 201 to 204.

[0051] In step 201, in response to receiving a landing command, the real-time position and the return position of the UAV are determined, and a first path is planned based on the real-time position and the return position, where the return position is the position of the UAV hangar.

[0052] In the exemplary embodiment of the present application, the landing instruction is an explicit signal or command received by the drone control system, indicating that the drone has transitioned from the current flight state to the landing preparation and execution phase. The landing instruction can be generated in a variety of ways.

[0053] In one embodiment, when the drone's control application receives a landing interaction, it generates a landing command based on the landing interaction. For example, while the drone is in flight, the operator can issue a landing command through the control application connected to the drone. The control application (APP) can be installed on a vehicle-mounted terminal, a mobile phone, or a ground control station.

[0054] The operator can perform interactive landing operations within the control application interface. For example, the control application interface displays landing controls that, when triggered, generate landing instructions. Another example is the control application interface displaying an input area that guides the operator to enter landing parameters. The control application then normalizes these parameters and generates landing instructions.

[0055] In another embodiment, when the battery level of the drone is lower than a battery threshold, a landing command is automatically generated. Exemplarily, the drone control system collects the battery level of the drone in real time and continuously compares the battery level with a preset battery threshold. When the battery level of the drone is lower than the battery threshold, the drone control system automatically triggers a landing command generation mechanism to generate a landing command. The battery threshold is set based on the remaining endurance of the drone, flight speed, environmental factors (such as wind speed, altitude), etc., to ensure that the drone has enough power to return safely before the battery is exhausted.

[0056] The embodiments of the present application generate landing instructions based on interactive landing operations, enabling operators to flexibly control the generation time of landing instructions based on actual mission requirements and on-site environment, and use landing instructions to control the landing of the drone, thereby improving the flexibility and personalization of the drone landing process and better meeting landing requirements in different scenarios. The method of automatically generating landing instructions based on the battery level falling below a battery threshold automatically generates landing instructions when the drone's battery level falls below the battery threshold, preventing the drone from being unable to return due to exhaustion, reducing the risk of damage to the drone, and ensuring the safety and reliability of the drone.

[0057] In an exemplary embodiment of the present application, after receiving the landing instruction, the real-time position and return position of the drone are determined. The process of determining the real-time position of the drone includes but is not limited to steps A1 to A5.

[0058] In step A1, a first signal is generated by using the global positioning system of the drone, and first position information of the drone is determined based on the first signal.

[0059] For example, a drone is equipped with a Global Positioning System (GPS) module. The GPS module continuously receives positioning signals (first signals) from multiple satellites (e.g., at least four satellites). The first signals include the satellite's position information and timestamp information. The GPS module processes and analyzes the multiple first signals and calculates the drone's longitude, latitude, and altitude coordinates in the Earth coordinate system using the principle of triangulation, thereby obtaining the drone's first position information.

[0060] In step A2, a second signal is generated by using an inertial measurement unit of the drone, and second position information of the drone is determined based on the second signal.

[0061] For example, the drone is equipped with an inertial measurement unit (IMU), which is usually composed of an accelerometer and a gyroscope. The accelerometer will continuously measure the acceleration data of the drone on the three axes (X, Y, and Z), and the gyroscope will continuously measure the angular velocity data of the drone around the three axes. The acceleration data and angular velocity data together constitute the second signal. The IMU performs integration operations and other processing and analysis on the second signal at multiple moments. First, the velocity information of the drone on each axis is obtained based on the integration of the acceleration data, and then the displacement information of the drone on each axis is obtained based on the integration of the velocity information. Then, combined with the initial position information of the drone, the position change of the drone in the body coordinate system is calculated, and then it is converted to the earth coordinate system to obtain the second position information of the drone.

[0062] In step A3, a third signal is generated by using the data acquisition module of the drone, and third position information of the drone is determined based on the third signal.

[0063] For example, the drone is equipped with an image acquisition module (e.g., a binocular camera). This module is used to capture images of the drone's surroundings, with the captured image data constituting the third signal. Using visual odometry (VO) technology, two adjacent image frames are processed to determine the change in the drone's position between the two adjacent images. The position changes of the drone in each image are accumulated and combined with the drone's initial position information to calculate the drone's third position information.

[0064] For example, a feature extraction algorithm is first used to detect feature points such as corners and edges in the image. A feature matching algorithm (such as Hamming distance matching) is then used to find corresponding pairs of feature points in the two image frames. Based on the internal and external parameters corresponding to the image acquisition module, the coordinates of the feature points in three-dimensional space and the rotation and translation vectors of the drone during the interval between the two image frames are calculated, thereby determining the displacement of the drone relative to the previous image frame. By repeating this process and combining it with the drone's initial position information, the drone's third position information is calculated.

[0065] In step A4, reference weights corresponding to the first position information, the second position information, and the third position information are determined based on the signal qualities of the first signal, the second signal, and the third signal, respectively.

[0066] Exemplarily, the quality of the GPS signal (first signal), the IMU signal (second signal) and the image acquisition module output signal (third signal) are evaluated respectively. For the GPS signal, the signal quality is measured by the signal-to-noise ratio (SNR). The higher the signal-to-noise ratio, the better the signal quality and the higher the positioning reliability. The signal quality of the IMU signal is evaluated by detecting the stability and error range of the sensor data. The smaller the error fluctuation, the better the signal quality. The signal quality of the image acquisition module is quantified based on indicators such as image clarity, the number of feature points or the density of the lidar point cloud. The higher the image clarity, the more complete the details are retained, and the better the signal quality; the more feature points in the image and the more evenly distributed they are, the richer the information contained in the image, the more effective data used for positioning, and the better the signal quality.

[0067] Based on signal quality, corresponding reference weights are assigned to the first, second, and third location information. Better signal quality indicates more reliable location information provided by the signal, and the corresponding reference weight increases. For example, when the GPS signal is strong, the reference weight of the first location information will be higher. When the GPS signal is blocked, the reference weight of the IMU or data acquisition module signal can be increased.

[0068] In step A5, the real-time position of the UAV is determined using the first position information, the second position information, the third position information, and the reference weight.

[0069] The first position information, the second position information, the third position information and their corresponding reference weights are used to fuse the three position information through weighted averaging or other fusion algorithms to determine the real-time position of the drone.

[0070] It should be noted that the method for determining the real-time position of a drone in this application is an exemplary description. The real-time position of a drone can also be determined based on one or two of the first position information, the second position information or the third position information. This application does not impose any restrictions on this.

[0071] The embodiment of the present application uses the signals of the drone's global positioning system module, inertial measurement unit and data acquisition module to determine the corresponding position information based on each signal, and dynamically allocates reference weights according to the quality of each signal. The real-time position of the drone is determined using each position information and the corresponding reference weight, which can give full play to the advantages of different positioning methods, avoid the inaccurate real-time position of the drone determined by a single positioning method, and improve the positioning precision and accuracy of the real-time position.

[0072] In an exemplary embodiment of the present application, the return position is a pre-set or automatically recorded approximate position for the drone to return to when landing. The return position can be specified by the operator on a map through a control application, such as specifying the drone hangar position, a safe landing position, etc. This application is explained using the drone hangar position as an example. In the process of communication between the drone and the drone hangar, the return position corresponding to the drone hangar has a high refresh rate compared to the vehicle position, but its accuracy is poor. In the process of communication between the drone and the drone hangar, the return position corresponding to the drone hangar has a high refresh rate, but poor accuracy, compared to the vehicle position. The drone hangar is fixed on the vehicle, and the return position can change during the flight or landing of the drone. For example, when the vehicle is driving, the drone hangar fixed on the vehicle also moves, causing the return position to change. In addition, the return position can also be a fixed position. For example, when the vehicle is stationary, the return position does not change.

[0073] After determining the real-time position and the return position, a first route is planned based on the real-time position and the return position. The process of planning the first route based on the real-time position and the return position includes steps B1 to B3.

[0074] In step B1, a first reference path is determined based on the real-time position and the return position. The first reference path is a straight line path between the real-time position and the return position. The first reference path includes a plurality of first nodes.

[0075] Exemplarily, after determining the real-time location and the return location, a straight line path connecting the real-time location and the return location is determined, i.e., a first reference path. The first reference path is discretized into a plurality of first nodes, which are evenly or unevenly distributed on the first reference path. Each node represents a specific location point on the first reference path.

[0076] In step B2, first reference information is determined, where the first reference information includes at least one of terrain information, no-fly zone information, weather information, or obstacle information.

[0077] For example, the drone obtains terrain information by using a built-in terrain mapping module to obtain data such as terrain undulations, mountain heights, and canyon distribution in the surrounding area; or obtains terrain information around the drone from a pre-stored high-precision database combined with the drone's real-time position.

[0078] Obtain no-fly zone information through the control application or relevant management departments. No-fly zones include but are not limited to areas around airports, military control areas, densely populated areas, and prohibited flight areas set by operators.

[0079] The drone is equipped with a meteorological detection sensor, which monitors meteorological conditions such as wind speed, wind direction, rainfall, and lightning in real time to obtain meteorological information. It can also receive regional weather forecast data released by the meteorological department to determine the meteorological data around the drone's real-time location at the current moment and obtain meteorological information.

[0080] Drones are equipped with sensors such as lidar and visual cameras. These sensors scan the surrounding environment in real time, detect the position, shape and size of obstacles such as buildings, trees, wires, and other aircraft in the air, and obtain obstacle information.

[0081] The first reference information is determined using one or more of terrain information, no-fly zone information, weather information, or obstacle information. It should be noted that the first reference information in this application is an exemplary description, and the first reference information can also be determined based on actual conditions, and this application does not impose any limitation on this.

[0082] In step B3, the position of the first node in the first reference path is optimized based on the first reference information, and the first path is determined based on the optimized position of the first node.

[0083] Exemplarily, the terrain information, no-fly zone information, meteorological information or obstacle information in the first reference information are used to determine the constraint conditions. For example, if there is a complex terrain (such as peaks, canyons, etc.), the elevation data of the terrain is obtained, and the terrain surface is converted into a series of height inequality constraints to obtain the constraint conditions corresponding to the terrain information. If there is a no-fly zone, the boundary of the no-fly zone is converted into an inequality constraint to ensure that the first node does not fall into the no-fly zone, and the constraint conditions corresponding to the no-fly zone information are obtained. If there are adverse meteorological factors (such as strong winds, heavy rains, etc.), based on meteorological forecast data or real-time monitoring data, the meteorological conditions are converted into inequality constraints that limit the flight speed, direction or flight altitude of the drone, and the constraint conditions corresponding to the meteorological information are obtained. If there are obstacles (such as buildings, trees, etc.), the spatial shape and position information of the obstacles are converted into distance inequality constraints to obtain the constraint conditions corresponding to the obstacle information.

[0084] The positions of the first nodes in the first reference path are optimized using the constraints. The positions of the first nodes are continuously optimized until all the first nodes in the first path meet the constraints. The first paths are obtained by sequentially connecting the optimized first node positions.

[0085] The embodiment of the present application uses a straight line path between the real-time position and the return position as the first reference path, and then optimizes the first node in the first reference path in combination with various information such as terrain, no-fly zones, weather and obstacles. It can comprehensively consider various factors during the flight of the drone, so that the planned first path meets flight safety requirements, improves the success rate of the drone landing process, reduces the risks of collision and entry into no-fly zones due to unreasonable paths, and ensures the flight safety of the drone when flying based on the first path.

[0086] In step 202, during the process of controlling the drone to fly based on the first path, if the first distance between the real-time position of the drone and the return position is less than or equal to the first distance threshold, the vehicle position of the vehicle is obtained, and the vehicle is the vehicle where the drone lands.

[0087] In an exemplary embodiment of the present application, after determining the first path, the drone control system controls the drone to fly based on the first path. During the flight process based on the first path, the drone control system continuously monitors the distance (first distance) between the real-time position of the drone and the return position, and compares the first distance with the first distance threshold. When the first distance is less than or equal to the first distance threshold, it indicates that the drone is near the return position, and the drone control system automatically obtains the vehicle position, which is a vehicle pre-designated for drone landing, such as a vehicle in a fixed drone hangar. For example, the vehicle is equipped with a positioning module (such as a GPS module), and the positioning module transmits the vehicle position to the drone control system in real time. Among them, the first distance threshold can be set based on the actual situation of the drone landing process, and this application does not limit this. For example, the first distance threshold is 5 meters.

[0088] In step 203, a reference position is determined using the vehicle position and the return position, and a second path is planned based on the real-time position of the UAV and the reference position. The reference position is a base position of the UAV when preparing to land.

[0089] In an exemplary embodiment of the present application, after obtaining the vehicle position, the reference position is determined using the vehicle position and the return position. The process of determining the reference position using the vehicle position and the return position includes but is not limited to steps C1 to C4.

[0090] In step C1 , a first offset vector of the vehicle position relative to the home position is determined in the same coordinate system.

[0091] For example, the coordinate reference systems of the drone's real-time position, the vehicle's position, and the return position are unified (e.g., to the Earth coordinate system) to ensure consistency of the position data. In the Earth coordinate system, the coordinate difference between the vehicle's position and the return position is calculated using the three-dimensional coordinates of the drone's real-time position, the return position, and the vehicle's position to generate a first offset vector.

[0092] In step C2, a safety distance threshold of the UAV corresponding to the vehicle position is obtained, and the first offset vector is corrected using the safety distance threshold to obtain a second offset vector.

[0093] For example, a safety distance threshold is preset based on the drone's model, payload, and flight status. The safety distance threshold is used to ensure flight safety when the drone approaches a vehicle. The modulus of the first offset vector is adjusted according to the safety distance threshold, or the direction of the first offset vector is fine-tuned to meet the safety distance requirement, thereby generating the second offset vector.

[0094] In step C3, the time it takes for the UAV to reach the vehicle's location is predicted, and the second offset vector is corrected using the predicted time and the flight parameters of the UAV to obtain a third offset vector.

[0095] Exemplarily, the time it takes for the drone to reach the vehicle's location is predicted to obtain a predicted time. For example, based on information such as the drone's current flight speed, flight direction, and distance from the vehicle's location, combined with the drone's flight parameters (such as maximum acceleration, maximum deceleration, etc.), the predicted time it takes for the drone to reach the vehicle's location is predicted. It should be noted that when the vehicle's position changes while the vehicle is in motion, the predicted time changes accordingly. The second offset vector is then corrected using the predicted time and flight parameters to obtain a third offset vector. For example, at least one of the direction and magnitude of the second offset vector is further adjusted to obtain a third offset vector.

[0096] In step C4 , a reference position is determined using the home position and the third offset vector.

[0097] Exemplarily, the coordinates of the return position are vector-superimposed on the third offset vector to obtain the coordinates of the reference position, where the reference position is a reference point for preparing to land, for example, a position above the vehicle.

[0098] The embodiment of the present application determines the first offset vector of the vehicle position relative to the return position in the same coordinate system, providing a unified basis for subsequent calculations. The first offset vector is corrected in combination with the safety distance threshold to obtain a second offset vector, thereby avoiding collisions between the drone and the vehicle or the surrounding environment during landing and ensuring the safety of the drone landing process. Then, the second offset vector is further corrected based on the predicted time it takes for the drone to reach the vehicle position and the flight parameters to obtain a third offset vector. The third offset vector and the return position are used to determine the reference position, so that the drone can reach the reference position more accurately and land based on the reference position in the subsequent process, thereby improving the accuracy, efficiency and safety of the drone landing.

[0099] In an exemplary embodiment of the present application, after determining the reference position, a second path is planned based on the real-time position of the drone and the reference position. The process of planning the second path based on the real-time position of the drone and the reference position includes but is not limited to steps D1 to D3.

[0100] In step D1, a second reference path is determined based on the real-time position and the reference position of the UAV. The second reference path is a straight line path between the real-time position and the reference position, and includes a plurality of second nodes.

[0101] It should be noted that the process of determining the second reference path is similar to the process of determining the first reference path. Please refer to the relevant description of step B1, and this application will not elaborate on it here.

[0102] In step D2, second reference information and a target optimization function are determined, the second reference information including real-time motion parameters of the vehicle position, and the target optimization function is determined based on at least one of the length, energy consumption, or collision risk of the second reference path.

[0103] Exemplarily, the motion parameters of the vehicle position are obtained in real time through a wireless communication link between the vehicle and the drone (such as 4G / 5G (fourth generation / fifth generation mobile communication technology) or a dedicated data link), where the motion parameters include but are not limited to moving speed, moving defense line, and moving acceleration.

[0104] The target optimization function is constructed based on indicators such as path length, energy consumption and collision risk. The target optimization function is: Among them, u is the second node to be optimized, M1 is the reference position, is the square of the Euclidean distance between u and M1, which is used to optimize the path length. λ is the first weight, ||u||1 is the Manhattan distance of u. The more dispersed the second node in the second reference path is and the larger the absolute value of the coordinate is, the larger the Manhattan distance is, which is used to optimize energy consumption. α is the second weight, M2(t) is the predicted position of the vehicle at time t, ||u-M2(t)|| 2 is the square of the distance between u and the predicted position of the vehicle at time t. β is the third weight, and Map(u) is the obstacle information around the second node to be optimized. The closer and larger the obstacle, the larger the value of Map(u), which is used to optimize the collision risk. It should be noted that the values ​​of the first, second, and third weights can be set based on actual conditions and are not limited in this application.

[0105] In step D3, the position of the second node in the second reference path is optimized using the second reference information and the target optimization function, and the second path is determined based on the optimized position of the second node.

[0106] For example, based on the real-time motion parameters of the vehicle positions in the second reference information, constraints related to vehicle motion coordination are identified within the target optimization function. The target optimization function is then combined with each second node position as an optimization variable and substituted into the function calculation. The second node positions are iteratively adjusted to minimize the target optimization function while satisfying the constraints, continuously optimizing each second node position. Once the target optimization function converges, a second path is determined based on the optimized second node sequence.

[0107] It should be noted that the constraint conditions have been described in step B3, and reference may be made to the relevant description of step B1, which will not be repeated in this application. In addition, the method for determining the second path in this application is an exemplary description, and other methods may also be used to determine the second path, which is not limited in this application.

[0108] The embodiment of the present application optimizes the second node in the second reference path by taking the straight path between the real-time position and the reference position of the UAV as the second reference path, and combining the real-time motion parameters of the vehicle position and the target optimization function determined based on factors such as path length, energy consumption or collision risk. It comprehensively considers various dynamic factors in the UAV flight and landing process, such as vehicle movement, path length, energy consumption and collision risk, so that the planned second path meets actual flight requirements and is safe and efficient, thereby improving the accuracy and success rate of the UAV landing and reducing the collision risk and energy waste caused by unreasonable path.

[0109] In step 204, during the process of controlling the drone to fly based on the second path, if the second distance between the real-time position of the drone and the reference position in the vertical direction is less than or equal to the second distance threshold, the drone is controlled to perform a landing operation until the drone lands at the target position.

[0110] In an exemplary embodiment of the present application, after determining the second path, the drone control system controls the drone to fly based on the second path. During the flight process based on the second path, the drone control system continuously monitors the distance (second distance) between the drone's real-time position and the reference position in the vertical direction, and compares the second distance with the second distance threshold. When the second distance is less than or equal to the second distance threshold, it indicates that the drone has arrived near the reference position, the drone control system obtains the target position, and generates a visual landing control instruction based on the target position. Among them, the second distance threshold can be set based on the actual situation of the drone landing process, and this application does not limit this.

[0111] The target position has a landing mark, and the process of generating a visual landing control instruction based on the target position includes but is not limited to steps E1 to E5.

[0112] In step E1, a reference image captured by the UAV in the vertical direction is obtained, where the reference image includes a landing mark of the target position.

[0113] For example, the drone is equipped with an image acquisition device (such as a high-definition camera). During the landing process, the acquisition angle of the image acquisition device is controlled to capture images in a vertical direction. The captured image, including the image of the landing mark, is determined as the reference image. The landing mark can be a specific pattern, color combination, or shape used to guide the drone to land accurately. For example, the landing mark can be a QR code of a specific shape, a reflective mark, or a geometric pattern.

[0114] In step E2, feature extraction is performed on the landing mark in the reference image to obtain a first feature.

[0115] For example, key information representing the landing sign is determined in the reference image. This key information includes, but is not limited to, the edges, corners, and texture of the reference image. An edge detection algorithm (such as the Canny edge detection algorithm) is then used to extract the edge information of the landing sign to obtain edge features. A corner detection algorithm (such as the Harris corner detection algorithm) is then used to extract the corner information of the landing sign to obtain corner features. These features are then fused to obtain the first feature.

[0116] In step E3, the first feature is matched with the second feature to obtain a matching result. The second feature is a landing mark feature template pre-stored in the drone, which is used to determine whether the landing mark collected in the reference image is the landing mark corresponding to the drone. The matching result is used to indicate whether to control the drone to perform a landing operation.

[0117] For example, the second feature is a feature template of a landing sign, obtained by pre-collecting a large number of landing sign images and performing feature extraction. The feature template of the drone's landing sign is uniquely mapped to the drone. After obtaining the first feature, the similarity between the first and second features is calculated based on a feature matching algorithm. If the similarity exceeds a certain threshold, the match is considered successful, meaning that the landing sign in the reference image is the corresponding landing sign for the drone, and the drone can be controlled to execute a landing operation. Otherwise, the match is considered a failure, and the drone is controlled to continue flying or other measures are taken.

[0118] In step E4, when the landing mark represented by the first feature of the matching result is the landing mark corresponding to the drone, the first position information of the drone relative to the landing mark is determined using the pixel coordinates of the landing mark, the actual size of the landing mark, and the acquisition parameters of the reference image collected by the drone.

[0119] Exemplarily, the pixel coordinates are the position coordinates of the landing mark in the reference image, which are obtained through image processing. The actual size of the landing mark is the real size of the landing mark in the real world and is known in advance. The acquisition parameters include the intrinsic parameters, extrinsic parameters, sensor size, etc. of the camera. With this information, the conversion relationship between the pixel coordinate system and the earth coordinate system is established using the camera imaging principle, so as to calculate the position and attitude information of the UAV relative to the landing mark, that is, the first attitude information. The first attitude information includes the position (such as three-dimensional coordinates) and attitude (such as pitch angle, yaw angle, roll angle) of the UAV.

[0120] In step E5, a visual landing control instruction for controlling the UAV to perform a landing operation is generated based on the first posture information and the second posture information, where the second posture information is posture information corresponding to a safe landing to a landing mark.

[0121] Exemplarily, the second posture information is the posture information corresponding to a safe landing on the landing mark, which is a pre-set ideal position and posture that the drone should reach when landing. By comparing the first posture information and the second posture information, the position and posture change that the drone needs to adjust are calculated. Then, the corresponding visual landing control instructions are generated based on the position and posture change. The visual landing control instructions are used to control the flight attitude and speed of the drone, so that it gradually adjusts to the state corresponding to the second posture information, so that the drone lands on the landing mark at the target position. For example, if the first posture information shows that the drone is too high relative to the landing mark and its posture is skewed, then the generated visual landing control instructions will control the drone to lower its altitude and adjust its posture so that it lands accurately on the landing mark.

[0122] The embodiment of the present application obtains a reference image containing a landing mark collected by the drone in the vertical direction, extracts and matches the landing mark in the image, and after a successful match, uses the pixel coordinates, actual size and acquisition parameters of the landing mark to calculate the first pose information of the drone relative to the landing mark, and generates visual landing control instructions based on the first pose information and the safe landing pose information (second pose information) to accurately guide the drone to identify and locate the target landing mark, adjust the flight attitude and position of the drone in real time, ensure that the drone lands safely and accurately at the target location, and improve the automation and accuracy of the drone landing.

[0123] The present application also provides a device for controlling the landing of a drone. Figure 3 FIG. 1 is a schematic diagram of a device for controlling the landing of a drone provided in an embodiment of the present application. Figure 3 As shown, the device includes:

[0124] A determination module 301 is configured to determine a real-time position and a return position of the UAV in response to receiving a landing instruction, and plan a first path based on the real-time position and the return position, where the return position is a location of a UAV hangar;

[0125] an acquisition module 302 for acquiring a vehicle position of a vehicle on which the drone has landed if a first distance between the real-time position of the drone and the return position is less than or equal to a first distance threshold during the process of controlling the drone to fly based on the first path;

[0126] Determination module 301, using the vehicle position and the return position to determine a reference position, and planning a second path based on the real-time position of the UAV and the reference position, where the reference position is a reference position of the UAV when preparing to land;

[0127] The control module 303 is used to control the drone to perform a landing operation until the drone lands at the target position if the second distance between the real-time position of the drone and the reference position in the vertical direction is less than or equal to the second distance threshold during the process of controlling the drone to fly based on the second path.

[0128] In one possible implementation, the determination module 301 is used to determine a first reference path based on the real-time position and the return position, where the first reference path is a straight line path between the real-time position and the return position, and the first reference path includes multiple first nodes; determine first reference information, where the first reference information includes at least one of terrain information, no-fly zone information, weather information, or obstacle information; optimize the position of the first node in the first reference path based on the first reference information, and determine the first path based on the optimized position of the first node.

[0129] In one possible implementation, the determination module 301 is used to determine a first offset vector of the vehicle position relative to the return position in the same coordinate system; obtain a safety distance threshold of the drone corresponding to the vehicle position, and use the safety distance threshold to correct the first offset vector to obtain a second offset vector; predict the time it takes for the drone to reach the vehicle position, and use the predicted time and the flight parameters of the drone to correct the second offset vector to obtain a third offset vector; and determine a reference position using the return position and the third offset vector.

[0130] In one possible implementation, a determination module 301 is used to determine a second reference path based on the real-time position and reference position of the UAV, where the second reference path is a straight line path between the real-time position and the reference position, and the second reference path includes multiple second nodes; determine second reference information and a target optimization function, where the second reference information includes real-time motion parameters of the vehicle position, and the target optimization function is determined based on at least one of the length, energy consumption, or collision risk of the second reference path; use the second reference information and the target optimization function to optimize the position of the second node in the second reference path, and determine the second path based on the optimized position of the second node.

[0131] In a possible implementation, the target location has a landing mark, and the acquisition module 302 is further configured to acquire a reference image captured by the drone in a vertical direction, the reference image including the landing mark of the target location;

[0132] The control module 303 is also used to extract features of the landing mark in the reference image to obtain a first feature; match the first feature with the second feature to obtain a matching result, where the second feature is a landing mark feature template pre-stored in the drone, and is used to determine whether the landing mark collected in the reference image is the landing mark corresponding to the drone. The matching result is used to indicate whether to control the drone to perform a landing operation.

[0133] In one possible implementation, the control module 303 is further used to determine the first pose information of the drone relative to the landing mark using the pixel coordinates of the landing mark, the actual size of the landing mark, and the acquisition parameters of the reference image collected by the drone when the landing mark represented by the first feature in the matching result is the landing mark corresponding to the drone; and generate a visual landing control instruction for controlling the drone to perform a landing operation based on the first pose information and the second pose information, where the second pose information is the pose information corresponding to the safe landing to the landing mark.

[0134] In one possible implementation, the device also includes a generation module (not shown in the figure), which is used to generate a landing instruction based on the landing interaction operation when the control application of the drone receives a landing interaction operation; or automatically generate a landing instruction when the battery level of the drone is lower than a battery threshold.

[0135] In one possible implementation, the determination module 301 is used to generate a first signal using the global positioning system of the drone, and determine the first position information of the drone based on the first signal; generate a second signal using the inertial measurement unit of the drone, and determine the second position information of the drone based on the second signal; generate a third signal using the data acquisition module of the drone, and determine the third position information of the drone based on the third signal; determine the reference weights corresponding to the first position information, the second position information and the third position information based on the signal qualities of the first signal, the second signal and the third signal respectively; and determine the real-time position of the drone using the first position information, the second position information, the third position information and the reference weights.

[0136] After receiving a landing command, the present application plans a first path based on the drone's real-time position and return position. When the drone's real-time position and return position are less than or equal to a first distance threshold, the reference position is re-determined and a second path is planned based on the vehicle's position and return position. Through multiple path planning, the drone is dynamically adapted to the mobile landing point (vehicle position). In the vertical direction, when the drone's real-time position and reference position are less than or equal to a second distance threshold, the drone is controlled to perform a landing operation, avoiding the drone's reliance on high-precision positioning during the landing process and reducing the cost of the drone. Furthermore, by using phased path planning, the system can flexibly adapt to changes in the vehicle's position during the drone's return, improving the accuracy and safety of the drone's landing.

[0137] It should be understood that the above-described apparatus, when implementing its functions, is merely illustrated by the division of the above-described functional modules. In actual applications, the above-described functions can be distributed among different functional modules as needed, that is, the internal structure of the drone can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above-described embodiments are based on the same concept. The specific implementation process is detailed in the method embodiments and will not be repeated here.

[0138] The present application may also provide a drone landing system, comprising: a drone body and a drone control system, wherein the drone control system is configured to control the drone body to execute any of the above-mentioned methods for controlling drone landing. The details of the drone control system's method for controlling drone landing have been described in detail above and will not be repeated here.

[0139] It should be understood that the term "plurality" used herein refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0140] The above description is merely an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for controlling the landing of a drone, characterized in that: The method comprises: In response to receiving the landing instruction, determining the real-time position and the return position of the UAV, and planning a first path based on the real-time position and the return position, wherein the return position is the position of the UAV hangar; During the process of controlling the drone to fly based on the first path, if a first distance between the real-time position of the drone and the return position is less than or equal to a first distance threshold, obtaining a vehicle position of a vehicle where the drone lands; Determining a reference position using the vehicle position and the return position, and planning a second path based on the real-time position of the drone and the reference position, wherein the reference position is a reference position of the drone when preparing to land; During the process of controlling the drone to fly based on the second path, if a second distance in the vertical direction between the real-time position of the drone and the reference position is less than or equal to a second distance threshold, the drone is controlled to perform a landing operation until the drone lands at the target position.

2. The method according to claim 1, characterized in that Planning a first path based on the real-time position and the return position includes: Determine a first reference path based on the real-time position and the return position, where the first reference path is a straight line path between the real-time position and the return position, and includes a plurality of first nodes; determining first reference information, where the first reference information includes at least one of terrain information, no-fly zone information, weather information, or obstacle information; The position of the first node in the first reference path is optimized based on the first reference information, and the first path is determined based on the optimized position of the first node.

3. The method according to claim 1, characterized in that The determining of a reference position by using the vehicle position and the return position includes: Determining a first offset vector of the vehicle position relative to the return position in the same coordinate system; Obtaining a safety distance threshold of the UAV corresponding to the vehicle position, and correcting the first offset vector using the safety distance threshold to obtain a second offset vector; Predicting the time it takes for the UAV to reach the vehicle location, and correcting the second offset vector using the predicted time and flight parameters of the UAV to obtain a third offset vector; The reference position is determined using the return position and the third offset vector.

4. The method according to claim 1, wherein Planning a second path based on the real-time position of the UAV and the reference position includes: Determining a second reference path based on the real-time position of the UAV and the reference position, where the second reference path is a straight line path between the real-time position and the reference position, and includes a plurality of second nodes on the second reference path; determining second reference information and a target optimization function, wherein the second reference information includes real-time motion parameters of the vehicle position, and the target optimization function is determined based on at least one of a length, energy consumption, or collision risk of the second reference path; The position of the second node in the second reference path is optimized using the second reference information and the target optimization function, and the second path is determined based on the optimized position of the second node.

5. The method according to claim 1, wherein The target position has a landing mark, and before controlling the UAV to perform the landing operation, the method further includes: Acquire a reference image captured by the UAV in a vertical direction, wherein the reference image includes a landing mark of the target position; Extracting features of the landing mark in the reference image to obtain a first feature; The first feature is matched with the second feature to obtain a matching result, where the second feature is a landing identification feature template pre-stored in the drone, and is used to determine whether the landing identification captured in the reference image is the landing identification corresponding to the drone. The matching result is used to indicate whether to control the drone to perform a landing operation.

6. The method according to claim 5, characterized in that The method further comprises: When the matching result indicates that the landing mark of the first feature is the landing mark corresponding to the drone, determining the first pose information of the drone relative to the landing mark using the pixel coordinates of the landing mark, the actual size of the landing mark, and the acquisition parameters of the reference image acquired by the drone; A visual landing control instruction for controlling the UAV to perform a landing operation is generated based on the first posture information and the second posture information, wherein the second posture information is posture information corresponding to safely landing at the landing mark.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: When the control application of the drone receives a landing interaction operation, generating the landing instruction based on the landing interaction operation; Alternatively, when the battery level of the UAV is lower than a battery threshold, the landing instruction is automatically generated.

8. The method according to any one of claims 1 to 6, characterized in that: Determining the real-time location of the drone includes: generating a first signal using a global positioning system of the drone, and determining first location information of the drone based on the first signal; generating a second signal using an inertial measurement unit of the drone, and determining second position information of the drone based on the second signal; generating a third signal using a data acquisition module of the drone, and determining third position information of the drone based on the third signal; determining, based on signal qualities of the first signal, the second signal, and the third signal, reference weights corresponding to the first location information, the second location information, and the third location information, respectively; The real-time position of the UAV is determined using the first position information, the second position information, the third position information, and the reference weight.

9. A device for controlling the landing of a drone, characterized in that: The device comprises: a determination module, configured to determine a real-time position and a return position of the UAV in response to receiving a landing instruction, and plan a first path based on the real-time position and the return position, wherein the return position is a position of a UAV hangar; an acquisition module, configured to acquire a vehicle position of a vehicle on which the drone has landed if a first distance between the real-time position of the drone and the return position is less than or equal to a first distance threshold during the process of controlling the drone to fly based on the first path; The determining module determines a reference position using the vehicle position and the return position, and plans a second path based on the real-time position of the UAV and the reference position, wherein the reference position is a reference position of the UAV when preparing to land; The control module is configured to control the drone to perform a landing operation until the drone lands at a target location if a second distance in a vertical direction between the real-time position of the drone and the reference position is less than or equal to a second distance threshold during the process of controlling the drone to fly based on the second path.

10. A system for landing a drone, characterized in that: The system comprises: The drone itself; A drone control system, used to control the drone body to execute the method for controlling the landing of a drone as described in any one of claims 1 to 8.

Images