Unmanned aerial vehicle landing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to an unmanned aerial vehicle landing method and device, computer equipment, a readable storage medium and a program product. When the landing mark is not recognized, the landing mark is recognized and the unmanned aerial vehicle is controlled to be in a landing posture according to a search picture obtained by the unmanned aerial vehicle flying at a second shooting view angle and according to a target mode, and the moving track of the landing position is determined according to the vehicle driving information. And controlling the unmanned aerial vehicle to land to the landing position based on the predicted moving track, the interference compensation control parameter corresponding to the environment interference information of the unmanned aerial vehicle and the landing pose. Compared with a traditional method of directly landing after identifying the landing target, the scheme searches the unidentified landing identifier through a plurality of shooting view angles, predicts the trajectory of the landing position, outputs the interference compensation control parameters of the environmental interference information, and combines the predicted trajectory of the landing position to improve the accuracy of the landing target. And the compensated control parameters and the landing pose control the unmanned aerial vehicle to land to the moving landing position, so that the landing accuracy of the unmanned aerial vehicle is improved.

Description

Technical Field

[0001] The present application relates to the field of drone technology, and in particular to a drone landing method, device, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of computer technology, drone technology has gradually matured. After the drone performs tasks such as shooting, it needs to land on a moving target to realize the recovery of the drone. Therefore, it is necessary to ensure the accuracy of the drone landing on the moving target. At present, the method of landing a drone on a moving target is usually to land directly after identifying the target. However, due to the influence of the environment and the initial position of the drone, the drone will not be able to land when it loses the identified target, resulting in a decrease in the landing accuracy.

[0003] Therefore, the current UAV landing method for moving targets has the defect of low landing accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a drone landing method, device, computer equipment, computer-readable storage medium and computer program product that can improve landing accuracy in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for landing a drone, the method comprising:

[0006] Obtain the first picture taken by the drone from the first shooting perspective;

[0007] If the landing mark on the vehicle is not recognized in the first image, the drone is controlled to shoot at a second shooting angle and fly in a target mode to obtain a search image for the landing mark;

[0008] According to the landing mark identified in the search screen, controlling the drone to be in a landing position;

[0009] Acquire environmental interference information of the UAV, and determine interference compensation control parameters corresponding to the UAV according to the environmental interference information;

[0010] Obtaining a predicted moving trajectory of a landing position indicated by the landing mark according to the driving information of the vehicle;

[0011] The UAV is controlled to land at the landing position according to the predicted movement trajectory, the interference compensation control parameters and the landing posture.

[0012] In one of the embodiments, the first shooting angle includes a first shooting angle of an image acquisition device in the drone;

[0013] The step of obtaining a first picture taken by the drone at a first shooting angle includes:

[0014] Acquire the current longitude and latitude information of the vehicle, and control the drone to move to a position corresponding to the current longitude and latitude information;

[0015] According to the position corresponding to the current longitude and latitude information, the image acquisition device is controlled to capture the first picture at the first shooting angle.

[0016] In one of the embodiments, the second shooting angle includes a second shooting angle of an image acquisition device in the drone;

[0017] If the landing mark on the vehicle is not recognized in the first picture, the drone is controlled to shoot at a second shooting angle and fly in a target mode to obtain a search picture for the landing mark, including:

[0018] If the landing mark is not recognized in the first picture, determining that the target mode is a yaw mode;

[0019] Controlling the image acquisition device to fly at the second shooting angle and the UAV to fly in the yaw mode;

[0020] Acquire multiple frames of images taken by the image acquisition device when the UAV flies in the yaw mode;

[0021] The search screen is obtained according to the multiple frames of images.

[0022] In one embodiment, controlling the drone to be in a landing position according to the landing mark identified in the search screen includes:

[0023] Restoring the second shooting angle of view to the first shooting angle of view;

[0024] According to the position of the landing mark in the search screen, the drone is controlled to move to the landing posture; the landing posture makes the center of the landing mark located at the center of the screen captured by the first shooting angle.

[0025] In one embodiment, the vehicle or the drone is provided with a wind speed and direction sensor; the environmental interference information includes wind speed information and wind direction information;

[0026] The obtaining of environmental interference information of the UAV and determining interference compensation control parameters corresponding to the UAV according to the environmental interference information includes:

[0027] Obtaining the nose direction of the UAV, and the wind speed information and wind direction information collected by the wind speed and wind direction sensor;

[0028] According to the nose direction, the wind speed information and the wind direction information, a compensation pitch amount and a compensation roll amount corresponding to the UAV are determined as the interference compensation control parameters.

[0029] In one embodiment, controlling the drone to land at the landing position according to the predicted movement trajectory, the interference compensation control parameter and the landing posture includes:

[0030] Determining predicted positions of the landing mark corresponding to a plurality of future time points according to the predicted movement trajectory;

[0031] Determining a predicted target position corresponding to a future target time point from the plurality of predicted target positions;

[0032] Determining flight control parameters corresponding to the UAV according to the identified predicted target position and the interference compensation control parameters;

[0033] According to the flight control parameters, the UAV is controlled to land at the landing position located at the identified predicted target position at the future target time point based on the landing posture.

[0034] In a second aspect, the present application provides a drone landing device, the device comprising:

[0035] An acquisition module is used to acquire a first picture taken by the drone at a first shooting angle;

[0036] A search module, configured to control the drone to shoot at a second shooting angle and fly in a target mode to obtain a search image for the landing mark if the landing mark on the vehicle is not recognized in the first image;

[0037] A control module, used for controlling the UAV to be in a landing position according to the landing mark identified in the search screen;

[0038] A compensation module, used to obtain environmental interference information of the UAV, and determine interference compensation control parameters corresponding to the UAV according to the environmental interference information;

[0039] A prediction module, used to obtain a predicted moving trajectory of the landing position indicated by the landing mark according to the driving information of the vehicle;

[0040] A landing module is used to control the UAV to land at the landing position according to the predicted movement trajectory, the interference compensation control parameters and the landing posture.

[0041] In a third aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0043] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0044] The above-mentioned drone landing method, device, computer equipment, computer-readable storage medium and computer program product, when the landing mark is not recognized in the first picture taken by the drone at the first shooting angle, the drone is controlled to fly at the second shooting angle and in the target mode to obtain the search picture of the landing mark, the drone is controlled to be in the landing posture according to the landing mark recognized in the search picture, the moving trajectory of the landing position is determined according to the vehicle driving information, and the drone is controlled to land at the landing position based on the predicted moving trajectory, the interference compensation control parameters corresponding to the environmental interference information of the drone and the landing posture. Compared with the traditional direct landing after identifying the landing target, this solution searches for the unrecognized landing mark by combining multiple shooting angles, predicts the trajectory of the landing position, and outputs the interference compensation control parameters for the environmental interference information, and controls the drone to land at the moving landing position in combination with the predicted trajectory of the landing position, the compensation control parameters and the landing posture, thereby improving the accuracy of the drone landing. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0046] FIG1 is a diagram showing an application environment of a method for landing a drone in one embodiment;

[0047] FIG2 is a schematic diagram of a process flow of a method for landing a drone in one embodiment;

[0048] FIG3 is a schematic diagram of a search step in one embodiment;

[0049] FIG4 is a schematic diagram of a search step in another embodiment;

[0050] FIG5 is a schematic diagram of wind force and wind speed in one embodiment;

[0051] FIG6 is a schematic diagram of a prediction step in one embodiment;

[0052] FIG7 is a block diagram of a UAV landing device according to an embodiment;

[0053] FIG. 8 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] The drone landing method provided in the embodiment of the present application can be applied to the application environment as shown in Figure 1. Among them, the vehicle-side controller communicates with the drone through the network. The vehicle-side controller can be set in the vehicle, and a vehicle-mounted landing platform, also called a landing position, can also be set in the vehicle, and a drone remote control can also be set. The vehicle-mounted landing platform is provided with a visual mark, also called a landing mark, and a wind direction and speed sensor can also be provided. The vehicle-side controller can control the drone through the drone remote control, for example, control the image acquisition device provided in the drone, use multiple shooting angles to identify the landing mark on the vehicle-mounted landing platform, and after identification, land on the vehicle-mounted landing platform on the moving vehicle according to the corresponding landing posture.

[0056] In an exemplary embodiment, as shown in FIG2 , a method for landing a drone is provided, which is described by taking the method applied to the vehicle-side controller in FIG1 as an example, and includes the following steps S202 to S208. Among them:

[0057] Step S202: obtaining a first picture taken by the drone at a first shooting angle.

[0058] Among them, the drone can be an unmanned aircraft. Currently, the drone needs to take off from a moving vehicle to perform a mission, and land on a moving vehicle after performing the mission to achieve recovery. The vehicle in this method is provided with a vehicle-mounted landing platform as a landing position, so it is necessary to control the drone to land accurately at the landing position on the vehicle. Among them, a landing mark is provided on the landing position. Since the control of the drone landing in an outdoor environment will be interfered by the reflection of sunlight, the landing mark cannot be identified under the interference of strong sunlight reflection. Therefore, the above-mentioned landing mark can be composed of a velvet material with ultra-low reflectivity, and its reflectivity is less than 1%, which can greatly reduce the probability that the landing mark cannot be identified due to sunlight reflection.

[0059] After completing the mission, the drone can land on a landing position such as a landing platform on a vehicle when the corresponding conditions are met. For example, when the mission completion information is recognized, or when a landing command is received from the vehicle-side controller through the drone remote control, it is determined that the conditions are met. When landing, the vehicle-side controller can control the drone to fly near the vehicle, and control the drone to shoot a first picture near the vehicle at a first shooting angle. Among them, an image acquisition device is provided on the drone, and the image acquisition device can have multiple shooting angles, and different shooting angles correspond to different shooting angles of the image acquisition device. The vehicle-side controller can obtain the first picture taken by the drone at the above-mentioned first shooting angle, so that the vehicle-side controller can determine whether it is necessary to acquire the picture at the second shooting angle based on whether the above-mentioned first picture can recognize the landing mark in the above-mentioned landing position.

[0060] Step S204, if the landing mark on the vehicle is not recognized in the first image, the drone is controlled to shoot at a second shooting angle and fly in a target mode to obtain a search image for the landing mark.

[0061] Among them, the above-mentioned first picture can be a picture taken by the drone at a first shooting angle. The first picture can be taken near the location of the vehicle. The vehicle-side controller can detect whether the landing mark on the vehicle can be identified in the first picture. If the landing mark is identified, the vehicle-side controller can control the drone to perform the landing process at the first shooting angle. If the landing mark on the vehicle is not identified in the first picture, in order to expand the recognition range, the vehicle-side controller can control the drone to shoot at a second shooting angle and fly in target mode to obtain a search picture for the landing mark. Among them, the second shooting angle and the first shooting angle can be two shooting angles with different shooting angles, and the target mode can be a yaw mode. In the yaw mode, the drone can rotate the fuselage with a certain point as the origin, and continue to shoot during the process, so as to obtain a search picture for the landing mark. Among them, the search picture can be a picture with a larger range than the first picture, and the vehicle-side controller can identify the landing mark based on the above search picture.

[0062] Step S206, controlling the UAV to be in a landing position according to the landing mark identified in the search screen.

[0063] Among them, the vehicle-side controller can identify the landing mark based on the search screen obtained by the drone flying in the second shooting angle and target mode. After the vehicle-side controller identifies the landing mark in the search screen with a larger screen range, it can control the drone to be in a landing posture. Among them, the landing posture represents a posture that allows the drone to execute the landing process. For example, the drone can make the image acquisition device set directly below the drone in the first shooting angle, and place the center of the landing mark in the center of the picture taken by the drone as the landing posture of the drone. At this time, the drone can realize the process of landing on the vehicle-mounted landing platform on the vehicle based on the above landing posture.

[0064] Step S208: Acquire environmental interference information of the UAV, and determine interference compensation control parameters corresponding to the UAV according to the environmental interference information.

[0065] Among them, the above-mentioned drone will encounter environmental interference information during the flight process. The environmental interference information can be the interference caused by external factors other than vehicles and drones to the flight of the drone during the flight of the drone. For example, it can include but is not limited to interference from ambient wind and interference from ambient brightness changes. In the outdoor natural environment, the direction and wind speed of the ambient wind are changing in real time. When the drone is affected by the ambient wind during the landing process, there will be an error between the actual flight distance executed by the drone and the flight speed corresponding to the established flight instructions given by the system, resulting in the final flight distance being inconsistent with the expected one, which in turn affects the final landing point and the target point. Therefore, it is necessary to determine the interference compensation control parameters in combination with the environmental interference information.

[0066] The above-mentioned vehicle or drone may be provided with corresponding sensors for detecting environmental interference information in the environment, so that the vehicle-side controller determines the interference compensation control parameters corresponding to the drone based on the detected environmental interference information. Among them, the interference compensation control parameters represent control parameters that enable the drone to offset the impact of environmental interference information on the normal flight and landing of the drone. For example, for environmental interference information of wind speed and direction, the vehicle-side controller can determine the flight control parameters that can offset the impact of wind speed and direction as interference compensation control parameters; for environmental brightness interference, the vehicle-side controller can control the drone to turn on the drone's lighting equipment and point it to the landing position on the vehicle when it detects that the ambient brightness is too low, as interference compensation control parameters.

[0067] Step S210: obtaining a predicted moving trajectory of the landing position indicated by the landing mark according to the driving information of the vehicle.

[0068] Among them, the above-mentioned vehicle can be a moving vehicle, that is, the landing mark on the vehicle can move with the movement of the vehicle. The vehicle-side controller can control the drone to land at the landing position on the moving vehicle. Among them, in order to make the drone land accurately, the vehicle-side controller can predict the landing position indicated by the landing mark. The vehicle-side controller can obtain the driving information of the vehicle, including but not limited to the speed and driving direction, so that the vehicle-side controller can obtain the predicted moving trajectory of the landing position indicated by the landing mark based on the driving information of the vehicle. Among them, the predicted moving trajectory of the landing position can be consistent with the driving direction and speed of the vehicle. The vehicle-side controller needs to control the drone to maintain the relative posture to the landing mark during the landing process, such as maintaining the relatively static posture between the drone and the landing mark, and then controlling the drone to land at the landing position of the above-mentioned moving vehicle at the corresponding speed and direction.

[0069] Step S212: Control the UAV to land at the landing position according to the predicted movement trajectory, the interference compensation control parameters and the landing posture.

[0070] Among them, the predicted moving trajectory can be the predicted moving trajectory of the landing position indicated by the landing mark on the above-mentioned vehicle, and the above-mentioned vehicle can be a moving vehicle. The vehicle-side controller can control the drone to land at the above-mentioned landing position according to the above-mentioned predicted moving trajectory, interference compensation control parameters and landing posture. Among them, the vehicle-side controller can determine the point position of the landing position corresponding to the landing mark at various time points in the future period of time based on the predicted moving trajectory, so that the vehicle-side controller can control the drone to follow the landing mark with the above-mentioned landing posture based on the above-mentioned predicted moving trajectory, and control the drone to land at the above-mentioned landing posture at the corresponding time point to the landing position indicated by the above-mentioned landing mark, thereby realizing the landing of the drone on the moving vehicle.

[0071] Among them, the vehicle-side controller recognizes the landing target after visual recognition based on the picture taken by the drone, then converts the relative position and orientation, and then performs flight control to achieve landing. There is a delay in the process, so the size of the landing position on the above-mentioned vehicle can be the size that satisfies the delay error. Among them, the vehicle-side controller can calculate the accuracy error that may be caused by the delay based on the difference between the flight speed of the drone and the driving speed of the vehicle, as well as the time required for the final landing, and the platform size of the landing position on the above-mentioned vehicle can cover this error. For example, the time from the last moment of the drone and the landing target posture information obtained by the vehicle-side controller to the final landing is 2s. At this time, the difference between the flight speed of the drone and the driving speed of the vehicle is 0.1m / s, plus the signal delay of 0.210~0.237s, the maximum is 2.337s, then the landing platform on the vehicle for the drone to land only needs to provide a landing platform space greater than ±0.2337m.

[0072] In the above-mentioned drone landing method, when the landing mark is not recognized in the first picture taken by the drone at the first shooting angle, the drone is made to fly at the second shooting angle and in the target mode to obtain the search picture of the landing mark, the drone is controlled to be in the landing posture according to the landing mark recognized in the search picture, the moving trajectory of the landing position is determined according to the vehicle driving information, and the drone is controlled to land at the landing position based on the predicted moving trajectory, the interference compensation control parameters corresponding to the environmental interference information of the drone, and the landing posture. Compared with the traditional direct landing after identifying the landing target, this solution searches for unrecognized landing marks by combining multiple shooting angles, predicts the trajectory of the landing position, and outputs the interference compensation control parameters for the environmental interference information, and controls the drone to land at the moving landing position in combination with the predicted trajectory of the landing position, the compensated control parameters and the landing posture, thereby improving the accuracy of the drone landing.

[0073] In one embodiment, obtaining a first picture taken by a drone at a first shooting angle includes: obtaining current longitude and latitude information of the vehicle, and controlling the drone to move to a position corresponding to the current longitude and latitude information; and according to the position corresponding to the current longitude and latitude information, controlling the image acquisition device to shoot the first picture at the first shooting angle.

[0074] In this embodiment, the first shooting angle of view of the above-mentioned drone includes the first shooting angle of the image acquisition device in the above-mentioned drone. The above-mentioned image acquisition device can be arranged below the drone, and the shooting angle can be adjusted. When the vehicle-side controller controls the drone to land, it can first control the drone to shoot the first picture based on the first shooting angle of view. Among them, the landing position can be a vehicle-mounted landing platform on the vehicle, then the vehicle-side controller can obtain the current longitude and latitude information of the vehicle, and control the above-mentioned drone to move to the position corresponding to the current longitude and latitude information through the drone remote control, that is, the drone flies above the vicinity of the vehicle. The vehicle-side controller can control the image acquisition device to shoot the first picture at the first shooting angle according to the position corresponding to the above-mentioned current longitude and latitude information.

[0075] Specifically, the vehicle-side controller can locate the longitude and latitude coordinates of both the vehicle and the drone based on the satellite positioning system, and then the vehicle-side controller can obtain the current longitude and latitude information corresponding to the vehicle. The vehicle-side controller can control the drone to fly to the vicinity of the vehicle through the drone remote controller based on the above current longitude and latitude information. Among them, the above first shooting angle can be -90°, and the vehicle-side controller can control the image acquisition device of the drone to shoot images at a vertical downward angle, that is, the above -90°, and continuously shoot images at a preset frequency to obtain the above first picture. The drone transmits the first picture taken to the drone remote controller in the vehicle in real time, and the vehicle-side controller obtains the above first picture in real time through the drone remote controller. The vehicle-side controller can identify the landing mark in the first image through a preset visual algorithm. When the vehicle-side controller identifies the landing mark in the first image, the relative posture of the current drone and the landing mark can be determined, so that the vehicle-side controller controls the drone to fly to the landing instruction when the landing mark is in the center of the first picture based on the preset flight control algorithm and the above relative posture.

[0076] Among them, the instructions for controlling the flight of the drone include but are not limited to up and down instructions, left and right instructions, front and back instructions, and rotation direction instructions. Among them, the up and down instructions control the lifting and lowering action of the drone, the left and right instructions control the rolling action of the drone, the front and back instructions control the pitch action of the drone, and the rotation direction instructions control the drone to perform yaw flight, that is, to rotate with a certain point as the origin. During the process of landing mark recognition and landing based on the first shooting angle, the vehicle-side controller can always move with the recognized landing mark when the landing mark is recognized to ensure that it is in the center of the first picture.

[0077] Through this embodiment, the vehicle-side controller can use the first picture captured by the first shooting angle to identify the landing mark, so that the vehicle-side controller can control the drone to land at the first shooting angle based on the identified landing mark, thereby improving the accuracy of the drone landing.

[0078] In one embodiment, if the landing mark on the vehicle is not recognized in the above-mentioned first picture, the above-mentioned UAV is controlled to shoot at the second shooting angle and fly in the target mode to obtain a search picture for the above-mentioned landing mark, including: if the above-mentioned landing mark is not recognized in the above-mentioned first picture, determining that the above-mentioned target mode is the yaw mode; controlling the above-mentioned image acquisition device to fly at the above-mentioned second shooting angle and the above-mentioned UAV in the above-mentioned yaw mode; obtaining multiple frames of images taken by the above-mentioned image acquisition device when the above-mentioned UAV flies in the above-mentioned yaw mode; and obtaining the above-mentioned search picture based on the above-mentioned multiple frames of images.

[0079] In this embodiment, the second shooting angle includes the second shooting angle of the image acquisition device in the drone. The vehicle-side controller can recognize the landing mark at the first shooting angle by default. When the vehicle-side controller does not recognize the landing mark in the first screen, the landing mark may not be fully recognized or the target is lost. Then the vehicle-side controller can control the drone to fly at the second shooting angle and in the target mode, thereby obtaining a search screen for the landing mark. Among them, if the vehicle-side controller does not recognize the landing mark in the first screen, the vehicle-side controller can determine that the target mode is the yaw mode. In the yaw mode, the drone rotates with a certain point as the origin. The vehicle-side controller can control the image acquisition device in the drone to adjust to the second shooting angle and control the drone to fly in the yaw mode, so that the drone can collect images in real time during the yaw mode flight. The vehicle-side controller can obtain multiple frames of images captured by the image acquisition device when the drone flies in the yaw mode, so that the vehicle-side controller can obtain the above-mentioned search screen based on the multiple frames of images. The search screen can be a screen with a larger screen range relative to the first screen, and then the vehicle-side controller can identify the landing mark in the search screen. For example, the vehicle-side controller identifies the landing mark in the search screen based on the above-mentioned preset visual algorithm. After the vehicle-side controller recognizes the landing mark in the above search screen, it can control the drone to land at the first shooting perspective.

[0080] In one embodiment, based on the landing mark identified in the search screen, the drone is controlled to be in a landing position, including: restoring the second shooting angle to the first shooting angle; based on the position of the landing mark in the search screen, controlling the drone to move to the landing position; the landing position makes the center of the landing mark located at the center of the picture taken by the first shooting angle.

[0081] In this embodiment, the vehicle-side controller can use the search screen to identify the landing mark in a larger range. When the vehicle-side controller uses the search screen to identify the landing mark, the vehicle-side controller can determine the position of the landing mark in the search screen and restore the second shooting angle to the first shooting angle. Based on the position of the landing mark in the search screen, the vehicle-side controller can control the drone to move to a landing posture. Among them, when the drone is in the landing posture, the center of the above-mentioned landing mark is located at the center of the picture taken by the first shooting angle. That is, after the vehicle-side controller determines the position of the landing mark in the search screen, it can fly to the position of the identified landing mark so that the landing mark is located in the center of the picture taken by the first shooting angle, so that the drone can land.

[0082] Specifically, as shown in FIG3 , FIG3 is a schematic diagram of a search step in an embodiment. The above-mentioned first shooting angle can be a shooting angle of -90°, the above-mentioned second shooting angle can be a shooting angle of -70°, and the image acquisition device of the drone can be a camera. The vehicle-side controller firstly acquires the first picture at the first shooting angle, and identifies the landing mark therein. If the first picture acquired by the vehicle-side controller through the first shooting angle, that is, the vertically downward image transmission picture of the image acquisition device cannot contain the landing mark, and the recognition of the landing mark cannot be completed, then the vehicle-side controller can adjust the angle of the image acquisition device of the drone to -70°, and control the drone to yaw and fly for one week to obtain the above-mentioned search picture, and the process searches for and identifies the landing mark. After identifying the landing mark, the vehicle-side controller controls the drone to fly to a position where the landing mark can be located at the center of the picture acquired by the image acquisition device, and then adjusts the camera unit to -90°, that is, the first shooting angle, and performs the recognition and landing of the landing mark according to the landing steps of the above-mentioned first shooting angle.

[0083] Therefore, when the satellite system deviation causes the image transmission image of the drone's image acquisition device to be unable to include the landing mark, the vehicle-side controller adjusts the angle of the drone camera to -70°, of which the angle when vertical to the ground is -90°, and controls the drone to rotate one circle, that is, yaw flight one circle, and searches for the identification mark in the process, which will increase the identification area by about 460%. For example, if the first picture taken at the first shooting angle is 7*12.5m, the size of the search picture taken based on the second shooting angle can be π*12.5㎡.

[0084] In some embodiments, as shown in FIG. 4, FIG. 4 is a schematic diagram of the search step in another embodiment. The vehicle-side controller can also increase the screen area of ​​the drone's camera at the first shooting angle, and at the same time increase the size of the landing mark to achieve landing recognition. For example, the landing mark can be expanded from the original 40cm*40cm to 80cm*80cm. The vehicle-side controller can also increase the height of the drone's recognition of the landing mark when the first screen cannot recognize the landing mark, for example, from the original 20m to 40m, so that the size of the first screen is expanded from 7*12.5m to the size of the search screen in this embodiment, that is, 14*25m, and the vehicle-side controller can recognize the landing mark from a larger range. In some other embodiments, the vehicle-side controller can also increase the identifiable area of ​​the image acquisition device to search for the landing mark, such as by reducing the focal length or using a wide-angle lens to achieve the expansion of the identifiable area.

[0085] Through the above-mentioned embodiments, when the vehicle-side controller cannot identify the landing mark based on the first shooting angle, the vehicle-side controller can use the second shooting angle to shoot the search screen to search for the landing mark. After identifying the landing mark in the search screen, the vehicle-side controller can use the first shooting angle to make the landing mark in the center of the picture taken by the drone, and control the drone to land, thereby improving the accuracy of the drone landing.

[0086] In one embodiment, environmental interference information of the UAV is obtained, and interference compensation control parameters corresponding to the UAV are determined based on the environmental interference information, including: obtaining the nose direction of the UAV, and wind speed information and wind direction information collected by the wind speed and wind direction sensor; and determining the compensation pitch amount and compensation roll amount corresponding to the UAV as the interference compensation control parameters based on the nose direction, the wind speed information and the wind direction information.

[0087] In this embodiment, the vehicle-side controller can determine the compensation control parameters for the drone in combination with the environmental interference information, so that the drone can offset the environmental interference factors and land. Among them, the above-mentioned vehicle or drone can be provided with a wind speed and direction sensor for collecting environmental interference information near the drone. Among them, the environmental interference information may include wind speed information and wind direction information. The wind speed information and wind direction information may be the wind speed and wind direction information near the drone, which will cause the drone's fuselage to deviate and thus affect the landing process. Therefore, the vehicle-side controller can use environmental interference information such as wind speed and wind direction to determine the interference compensation control parameters corresponding to the drone.

[0088] For example, the vehicle-side controller can obtain the nose direction of the UAV, and the nose direction can be sent by the UAV to the vehicle-side controller. The vehicle-side controller can also obtain the wind speed information and wind direction information collected by the above-mentioned wind speed and wind direction sensor, so that the vehicle-side controller can determine the corresponding compensation pitch and compensation roll of the UAV according to the above-mentioned nose direction, wind speed information and wind direction information as interference compensation control parameters. Among them, the compensation pitch amount represents the pitch amount that should be output when resisting the interference of wind speed information and wind direction information based on the above-mentioned nose direction, and the compensation roll amount represents the roll amount that should be output when resisting the interference of wind speed information and wind direction information based on the above-mentioned nose direction. That is, the interference compensation control parameter can be a flight control parameter that enables the UAV to offset the impact of environmental interference information on the flight of the UAV.

[0089] Specifically, as shown in Figure 5, Figure 5 is a schematic diagram of wind force and wind speed in one embodiment. The wind speed and wind direction sensor installed on the vehicle or the drone transmits real-time wind speed and wind direction data to the vehicle-side controller at a preset frequency. The vehicle-side controller calculates the pitch and roll amounts to be compensated based on the transmitted wind speed information and wind direction information, combined with the nose direction of the drone, and then obtains the compensation amount corresponding to the corresponding wind speed and wind direction, that is, the above-mentioned interference compensation control parameters, and outputs the corresponding flight control instructions, so that the drone performs speed compensation to offset the interference of the ambient wind on the flight speed of the drone.

[0090] Through this embodiment, the vehicle-side controller can use wind speed and direction sensors to collect wind speed and direction information that affects the flight of the drone, and then determine corresponding interference compensation control parameters based on the wind speed and direction information, and control the flight of the drone based on the interference compensation control parameters to offset the environmental interference information that affects the landing process of the drone, thereby improving the accuracy of the drone landing.

[0091] In one embodiment, according to the predicted movement trajectory, the interference compensation control parameters and the landing posture, the UAV is controlled to land at the landing position, including: determining the predicted identification position corresponding to the landing identification at multiple future time points according to the predicted movement trajectory; determining the predicted identification target position corresponding to the future target time point from the multiple predicted identification positions; determining the flight control parameters corresponding to the UAV according to the predicted identification target position and the interference compensation control parameters; and controlling the UAV to land at the landing position located at the predicted identification target position at the future target time point based on the landing posture according to the flight control parameters.

[0092] In this embodiment, the above-mentioned vehicle can be a moving vehicle, and the vehicle-side controller can control the landing position of the drone on the moving vehicle. Then the vehicle-side controller can predict the driving trajectory of the vehicle, realize the prediction of the moving trajectory of the landing position, and make the drone land accurately at the landing position. During the driving process of the vehicle, the vehicle-side controller can obtain the driving data of the vehicle, such as obtaining the driving direction and speed of the vehicle in the future, to predict the point information of the landing position of the vehicle in the same period of time in the future, so that the vehicle-side controller can set the target point of the drone landing at the above-mentioned predicted target point to offset the position difference of the landing position movement in the final landing time period. Thereby, the drone can be controlled to land accurately at the moving target point.

[0093] Therefore, the vehicle-side controller can obtain the driving data of the vehicle and obtain the predicted moving trajectory corresponding to the vehicle. The vehicle-side controller can determine the predicted identification position corresponding to the landing mark at multiple future time points based on the above-mentioned predicted moving trajectory. Thus, the vehicle-side controller can determine the predicted identification target position corresponding to the future target time point from multiple predicted identification positions, wherein the future target time point can be one of the above-mentioned multiple future time points, and the predicted identification target position can be the predicted identification position corresponding to the future target time point. The vehicle-side controller can determine the time and position suitable for the drone to land at the landing position indicated by the above-mentioned landing mark based on the current driving speed and direction of the vehicle, as well as the current flight speed and direction of the drone, that is, obtain the above-mentioned future target time point and the corresponding predicted identification target position.

[0094] The vehicle-side controller can determine the corresponding flight control parameters of the UAV based on the above-mentioned predicted target position and interference compensation control parameters. For example, based on the predicted target position, the vehicle-side controller determines the original flight control parameters required for the UAV to fly to the predicted target position at the above-mentioned future target time point. The vehicle-side controller then combines the above-mentioned original flight control parameters with the interference compensation control parameters to determine the flight control parameters that can offset the environmental interference information and fly to the predicted target position. Thus, the vehicle-side controller can control the UAV to land at the landing position located at the predicted target position at the future target time point based on the landing posture according to the above-mentioned flight control parameters, so as to land the UAV on a moving vehicle. Among them, the landing posture can be a posture taken from the first shooting angle of the UAV, and the center of the landing mark is at the center of the first picture taken from the first angle. The vehicle-side controller can control the UAV to maintain the above-mentioned landing posture during the flight to land.

[0095] Specifically, as shown in Figure 6, Figure 6 is a schematic diagram of the prediction step in one embodiment. Starting from the current position of the target point of the landing mark, the vehicle-side controller predicts the position of the landing target point at each future time point to obtain the predicted landing target point trajectory, that is, the above-mentioned predicted moving trajectory. During the landing process of the drone, the vehicle-side controller infers the coordinate information of the landing platform at a certain time in the future through the information of the vehicle speed and driving direction, that is, the predicted target position of the mark corresponding to the machine at the above-mentioned future target time point, so that the vehicle-side controller calculates the required flight speed and direction of the drone based on this information, that is, the above-mentioned flight control parameters, so that the drone can fly to the location of the landing platform at the above-mentioned future target time point based on the above-mentioned flight control parameters at a given time point to complete the landing.

[0096] Through this embodiment, the vehicle-side controller can combine the vehicle's predicted moving trajectory to determine the future target time point for the UAV's final landing and identify the predicted target position, and then control the UAV to land at the corresponding time point at the location of the vehicle-mounted landing platform at that time point, thereby improving the accuracy of the UAV's landing.

[0097] In one embodiment, a method for landing a drone is also provided. In this embodiment, the vehicle-side controller controls the drone to fly to the vicinity of the vehicle through the drone remote controller based on the latitude and longitude coordinates of the vehicle and the drone given by the satellite positioning system.

[0098] The vehicle-side controller can control the drone's camera unit to continuously capture images at a fixed frequency at a vertical downward angle of -90°. The drone remote controller transmits the image to the vehicle-side controller in real time, identifies the landing mark through the visual algorithm in the vehicle controller, and calculates the relative position of the current drone and the landing mark. The flight control algorithm of the vehicle control system calculates the command to control the drone to fly to a position where the landing mark is in the center of the picture taken by the drone through the relative position. Among them, the commands to control the flight of the drone include: up and down (lift and drop), left and right (roll), front and back (pitch), rotation direction (yaw), etc.

[0099] If the vertically downward image transmission image of the drone's camera unit cannot include the landing mark and cannot complete the recognition of the landing mark, the vehicle-side controller can adjust the angle of the drone camera to -70° and control the drone to fly a circle, searching for the identification mark in the process. After recognizing the landing mark, the drone is controlled to fly to the center of the picture, and then the camera unit is adjusted to -90°, and the landing control is performed through the landing method of the above-mentioned first shooting angle. The wind speed and direction sensor installed on the vehicle or drone can transmit real-time wind speed and wind direction data to the vehicle-side controller at a fixed frequency. The vehicle-side controller calculates the pitch and roll amounts that need to be compensated based on the above data and the direction of the aircraft's nose, and outputs the flight control command to make the drone make speed compensation to offset the interference of the ambient wind on the drone's flight speed.

[0100] During the landing process, the vehicle-side controller can infer the coordinate information of the landing platform at a certain time in the future through the information of vehicle speed and driving direction. The vehicle-side controller calculates the required flight speed and direction of the drone based on this information, so that the drone can fly to the location of the landing platform at the above-mentioned future time at a predetermined time point and complete the landing.

[0101] Through the above embodiments, the vehicle-side controller can search for unrecognized landing marks by combining multiple shooting angles, predict the trajectory of the landing position, and output interference compensation control parameters for environmental interference information. Combined with the predicted trajectory of the landing position, the compensated control parameters and the landing posture, the drone is controlled to land at the moving landing position, thereby improving the accuracy of the drone landing.

[0102] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0103] Based on the same inventive concept, the embodiment of the present application also provides a drone landing device for implementing the drone landing method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more drone landing device embodiments provided below can refer to the limitations of the drone landing method above, and will not be repeated here.

[0104] In an exemplary embodiment, as shown in FIG. 7 , a drone landing device is provided, including: an acquisition module 500, a search module 502, a control module 504, a compensation module 506, a prediction module 508 and a landing module 510, wherein:

[0105] The acquisition module 500 is used to acquire a first picture taken by the drone at a first shooting angle.

[0106] The search module 502 is used to control the drone to shoot at a second shooting angle and fly in a target mode to obtain a search image for the landing mark if the landing mark on the vehicle is not recognized in the first image.

[0107] The control module 504 is used to control the UAV to be in a landing position according to the landing mark identified in the search screen.

[0108] The compensation module 506 is used to obtain the environmental interference information of the above-mentioned UAV, and determine the interference compensation control parameters corresponding to the above-mentioned UAV according to the above-mentioned environmental interference information.

[0109] The prediction module 508 is used to obtain a predicted moving trajectory of the landing position indicated by the landing mark according to the driving information of the vehicle.

[0110] The landing module 510 is used to control the UAV to land at the landing position according to the predicted movement trajectory, the interference compensation control parameters and the landing posture.

[0111] In one embodiment, the acquisition module 500 is used to acquire the current longitude and latitude information of the vehicle, and control the drone to move to a position corresponding to the current longitude and latitude information; and according to the position corresponding to the current longitude and latitude information, control the image acquisition device to capture the first picture at the first shooting angle.

[0112] In one embodiment, the search module 502 is used to determine that the target mode is the yaw mode if the landing mark is not recognized in the first screen; control the image acquisition device to fly at the second shooting angle and the UAV to fly in the yaw mode; obtain multiple frames of images taken by the image acquisition device when the UAV flies in the yaw mode; and obtain the search screen based on the multiple frames of images.

[0113] In one embodiment, the control module 504 is used to restore the second shooting angle to the first shooting angle; according to the position of the landing mark in the search screen, control the drone to move to the landing posture; the landing posture makes the center of the landing mark located at the center of the screen captured by the first shooting angle.

[0114] In one embodiment, the compensation module 506 is used to obtain the nose direction of the UAV and the wind speed information and wind direction information collected by the wind speed and wind direction sensor; based on the nose direction, the wind speed information and the wind direction information, the compensation pitch amount and the compensation roll amount corresponding to the UAV are determined as the interference compensation control parameters.

[0115] In one embodiment, the landing module 510 is used to determine the predicted position of the landing mark corresponding to multiple future time points according to the predicted moving trajectory; determine the predicted target position of the landing mark corresponding to the future target time point from the multiple predicted positions; determine the flight control parameters corresponding to the UAV according to the predicted target position and the interference compensation control parameters; and control the UAV to land at the landing position located at the predicted target position of the landing mark based on the landing posture at the future target time point according to the flight control parameters.

[0116] Each module in the above-mentioned drone landing device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0117] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG8. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for landing a drone is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0118] Those skilled in the art will understand that the structure shown in FIG. 8 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0119] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above-mentioned drone landing method when executing the computer program.

[0120] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned drone landing method is implemented.

[0121] In one embodiment, a computer program product is provided, comprising a computer program, which implements the above-mentioned drone landing method when executed by a processor.

[0122] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0123] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0124] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0125] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for landing a drone, characterized in that: The method comprises: Obtain the first picture taken by the drone from the first shooting perspective; If the landing mark on the vehicle is not recognized in the first image, the drone is controlled to shoot at a second shooting angle and fly in a target mode to obtain a search image for the landing mark; According to the landing mark identified in the search screen, controlling the drone to be in a landing position; Acquire environmental interference information of the UAV, and determine interference compensation control parameters corresponding to the UAV according to the environmental interference information; Obtaining a predicted moving trajectory of a landing position indicated by the landing mark according to the driving information of the vehicle; The UAV is controlled to land at the landing position according to the predicted movement trajectory, the interference compensation control parameters and the landing posture.

2. The method according to claim 1, characterized in that The first shooting angle of view includes a first shooting angle of an image acquisition device in the drone; The step of obtaining a first picture taken by the drone at a first shooting angle includes: Acquire the current longitude and latitude information of the vehicle, and control the drone to move to a position corresponding to the current longitude and latitude information; According to the position corresponding to the current longitude and latitude information, the image acquisition device is controlled to capture the first picture at the first shooting angle.

3. The method according to claim 2, characterized in that The second shooting angle of view includes a second shooting angle of the image acquisition device in the drone; If the landing mark on the vehicle is not recognized in the first picture, the drone is controlled to shoot at a second shooting angle and fly in a target mode to obtain a search picture for the landing mark, including: If the landing mark is not recognized in the first picture, determining that the target mode is a yaw mode; Controlling the image acquisition device to fly at the second shooting angle and the UAV to fly in the yaw mode; Acquire multiple frames of images taken by the image acquisition device when the UAV flies in the yaw mode; The search screen is obtained according to the multiple frames of images.

4. The method according to claim 1, characterized in that The step of controlling the drone to be in a landing position according to the landing mark identified in the search screen includes: Restoring the second shooting angle of view to the first shooting angle of view; According to the position of the landing mark in the search screen, the drone is controlled to move to the landing posture; the landing posture makes the center of the landing mark located at the center of the screen captured by the first shooting angle.

5. The method according to claim 1, characterized in that The vehicle or the drone is provided with a wind speed and direction sensor; the environmental interference information includes wind speed information and wind direction information; The obtaining of environmental interference information of the UAV and determining interference compensation control parameters corresponding to the UAV according to the environmental interference information includes: Obtaining the nose direction of the UAV, and the wind speed information and wind direction information collected by the wind speed and wind direction sensor; According to the nose direction, the wind speed information and the wind direction information, a compensation pitch amount and a compensation roll amount corresponding to the UAV are determined as the interference compensation control parameters.

6. The method according to any one of claims 1 to 5, characterized in that: The controlling the UAV to land at the landing position according to the predicted movement trajectory, the interference compensation control parameter and the landing posture comprises: Determining predicted positions of the landing mark corresponding to a plurality of future time points according to the predicted movement trajectory; Determining a predicted target position corresponding to a future target time point from the plurality of predicted target positions; Determining flight control parameters corresponding to the UAV according to the identified predicted target position and the interference compensation control parameters; According to the flight control parameters, the UAV is controlled to land at the landing position located at the identified predicted target position at the future target time point based on the landing posture.

7. A UAV landing device, characterized in that: The device comprises: An acquisition module is used to acquire a first picture taken by the drone at a first shooting angle; A search module, configured to control the drone to shoot at a second shooting angle and fly in a target mode to obtain a search image for the landing mark if the landing mark on the vehicle is not recognized in the first image; A control module, used for controlling the UAV to be in a landing position according to the landing mark identified in the search screen; A compensation module, used to obtain environmental interference information of the UAV, and determine interference compensation control parameters corresponding to the UAV according to the environmental interference information; A prediction module, used to obtain a predicted moving trajectory of the landing position indicated by the landing mark according to the driving information of the vehicle; A landing module is used to control the UAV to land at the landing position according to the predicted movement trajectory, the interference compensation control parameters and the landing posture.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images