A method and apparatus for controlling landing of a drone

Abstract

The application provides a kind of unmanned aerial vehicle landing control method and device, it is related to unmanned aerial vehicle technical field.The unmanned aerial vehicle landing control method includes: the video stream information including nested positioning label and direction positioning label is input into neural network model, obtains the output value including first center point coordinate value, second center point coordinate value, third center point coordinate value, the width pixel value of nested positioning label and the height pixel value of nested positioning label, determines the angle between unmanned aerial vehicle and landing landmark and actual displacement difference by these output values, generates the control instruction of unmanned aerial vehicle according to the height of unmanned aerial vehicle, angle and actual displacement difference, controls unmanned aerial vehicle landing.The technical scheme of the application only needs to input video stream information into neural network model to obtain output value, the number of output value is less, the time for determining angle and actual displacement difference by less output value is shorter, so that the time for controlling unmanned aerial vehicle landing is shorter, and the efficiency of controlling unmanned aerial vehicle landing is improved.

Description

A method and apparatus for controlling the landing of unmanned aerial vehicles (UAVs) Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a UAV landing control method and apparatus. Background Technology

[0002] With the continuous development of unmanned aerial vehicle (UAV) systems, UAVs are being used more and more widely. UAVs are typically used in tasks requiring high-precision control, such as cargo delivery, search and rescue, and automatic recharging. In performing these tasks, UAVs need to land precisely in complex environments; only with a precise landing can a UAV effectively execute its mission.

[0003] Currently, to control the landing of a drone, it is necessary to obtain the intrinsic parameter matrix of the drone's onboard camera, solve the PnP problem using the IPPE algorithm based on the intrinsic parameter matrix, estimate the relative pose between the drone and the landmark using the solved PnP problem, and then adjust the drone's attitude and position.

[0004] However, solving the PnP problem using the IPPE algorithm based on the intrinsic parameter matrix involves transforming a two-dimensional image into a three-dimensional space using the intrinsic parameter matrix to determine the relative pose of the UAV and the landmark in the three-dimensional space. There are many parameters involved in determining the relative pose of the UAV and the landmark in the three-dimensional space, more than 10 parameters. Solving the PnP problem with many parameters makes it more complex and time-consuming. The time spent estimating the relative pose between the UAV and the landmark through the solved PnP problem is also long, resulting in a longer UAV landing control time and lower efficiency. Summary of the Invention

[0005] The purpose of this invention is to provide a method and apparatus for controlling the landing of unmanned aerial vehicles (UAVs) to solve the problem of low efficiency in UAV landing control.

[0006] To address the aforementioned technical problems, the embodiments of the present invention provide the following technical solutions:

[0007] The first aspect of this invention provides a method for controlling the landing of an unmanned aerial vehicle (UAV), the method comprising:

[0008] Acquire video stream information sent by the drone at a preset location. The video stream information includes information on preset landing landmarks, which include nested positioning tags and directional positioning tags.

[0009] The video frames corresponding to the video stream information are input into the neural network model to obtain the output values ​​of the neural network model. The output values ​​include the coordinates of the first center point of the nested positioning tag, the coordinates of the second center point of the orientation positioning tag, the coordinates of the third center point of the video frame, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag.

[0010] Based on the coordinates of the first center point, the second center point, the third center point, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag, determine the angle and actual displacement difference between the UAV and the landing landmark;

[0011] Based on the drone's altitude, included angle, and actual displacement difference, control commands for the drone are generated.

[0012] Control the drone to land according to the control commands.

[0013] In some modified embodiments of the first aspect of the present invention, the landing landmark is circular, and a nested positioning label with two H-shaped patterns is nested at the center point of the circle. A directional positioning label with rounded rectangular patterns is provided inside the circle and outside the two H-shaped patterns. The rounded rectangular patterns are directly above the outer H-shaped patterns. The center points of the outer H-shaped patterns coincide with those of the inner H-shaped patterns, and the outer H-shaped patterns and the inner H-shaped patterns are on the same plane.

[0014] In some modified embodiments of the first aspect of the present invention, determining the angle and actual displacement difference between the UAV and the landing landmark based on the coordinates of the first center point, the coordinates of the second center point, the coordinates of the third center point, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag includes:

[0015] Get the width and height of nested positioning tags;

[0016] The relative displacement difference between the drone and the landing landmark is determined using the following first formula:

[0017]

[0018] Where distance_pixel_x is the relative displacement difference between the UAV and the landing landmark in the x-axis direction, distance_pixel_y is the relative displacement difference between the UAV and the landing landmark in the y-axis direction, (x1,y1) is the coordinate value of the first center point, and (x2,y2) is the coordinate value of the third center point.

[0019] The actual displacement difference between the drone and the landing landmark is determined using the following second formula:

[0020]

[0021]

[0022] Where distance_x is the actual displacement difference between the drone and the landing landmark in the x-axis direction, distance_y is the actual displacement difference between the drone and the landing landmark in the y-axis direction, k is the number of centimeters per unit pixel, w is the width of the nested positioning tag, h is the height of the nested positioning tag, w_pixel is the width of the nested positioning tag in pixels, and h_pixel is the height of the nested positioning tag in pixels.

[0023] In some modified embodiments of the first aspect of the present invention, the actual displacement difference between the UAV and the landing landmark is determined according to the following second formula, including:

[0024] When the height of the drone is greater than the first preset value, in the second formula, w is the width of the nested positioning tag of the outer H-shaped pattern, h is the height of the nested positioning tag of the outer H-shaped pattern, w_pixel is the width of the nested positioning tag of the outer H-shaped pattern in pixels, and h_pixel is the height of the nested positioning tag of the outer H-shaped pattern in pixels.

[0025] When the drone's height is less than the first preset value, in the second formula, w is the width of the nested positioning tag of the inner H-shaped pattern, h is the height of the nested positioning tag of the inner H-shaped pattern, w_pixel is the width of the nested positioning tag of the inner H-shaped pattern in pixels, and h_pixel is the height of the nested positioning tag of the inner H-shaped pattern in pixels.

[0026] In some modified embodiments of the first aspect of the present invention, control commands for the UAV are generated based on the UAV's altitude, included angle, and actual displacement difference, including:

[0027] Determine whether the included angle is less than the preset included angle value and whether the actual displacement difference is less than the preset displacement difference value;

[0028] If so, a descent command will be generated when the drone's altitude is greater than the second preset value, and a landing command will be generated when the drone's altitude is less than the second preset value.

[0029] If not, the included angle and the actual displacement difference are input into the closed-loop control system, so that the PID algorithm of the closed-loop control system can output the linear velocity or angular velocity value of the UAV, and generate translation or rotation commands based on the linear velocity or angular velocity value.

[0030] In some modified embodiments of the first aspect of the present invention, the included angle and the actual displacement difference are input to the closed-loop control system, so that the PID algorithm of the closed-loop control system outputs the linear velocity value or angular velocity value of the UAV, and generates translation or rotation commands based on the linear velocity value or angular velocity value, including:

[0031] When the drone's altitude is greater than the second preset value, the included angle and the actual displacement difference are input into the closed-loop control system so that the first linear velocity value of the drone can be output through the PID algorithm of the closed-loop control system.

[0032] Generate the first translation command based on the first linear velocity value;

[0033] When the drone's altitude is greater than the first preset value and less than the second preset value, the included angle and the actual displacement difference are input to the closed-loop control system. The PID algorithm of the closed-loop control system outputs the drone's second linear velocity value and angular velocity value, where the first preset value is less than the second preset value.

[0034] Based on the second linear velocity value and angular velocity value, generate the second translation command and rotation command;

[0035] When the drone's altitude is less than the first preset value, the included angle and the actual displacement difference are input into the closed-loop control system, so that the third linear velocity value of the drone can be output through the PID algorithm of the closed-loop control system.

[0036] Based on the third linear velocity value, a third translation command is generated, and a landing command is generated when the current actual displacement difference is less than the preset displacement difference value.

[0037] In some modified embodiments of the first aspect of the present invention, controlling the drone to land according to control commands includes:

[0038] When generating the first translation command, control the UAV to translate the distance of the actual displacement difference at the first linear velocity value;

[0039] When generating the second translation and rotation commands, the drone is controlled to translate the actual displacement difference by a distance at a second linear velocity value, and during the translation process, the drone is controlled to rotate the included angle by an angular velocity value.

[0040] When generating the third translation command, control the drone to translate by the distance of the actual displacement difference at the third linear velocity value, and control the drone to land when the current actual displacement difference is less than the preset displacement difference value.

[0041] In some modified embodiments of the first aspect of the present invention, the neural network model is a combination of the YOLOv5s network and the GhostNet network.

[0042] A second aspect of the present invention provides a drone landing control device, the device comprising:

[0043] The acquisition module is used to acquire video stream information sent by the drone at a preset location. The video stream information includes information of a pre-set landing landmark, and the pre-set landing landmark includes nested positioning tags and the directional positioning tags.

[0044] The input module is used to input the video frames corresponding to the video stream information into the neural network model to obtain the output values ​​of the neural network model. The output values ​​include the coordinate values ​​of the first center point of the nested positioning tag, the coordinate values ​​of the second center point of the orientation positioning tag, the coordinate values ​​of the third center point of the video frame, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag.

[0045] The determination module is used to determine the angle and actual displacement difference between the UAV and the landing landmark based on the coordinates of the first center point, the second center point, the third center point, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag.

[0046] The generation module is used to generate control commands for the UAV based on its altitude, included angle, and actual displacement difference.

[0047] The control module is used to control the drone's landing according to control commands.

[0048] A third aspect of the present invention provides a computer-readable storage medium, the storage medium including a stored program, wherein, when the program is running, it controls the device where the storage medium is located to execute the unmanned aerial vehicle landing control method described in the first aspect or any optional embodiment of the first aspect.

[0049] Compared to existing technologies, the UAV landing control method and apparatus provided by this invention input video stream information containing nested positioning tags and directional positioning tags into a neural network model. This yields output values ​​including the coordinates of the first center point of the nested positioning tags, the coordinates of the second center point of the directional positioning tags, the coordinates of the third center point of the video frame, the width pixel value of the nested positioning tags, and the height pixel value of the nested positioning tags. These output values ​​determine the angle and actual displacement difference between the UAV and the landing landmark. Control commands for the UAV are generated based on the UAV's height, angle, and actual displacement difference to control the UAV's landing. This method differs from existing technologies that rely on solving... Compared to the PnP problem used in traditional methods for estimating the relative pose between a drone and a landmark, this invention only requires inputting video stream information into a neural network model to obtain output values. The number of output values ​​is reduced, allowing for the determination of the angle and actual displacement difference between the drone and the landing landmark using fewer output values. This eliminates the need to solve the PnP problem with numerous parameters to estimate the relative pose, resulting in a shorter time frame for determining the angle and actual displacement difference. Consequently, the time required to control the drone's descent using the control commands generated based on the drone's altitude, angle, and actual displacement difference is shorter, thus improving the efficiency of drone descent control. Attached Figure Description

[0050] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, with the same or corresponding reference numerals denoteing the same or corresponding parts, wherein:

[0051] Figure 1 schematically illustrates a flowchart of a UAV landing control method.

[0052] Figure 2 schematically illustrates a flowchart of the UAV landing control method (II).

[0053] Figure 3 schematically shows a diagram of the landing landmark;

[0054] Figure 4 schematically shows the results of the UAV landing landmark identification and positioning at a high altitude;

[0055] Figure 5 schematically illustrates the landing landmark identification and positioning results of the UAV between high and low altitudes;

[0056] Figure 6 schematically illustrates the results of landmark identification and positioning of a UAV landing at a low altitude;

[0057] Figure 7 schematically shows the structure of the UAV landing control device. Detailed Implementation

[0058] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0059] It should be noted that, unless otherwise stated, the technical or scientific terms used in this invention should have the ordinary meaning as understood by one of ordinary skill in the art.

[0060] For UAV landing control, existing technologies use the Intrinsic Parametric Parameters (IPPE) algorithm based on intrinsic parameter matrices to solve the Position-to-Position (PnP) problem, and then estimate the relative pose between the UAV and the landmark using the solved PnP. Therefore, this invention addresses the issue that existing technologies using the IPPE algorithm based on intrinsic parameter matrices to solve the PnP problem involve converting a two-dimensional image into a three-dimensional space, determining the relative pose of the UAV and the landmark in three-dimensional space, and the large number of parameters involved in determining this pose leads to a long time consumption for solving the PnP problem. This, in turn, results in a long time consumption for estimating the relative pose between the UAV and the landmark, ultimately leading to a longer UAV landing control time. Therefore, this invention seeks a method that uses fewer parameters to shorten the UAV landing control time, replacing the existing method that relies on numerous parameters to solve the PnP problem and thus results in a longer landing control time.

[0061] Therefore, this invention adopts a method that shortens the drone landing control time by using fewer parameters. Video stream information containing nested positioning tags and directional positioning tags is input into a neural network model, yielding output values ​​including the first center point coordinates of the nested positioning tags, the second center point coordinates of the directional positioning tags, the third center point coordinates of the video frame, the width pixel value of the nested positioning tags, and the height pixel value of the nested positioning tags. These output values ​​determine the angle and actual displacement difference between the drone and the landing landmark. Control commands for the drone, generated based on the drone's height, angle, and actual displacement difference, are then used to control the drone's landing. Thus, only video stream information needs to be input into the neural network model to obtain output values. The number of output values ​​is small, and the time required to determine the angle and actual displacement difference between the drone and the landing landmark using these fewer output values ​​is short. This results in a shorter time for the drone to land using control commands generated based on the drone's height, angle, and actual displacement difference, thereby improving the efficiency of drone landing control. The main idea of ​​this invention is to input video stream information containing nested positioning tags and directional positioning tags into a neural network model, which can obtain output values ​​including the first center point coordinates of the nested positioning tags, the second center point coordinates of the directional positioning tags, the third center point coordinates of the video frame, the width pixel value of the nested positioning tags, and the height pixel value of the nested positioning tags. These output values ​​can be used to determine the angle and actual displacement difference between the drone and the landing landmark. Based on the drone's height, angle, and actual displacement difference, control commands are generated to control the drone's landing.

[0062] The methods described in the embodiments of the present invention will be explained in detail below.

[0063] Figure 1 schematically illustrates a flowchart of a drone landing control method according to an embodiment of the present invention. Referring to Figure 1, the method may include:

[0064] S101. Obtain video stream information sent by the drone at the preset location.

[0065] The video stream information includes pre-set landing landmarks, which include nested positioning tags and directional positioning tags.

[0066] The landing landmarks are designed after acquiring the video stream information sent by the drone at the preset location. The landing landmarks include nested positioning tags and directional positioning tags.

[0067] The implementing entity of this invention is a ground station, which is equipped with a neural network model and a closed-loop control system. The ground station can communicate with the UAV and is mainly used to receive video stream information from the UAV. The UAV is equipped with a Global Navigation Satellite System (GNSS) module and a high-resolution camera, among other devices. The GNSS module is used for coarse-grained positioning of the UAV's landing location, i.e., the preset location. The high-resolution camera is mainly used to transmit clear and reliable video stream information, which is beneficial for the neural network model to identify and detect landing landmarks.

[0068] The landing marker is placed at the expected landing location of the drone, specifically at its center. The landing marker determines the relative position of the drone to its expected landing location. It is a circular marker with a flat surface, consisting of nested positioning tags and directional positioning tags. The overall size can be 0.55m × 0.55m, or other sizes; the dimensions are not limited here. Two layers of nested H-shaped positioning tags are nested at the center of the circle, with the inner H-shaped tag nested inside the outer H-shaped tag. Rounded rectangular directional positioning tags are placed inside the circle and outside the two H-shaped tags, directly above the outer H-shaped tag. The center points of the outer and inner H-shaped tags coincide, and they are on the same plane. The area of ​​the outer H-shaped tag is larger than that of the inner H-shaped tag. The outer H-shaped nested positioning tag can be used for high-altitude identification and positioning, while the inner H-shaped nested positioning tag can be used for identification and positioning at lower altitudes. This design addresses the issue of the drone's field of vision narrowing as altitude decreases, and is used to adjust for relative displacement differences with the drone. The rounded rectangle pattern directional positioning tag works in conjunction with the nested tags to adjust the drone's orientation.

[0069] The outer H-shaped nested positioning label measures 18.3cm × 14.5cm, the inner H-shaped nested positioning label measures 3.1cm × 2.5cm, and the rounded rectangle orientation label measures 5.3cm × 5.3cm. The dimensions of the outer, inner, and rounded rectangle nested positioning labels can also be other sizes; therefore, there are no specific limitations on the dimensions of these three types of positioning labels.

[0070] Figure 3 is a schematic diagram of the landing landmark of the present invention. Two nested positioning tags with H-shaped patterns are nested at the center point of the circle. The center point of the outer H-shaped pattern coincides with that of the inner H-shaped pattern, and the area of ​​the outer H-shaped pattern is larger than that of the inner H-shaped pattern. A small ring is set outside the inner H-shaped pattern so that the inner H-shaped pattern falls within the small ring, and the center point of the small ring coincides with the center point of the inner H-shaped pattern. A large ring is set outside the outer H-shaped pattern so that the outer H-shaped pattern falls within the large ring, and the center point of the large ring coincides with the center point of the outer H-shaped pattern. A directional positioning tag with rounded rectangular patterns is set inside the circle and outside the two H-shaped patterns so that the directional positioning tag with rounded rectangular patterns is between the large ring and the circle, and the rounded rectangular pattern is directly above the outer H-shaped pattern. The overall size of the circular landmark is 0.55m × 0.55m. The size of the nested positioning label with the outer H-shaped pattern is 18.3cm × 14.5cm, the size of the nested positioning label with the inner H-shaped pattern is 3.1cm × 2.5cm, and the size of the directional positioning label with the rounded rectangle pattern is 5.3cm × 5.3cm.

[0071] The preset location is the landing position of the UAV based on the coarse-grained positioning of the GNSS module. The UAV can fly to the preset location with its camera facing the ground and collect video stream information containing nested positioning tags and directional positioning tags at the preset location. The collected video stream information is sent to the ground station, which will receive the video stream information sent by the UAV at the preset location in real time.

[0072] S102. Input the video frames corresponding to the video stream information into the neural network model to obtain the output value of the neural network model.

[0073] The output values ​​include the coordinates of the first center point of the nested positioning tag, the coordinates of the second center point of the directional positioning tag, the coordinates of the third center point of the determined video frame, the width (in pixels) of the nested positioning tag, and the height (in pixels) of the nested positioning tag.

[0074] Specifically, the video frame corresponding to the video stream information containing nested positioning tags and directional positioning tags in step S101 is input into the neural network model so that the neural network model can locate the landing landmark's position information in real time, that is, locate the position of the nested positioning tags and directional positioning tags. The output values ​​of the neural network model are: the first center point coordinate value of the nested positioning tag, the second center point coordinate value of the directional positioning tag, the third center point coordinate value of the determined video frame, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag.

[0075] The preferred neural network model is a combination of YOLOv5s and GhostNet networks.

[0076] S103. Based on the coordinates of the first center point, the second center point, the third center point, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag, determine the angle and actual displacement difference between the UAV and the landing landmark.

[0077] Specifically, based on the coordinates of the first center point, the second center point, the third center point, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag obtained in step S102, the angle between the UAV and the landing landmark and the actual displacement difference are determined.

[0078] S104. Generate control commands for the UAV based on its altitude, included angle, and actual displacement difference.

[0079] Specifically, the ground station generates control commands for the UAV based on the UAV's altitude, the angle between the UAV and the landing landmark determined in step S103, and the actual displacement difference.

[0080] The altitude of a drone can be obtained through sensors, and the control commands for the drone are used to control its movement.

[0081] S105. Control the drone to land according to the control commands.

[0082] Specifically, the ground station controls the drone to land according to the control command generated in step S104, so that the drone lands at the expected location.

[0083] As can be seen from the implementation method in Figure 1 above, this embodiment of the invention achieves the purpose of quickly determining the angle and actual displacement difference between the UAV and the landing landmark by using fewer output values ​​of the neural network model. When controlling the landing of the UAV, the video stream information containing nested positioning tags and directional positioning tags is input into the neural network model. Output values ​​are obtained, including the first center point coordinate value of the nested positioning tag, the second center point coordinate value of the directional positioning tag, the third center point coordinate value of the video frame, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag. These output values ​​can be used to determine the angle and actual displacement difference between the UAV and the landing landmark. The control command of the UAV generated based on the height, angle, and actual displacement difference is used to control the landing of the UAV. In this way, only the video stream information needs to be input into the neural network model to obtain output values. The number of output values ​​is small, and the time to determine the angle and actual displacement difference between the UAV and the landing landmark using fewer output values ​​is short. This makes the time to control the landing of the UAV based on the control command generated based on the height, angle, and actual displacement difference of the UAV shorter, thereby improving the efficiency of controlling the landing of the UAV.

[0084] As a refinement and extension of the above embodiments, Figure 2 illustrates the specific operation of drone landing control. Figure 2 is a flowchart of the drone landing control method in this embodiment of the invention. Referring to Figure 2, the drone landing control method provided in this embodiment of the invention may include:

[0085] S201. Obtain video stream information sent by the drone at the preset location.

[0086] The video stream information includes pre-set landing landmarks, which include nested positioning tags and directional positioning tags.

[0087] Before acquiring video stream information transmitted by the drone at a preset location, landing landmarks can be designed first. Landing landmarks include nested positioning tags and directional positioning tags.

[0088] Some parts of step S201 are the same as those of step S101, so the details of step S201 will not be repeated here.

[0089] The preset location is the landing position of the UAV based on the coarse-grained positioning of the GNSS module. The UAV can fly to the preset location with its camera facing the ground and collect video stream information containing nested positioning tags and directional positioning tags at the preset location. The collected video stream information is sent to the ground station, which will receive the video stream information sent by the UAV at the preset location in real time.

[0090] Before acquiring the video stream information sent by the drone at the preset location, the necessary steps include:

[0091] Step A1: Collect image data and use data annotation tools to annotate the collected image data.

[0092] Specifically, the ground station can control the drone to take pictures and record video data at different altitudes and angles. The ground station will process the video data recorded by the drone by extracting frames, extracting images containing landing landmarks, and saving both the captured and extracted images to collect image data. The ground station will also use data annotation tools such as CVAT, VOTT, Labelme, and LabelImg for annotation.

[0093] By marking the nested positioning tags and orientation tags in the landing landmark, the positions of the nested positioning tags and orientation tags can be marked. This involves enclosing the nested positioning tags and orientation tags, recording the coordinates of the upper left and lower right corners of each positioning tag, and then deriving the coordinates of the lower left and upper right corners from the upper left and lower right corner coordinates.

[0094] Step A2: Perform data augmentation on the labeled image data to obtain the image dataset.

[0095] Specifically, data augmentation methods can be used to expand image data, making it more comprehensive. These methods include random rotation, shearing, scaling, translation, cropping, HSV enhancement, and mosaic transformation.

[0096] Step A3: Divide the image dataset.

[0097] Specifically, the image dataset is divided into three parts in a 7:2:1 ratio: training set, validation set, and test set. The training set is used to train the neural network model, the validation set is used to select the neural network model, and the test set is used to evaluate the neural network model.

[0098] Step A4: Based on the segmented image dataset, determine the neural network model to be deployed at the ground station.

[0099] Specifically, the training set can be input into the neural network model to obtain multiple neural network models; the validation set can be input into the multiple neural network models obtained after training for evaluation, and the neural network model with the best fitting effect can be selected; the test set can be input into the best neural network model to test the performance and classification ability of the best neural network model. If the performance and classification ability meet the expected values, the best neural network model is determined as the neural network model that needs to be deployed on the ground station.

[0100] The neural network model is a lightweight neural network model. The improved algorithm of the neural network model is based on the fusion of YOLOv5s and GhostNet. In other words, the neural network model is a combination of YOLOv5s and GhostNet networks. This combined neural network model improves detection speed, reduces the number of network parameters and computational cost, and does not reduce accuracy, effectively achieving a balance between algorithm speed and accuracy.

[0101] Step A5: Deploy the determined neural network model at the ground station to achieve precise landing functionality.

[0102] After deploying the neural network model at the ground station, the drone can be detected. Once the drone completes its mission, it will send a completion command to the ground station. After receiving the completion command, the ground station will send a return-to-home landing command. After receiving the return-to-home landing command, the drone will head to its destination according to the navigation information provided by the GNSS module, and point its high-resolution camera at the ground to monitor ground visual information and transmit the video stream information back to the ground station in real time.

[0103] S202. Input the video frames corresponding to the video stream information into the neural network model to obtain the output value of the neural network model.

[0104] The output values ​​include the coordinates of the first center point of the nested positioning tag, the coordinates of the second center point of the orientation positioning tag, the coordinates of the third center point of the video frame, the width (in pixels) of the nested positioning tag, and the height (in pixels) of the nested positioning tag.

[0105] Specifically, the ground station inputs the video frames corresponding to the video stream information into the deployed neural network model to locate the position information of nested positioning tags and directional positioning tags in the landing landmark in real time. The neural network model can locate the position information of different tags based on the drone's altitude. The neural network detection and positioning is divided into three altitude regions: high, between high and low, and low. A height greater than 5m is considered high, a height greater than 1m and less than 5m is considered between high and low, and a height less than 1m is considered low. When the drone is at a high altitude, that is, when the drone's altitude is greater than the second preset value (5m), the position information of the outer H-shaped nested positioning tag is located, and the corresponding output is the coordinate value of the first center point of the outer H-shaped nested positioning tag, the coordinate value of the third center point of the video frame, the width pixel value of the outer H-shaped nested positioning tag, and the height pixel value of the outer H-shaped nested positioning tag. When the drone is at a high or low altitude (i.e., its altitude is less than the second preset value but greater than the first preset value (1m), the location information and orientation of the outer H-shaped nested positioning tag are determined. The corresponding outputs are the coordinates of the first and second center points of the outer H-shaped nested positioning tag, the coordinates of the third center point of the video frame, the width (in pixels), and the height (in pixels) of the outer H-shaped nested positioning tag. When the drone is at a low altitude (i.e., its altitude is less than the first preset value (1m), the location information of the inner H-shaped nested positioning tag is determined. The corresponding outputs are the coordinates of the first center point of the inner H-shaped nested positioning tag, the coordinates of the third center point of the video frame, the width (in pixels), and the height (in pixels) of the inner H-shaped nested positioning tag.

[0106] Figure 4 is a schematic diagram of the landing landmark identification and positioning results of the UAV at a high altitude according to the present invention. When the height of the UAV is greater than 5m, the neural network model marks the nested positioning tag of the outer H-shaped pattern with a red box. The position information of the nested positioning tag of the outer H-shaped pattern on the landing landmark is detected. The neural network model will output the first center point coordinate value of the nested positioning tag of the outer H-shaped pattern, the third center point coordinate value of the video frame, the width pixel value of the nested positioning tag of the outer H-shaped pattern, and the height pixel value of the nested positioning tag of the outer H-shaped pattern.

[0107] Figure 5 is a schematic diagram of the landing landmark identification and positioning results of the UAV at high and low altitudes according to the present invention. When the height of the UAV is less than 5m and greater than 1m, the neural network model marks the nested positioning tag of the outer H-shaped pattern and the directional positioning tag of the rounded rectangle pattern with red boxes, locates the position information of the nested positioning tag of the outer H-shaped pattern and the directional positioning tag of the rounded rectangle pattern, and outputs the first center point coordinate value, the second center point coordinate value, the third center point coordinate value of the video frame, the width pixel value and the height pixel value of the nested positioning tag of the outer H-shaped pattern.

[0108] Figure 6 is a schematic diagram of the landing landmark identification and positioning results of the UAV at low altitudes according to the present invention. When the height of the UAV is less than 1m, the neural network model marks the nested positioning tag of the inner H-shaped pattern with a red box, locates the position information of the nested positioning tag of the inner H-shaped pattern, and outputs the coordinate value of the first center point of the nested positioning tag of the inner H-shaped pattern, the coordinate value of the third center point of the video frame, the width pixel value of the nested positioning tag of the inner H-shaped pattern, and the height pixel value of the nested positioning tag of the inner H-shaped pattern.

[0109] S203. Based on the coordinates of the first center point, the second center point, the third center point, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag, determine the angle and actual displacement difference between the UAV and the landing landmark.

[0110] Specifically, based on the coordinates of the first center point, the second center point, the third center point, the width (in pixels) of the nested positioning tags, and the height (in pixels) of the nested positioning tags, the angle and actual displacement difference between the drone and the landing landmark are determined, including:

[0111] Get the width and height of nested positioning tags;

[0112] The relative displacement difference between the drone and the landing landmark is determined using the following first formula:

[0113]

[0114] Where distance_pixel_x is the relative displacement difference between the UAV and the landing landmark in the x-axis direction, distance_pixel_y is the relative displacement difference between the UAV and the landing landmark in the y-axis direction, (x1,y1) is the coordinate value of the first center point, and (x2,y2) is the coordinate value of the third center point.

[0115] The actual displacement difference between the drone and the landing landmark is determined using the following second formula:

[0116]

[0117]

[0118] Where distance_x is the actual displacement difference between the drone and the landing landmark in the x-axis direction, distance_y is the actual displacement difference between the drone and the landing landmark in the y-axis direction, k is the number of centimeters per unit pixel, w is the width of the nested positioning tag, h is the height of the nested positioning tag, w_pixel is the width of the nested positioning tag in pixels, and h_pixel is the height of the nested positioning tag in pixels.

[0119] To determine the actual displacement difference between the drone and the landing landmark, including:

[0120] When the height of the drone is greater than the first preset value (i.e., the height of the drone is greater than the first preset value, and the height of the drone is greater than the first preset value but less than the second preset value), in the second formula, w is the width of the nested positioning label of the outer H-shaped pattern, h is the height of the nested positioning label of the outer H-shaped pattern, w_pixel is the width of the nested positioning label of the outer H-shaped pattern in pixels, and h_pixel is the height of the nested positioning label of the outer H-shaped pattern in pixels.

[0121] When the drone's height is less than the first preset value, in the second formula, w is the width of the nested positioning tag of the inner H-shaped pattern, h is the height of the nested positioning tag of the inner H-shaped pattern, w_pixel is the width of the nested positioning tag of the inner H-shaped pattern in pixels, and h_pixel is the height of the nested positioning tag of the inner H-shaped pattern in pixels.

[0122] The relative displacement difference is expressed in pixels. As the drone's flight altitude decreases, the landing pad appears larger in the field of view due to the narrower drone's field of view. Even with the same actual distance between the drone and the landing pad, the displacement difference expressed in pixels varies depending on the altitude. Therefore, the actual distance expressed in centimeters is needed to represent this displacement difference. After calculating the relative displacement difference between the drone and the landing pad, the pixel distance can be further converted to centimeters using the second formula to calculate the actual displacement difference.

[0123] Calculate the angle between the drone and the landing landmark using the following formula:

[0124] α=arctan(Δy / Δx)=arctan[(y1–y2) / (x1–x2)];

[0125] Where α is the angle between the UAV and the landing landmark, (x1,y1) is the coordinate value of the first center point, and (x2,y2) is the coordinate value of the second center point.

[0126] S204. Determine whether the included angle is less than the preset included angle value and whether the actual displacement difference is less than the preset displacement difference value.

[0127] The preset included angle is 5 degrees and the preset displacement difference is 5 centimeters.

[0128] Determine whether the included angle is less than the preset included angle value and whether the actual displacement difference is less than the preset displacement difference value. If yes, proceed to step S205; otherwise, proceed to steps S206-211.

[0129] S205. When the drone's altitude is greater than the second preset value, a descent command is generated; when the drone's altitude is less than the second preset value, a landing command is generated.

[0130] Specifically, it is determined whether the included angle is less than a preset included angle value and whether the actual displacement difference is less than a preset displacement difference value. If so, when the drone's altitude is greater than a second preset value, a descent command is generated, and steps S201-S204 are continued until the drone's altitude is less than the second preset value, at which point a landing command is generated. The second preset value is 5m.

[0131] The following steps S206-S213 are specific operational steps for inputting the included angle and the actual displacement difference into the closed-loop control system, so as to output the linear velocity value or angular velocity value of the UAV through the PID algorithm of the closed-loop control system, and generate translation or rotation commands based on the linear velocity value or angular velocity value.

[0132] S206. When the altitude of the UAV is greater than the second preset value, the included angle and the actual displacement difference are input to the closed-loop control system so that the first linear velocity value of the UAV is output through the PID algorithm of the closed-loop control system.

[0133] Specifically, it determines whether the included angle is less than a preset included angle value and whether the actual displacement difference is less than a preset displacement difference value. If not, when the height of the UAV is greater than the second preset value, the included angle and the actual displacement difference are input into the closed-loop control system so that the first linear velocity value of the UAV can be output through the PID algorithm of the closed-loop control system.

[0134] The PID algorithm for the closed-loop control system is an incremental PID, where the actual output is the increment plus the previous output, as shown in the formula:

[0135]

[0136] Where Vx(k) represents the lateral velocity at time k, Vx(k-1) represents the lateral velocity at time k-1, ΔCx(k) represents the difference between the lateral velocity at time k and the lateral velocity at time k-1, e(k) represents the negative actual displacement difference at time k, e(k-1) represents the negative actual displacement difference at time k-1, e(k-2) represents the negative actual displacement difference at time k-2, distance_x represents the actual displacement difference, and K P K I K D These are three constants, which can be obtained by manual adjustment.

[0137]

[0138] Where Vy(k) represents the longitudinal velocity at time k, Vy(k-1) represents the longitudinal velocity at time k-1, ΔVy(k) represents the difference between the longitudinal velocity at time k and the longitudinal velocity at time k-1, e(k) represents the negative actual displacement difference at time k, e(k-1) represents the negative actual displacement difference at time k-1, e(k-2) represents the negative actual displacement difference at time k-2, distance_x represents the actual displacement difference, and K P K I K D These are three constants, which can be obtained by manual adjustment.

[0139]

[0140] Where ω(k) represents the angular velocity at time k, ω(k-1) represents the angular velocity at time k-1, Δω(k) represents the difference between the angular velocities at time k and k-1, e(k) represents the negative actual displacement difference at time k, e(k-1) represents the negative actual displacement difference at time k-1, e(k-2) represents the negative actual displacement difference at time k-2, distance_x represents the actual displacement difference, and K P K I K D These are three constants, which can be obtained by manual adjustment.

[0141] S207. Generate the first translation command based on the first linear velocity value.

[0142] Specifically, when the drone's altitude exceeds the second preset value, it only needs to be controlled to move horizontally towards the landing marker. Therefore, after outputting the drone's first linear velocity value, only the first translation command needs to be generated based on the first linear velocity value. The ground station sends the first translation command to the drone for execution. The first translation command contains the first linear velocity value.

[0143] S208. When the altitude of the UAV is greater than the first preset value and less than the second preset value, the included angle and the actual displacement difference are input to the closed-loop control system so that the second linear velocity value and angular velocity value of the UAV are output through the PID algorithm of the closed-loop control system.

[0144] The first preset value is less than the second preset value. The first preset value is 1m.

[0145] Specifically, it is determined whether the included angle is less than a preset included angle value and whether the actual displacement difference is less than a preset displacement difference value. If not, when the drone's height is greater than a first preset value and less than a second preset value, the included angle and the actual displacement difference are input into the closed-loop control system. The PID algorithm of the closed-loop control system then outputs the drone's second linear velocity and angular velocity values. The formula for the PID algorithm here is the same as the formula in step S206.

[0146] S209. Generate the second translation command and rotation command based on the second linear velocity value and angular velocity value.

[0147] Specifically, when the drone's altitude is greater than the first preset value but less than the second preset value, it is necessary to control the drone to translate to the vicinity of the landing landmark and to rotate it to the specified direction. Therefore, after outputting the drone's second linear velocity and angular velocity values, it is necessary to generate second translation and rotation commands based on these values. The generated second translation command includes the second linear velocity value, and the generated rotation command includes the angular velocity value.

[0148] S210. When the height of the UAV is less than the first preset value, the included angle and the actual displacement difference are input to the closed-loop control system so that the third linear velocity value of the UAV is output through the PID algorithm of the closed-loop control system.

[0149] Specifically, it is determined whether the included angle is less than a preset included angle value and whether the actual displacement difference is less than a preset displacement difference value. If not, when the height of the UAV is less than the first preset value, the included angle and the actual displacement difference are input into the closed-loop control system so that the third linear velocity value of the UAV can be output through the PID algorithm of the closed-loop control system. The formula of the PID algorithm here is the formula in step S206.

[0150] S211. Generate a third translation command based on the third linear velocity value, and generate a landing command when the current actual displacement difference is less than the preset displacement difference value.

[0151] Specifically, when the drone's altitude is greater than the first preset value but less than the second preset value, it only needs to be controlled to move horizontally to the vicinity of the landing landmark. Therefore, after outputting the drone's third linear velocity value, a third translation command needs to be generated based on the third linear velocity value. Furthermore, a landing command is generated when the current actual displacement difference is less than the preset displacement difference value. The generated third translation command includes the third linear velocity value.

[0152] S212. When generating the first translation command, control the UAV to translate the distance of the actual displacement difference at the first linear velocity value.

[0153] Specifically, when generating the first translation command in step S207, the UAV is controlled to translate by the distance of the actual displacement difference using the first linear velocity value in the first translation command.

[0154] S213. When generating the second translation command and rotation command, control the UAV to translate the distance of the actual displacement difference at the second linear velocity value, and control the UAV to rotate the included angle at the angular velocity value during the translation process.

[0155] Specifically, when generating the second translation command and rotation command in step S209, the drone is controlled to translate the distance of the actual displacement difference at the second linear velocity value, and the drone is controlled to rotate the included angle at the angular velocity value during the translation process.

[0156] S214. When generating the third translation command, control the UAV to translate by the distance of the actual displacement difference at the third linear velocity value, and control the UAV to land when the current actual displacement difference is less than the preset displacement difference value.

[0157] Specifically, when the third translation command is generated in step S211, the drone is controlled to translate at a third linear velocity value by the distance of the actual displacement difference, and when the current actual displacement difference is less than a preset displacement difference value, the drone is controlled to land. The preset displacement difference value is 5 centimeters.

[0158] Based on the same inventive concept, as an implementation of the above-mentioned UAV landing control method, this embodiment of the invention also provides a UAV landing control device. Figure 7 is a structural diagram of the device in this embodiment of the invention. Referring to Figure 7, the device may include:

[0159] The acquisition module 701 is used to acquire video stream information sent by the UAV at a preset location. The video stream information contains information of a preset landing landmark, which includes nested positioning tags and directional positioning tags.

[0160] The input module 702 is used to input the video frames corresponding to the video stream information into the neural network model to obtain the output values ​​of the neural network model. The output values ​​include the coordinate values ​​of the first center point of the nested positioning tag, the coordinate values ​​of the second center point of the directional positioning tag, the coordinate values ​​of the third center point of the video frame, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag.

[0161] The determination module 703 is used to determine the angle and actual displacement difference between the UAV and the landing landmark based on the coordinate values ​​of the first center point, the second center point, the third center point, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag.

[0162] The generation module 704 is used to generate control commands for the UAV based on the UAV's altitude, included angle, and actual displacement difference.

[0163] The control module 705 is used to control the landing of the drone according to control commands.

[0164] Furthermore, module 703 is specifically used to obtain the width and height of the nested positioning labels;

[0165] The relative displacement difference between the drone and the landing landmark is determined using the following first formula:

[0166]

[0167] Where distance_pixel_x is the relative displacement difference between the UAV and the landing landmark in the x-axis direction, distance_pixel_y is the relative displacement difference between the UAV and the landing landmark in the y-axis direction, (x1,y1) is the coordinate value of the first center point, and (x2,y2) is the coordinate value of the third center point.

[0168] The actual displacement difference between the drone and the landing landmark is determined using the following second formula:

[0169]

[0170]

[0171] Where distance_x is the actual displacement difference between the drone and the landing landmark in the x-axis direction, distance_y is the actual displacement difference between the drone and the landing landmark in the y-axis direction, k is the number of centimeters per unit pixel, w is the width of the nested positioning tag, h is the height of the nested positioning tag, w_pixel is the width of the nested positioning tag in pixels, and h_pixel is the height of the nested positioning tag in pixels.

[0172] Furthermore, the landing landmark in the acquisition module 701 is circular. Nested positioning labels with two H-shaped patterns are nested at the center point of the circle. Directional positioning labels with rounded rectangle patterns are set inside the circle and outside the two H-shaped patterns. The rounded rectangle pattern is directly above the outer H-shaped pattern. The center point of the outer H-shaped pattern coincides with that of the inner H-shaped pattern, and the outer H-shaped pattern and the inner H-shaped pattern are on the same plane.

[0173] Furthermore, the determination module 703 is specifically used when the height of the drone is greater than the first preset value, in the second formula w is the width of the nested positioning tag of the outer H-shaped pattern, h is the height of the nested positioning tag of the outer H-shaped pattern, w_pixel is the width pixel of the nested positioning tag of the outer H-shaped pattern, and h_pixel is the height pixel of the nested positioning tag of the outer H-shaped pattern.

[0174] When the drone's height is less than the first preset value, in the second formula, w is the width of the nested positioning tag of the inner H-shaped pattern, h is the height of the nested positioning tag of the inner H-shaped pattern, w_pixel is the width of the nested positioning tag of the inner H-shaped pattern in pixels, and h_pixel is the height of the nested positioning tag of the inner H-shaped pattern in pixels.

[0175] Furthermore, the generation module 704 is specifically used to determine whether the included angle is less than a preset included angle value and whether the actual displacement difference is less than a preset displacement difference value; if so, when the height of the UAV is greater than the second preset value, a descent command is generated, and when the height of the UAV is less than the second preset value, a landing command is generated; if not, the included angle and the actual displacement difference are input to the closed-loop control system, so that the PID algorithm of the closed-loop control system can output the linear velocity value or angular velocity value of the UAV, and generate translation or rotation commands based on the linear velocity value or angular velocity value.

[0176] Furthermore, the generation module 704 is specifically used to: input the included angle and actual displacement difference into the closed-loop control system when the drone's altitude is greater than the second preset value, so as to output the drone's first linear velocity value through the PID algorithm of the closed-loop control system; generate a first translation command based on the first linear velocity value; input the included angle and actual displacement difference into the closed-loop control system when the drone's altitude is greater than the first preset value but less than the second preset value, so as to output the drone's second linear velocity value and angular velocity value through the PID algorithm of the closed-loop control system, where the first preset value is less than the second preset value; generate a second translation command and a rotation command based on the second linear velocity value and angular velocity value; input the included angle and actual displacement difference into the closed-loop control system when the drone's altitude is less than the first preset value, so as to output the drone's third linear velocity value through the PID algorithm of the closed-loop control system; generate a third translation command based on the third linear velocity value; and generate a landing command when the current actual displacement difference is less than a preset displacement difference value.

[0177] Furthermore, the control module 705 is specifically used to control the UAV to translate the actual displacement difference by a distance at a first linear velocity value when generating the first translation command; to control the UAV to translate the actual displacement difference by a distance at a second linear velocity value when generating the second translation command and the rotation command, and to control the UAV to rotate the included angle by an angular velocity value during the translation process; to control the UAV to translate the actual displacement difference by a distance at a third linear velocity value when generating the third translation command, and to control the UAV to land when the current actual displacement difference is less than a preset displacement difference value.

[0178] Furthermore, the input module 702 is specifically used for a neural network model that combines the YOLOv5s network and the GhostNet network.

[0179] It should be noted that the description of the above embodiments of the UAV landing control device is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the embodiments of the UAV landing control device of this invention, please refer to the description of the method embodiments of this invention for understanding.

[0180] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium, the computer-readable storage medium including a stored program, wherein, when the program is running, it controls the device where the storage medium is located to execute the methods in one or more of the above embodiments.

[0181] It should be noted that the descriptions of the above computer-readable storage medium embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the embodiments of the computer-readable storage medium of this invention, please refer to the descriptions of the method embodiments of this invention for understanding.

[0182] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for controlling the landing of an unmanned aerial vehicle (UAV), characterized in that, The method includes: acquiring video stream information sent by a drone at a preset location, the video stream information containing information of a pre-set landing landmark, the pre-set landing landmark including nested positioning tags and directional positioning tags; inputting video frames corresponding to the video stream information into a neural network model to obtain the output value of the neural network model, the output value including the first center point coordinate value of the nested positioning tag, the second center point coordinate value of the directional positioning tag, the third center point coordinate value of the video frame, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag; determining the angle and actual displacement difference between the drone and the landing landmark based on the first center point coordinate value, the second center point coordinate value, the third center point coordinate value, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag; and determining the angle and actual displacement difference between the drone and the landing landmark based on the drone's height and the angle. Based on the actual displacement difference, control commands for the UAV are generated; the UAV is controlled to land according to the control commands; the landing landmark is circular, and two nested positioning tags with H-shaped patterns are nested at the center point of the circle. Rounded rectangular directional positioning tags are set inside the circle and outside the two H-shaped patterns, with the rounded rectangular patterns directly above the outer H-shaped pattern. The center points of the outer and inner H-shaped patterns coincide, and the outer and inner H-shaped patterns are on the same plane; the angle and actual displacement difference between the UAV and the landing landmark are determined based on the first center point coordinates, the second center point coordinates, the third center point coordinates, the width pixel value of the nested positioning tags, and the height pixel value of the nested positioning tags, including: obtaining the width and height of the nested positioning tags; and determining the relative displacement difference between the UAV and the landing landmark according to the following first formula: ;in, The relative displacement difference between the UAV and the landing landmark in the x-axis direction is denoted as . The relative displacement difference between the UAV and the landing landmark in the y-axis direction is ( ) represents the coordinates of the first center point. The coordinates of the third center point are given; the actual displacement difference between the UAV and the landing landmark is determined according to the following second formula: ； ;in, The difference between the actual displacement of the UAV and the landing landmark in the x-axis direction. The difference between the actual displacement of the UAV and the landing landmark in the y-axis direction. The unit of pixels is centimeters. The width of the nested positioning label. The height of the nested positioning label. The width of the nested positioning label in pixels. The height in pixels of the nested positioning label.

2. The method according to claim 1, characterized in that, The determination of the actual displacement difference between the UAV and the landing landmark according to the following second formula includes: when the altitude of the UAV is greater than a first preset value, the second formula... h represents the width of the nested positioning label within the outer H-shaped pattern, and h represents the height of the nested positioning label within the outer H-shaped pattern. The width (in pixels) of the nested positioning label for the outer H-shaped pattern. The height pixels of the nested positioning tag of the outer H-shaped pattern; when the height of the drone is less than the first preset value, the second formula... h represents the width of the nested positioning label with the inner H-shaped pattern, and h represents the height of the nested positioning label with the inner H-shaped pattern. The width in pixels for the nested positioning label of the inner H-shaped pattern. The height in pixels of the nested positioning label of the inner H-shaped pattern.

3. The method according to claim 2, characterized in that, The step of generating control commands for the UAV based on its altitude, the included angle, and the actual displacement difference includes: determining whether the included angle is less than a preset included angle value and whether the actual displacement difference is less than a preset displacement difference value; if so, generating a descent command when the UAV's altitude is greater than a second preset value, and generating a landing command when the UAV's altitude is less than a second preset value; if not, inputting the included angle and the actual displacement difference into a closed-loop control system, so as to output the UAV's linear velocity value or angular velocity value through the PID algorithm of the closed-loop control system, and generating translation or rotation commands based on the linear velocity value or the angular velocity value.

4. The method according to claim 3, characterized in that, The step of inputting the included angle and the actual displacement difference into the closed-loop control system, so as to output the linear velocity or angular velocity value of the UAV through the PID algorithm of the closed-loop control system, and generating translation or rotation commands based on the linear velocity or angular velocity value, includes: when the height of the UAV is greater than the second preset value, inputting the included angle and the actual displacement difference into the closed-loop control system, so as to output the first linear velocity value of the UAV through the PID algorithm of the closed-loop control system; generating a first translation command based on the first linear velocity value; when the height of the UAV is greater than the first preset value and less than the second preset value, inputting the included angle and the actual displacement difference into the closed-loop control system, so as to output the first linear velocity value of the UAV through the PID algorithm of the closed-loop control system; generating a first translation command based on the first linear velocity value; and ... generating a translation or rotation command based on the first linear velocity value. The difference is input to the closed-loop control system to output the second linear velocity value and the angular velocity value of the UAV through the PID algorithm of the closed-loop control system, wherein the first preset value is less than the second preset value; a second translation command and a rotation command are generated based on the second linear velocity value and the angular velocity value; when the altitude of the UAV is less than the first preset value, the included angle and the actual displacement difference are input to the closed-loop control system to output the third linear velocity value of the UAV through the PID algorithm of the closed-loop control system; a third translation command is generated based on the third linear velocity value, and a landing command is generated when the current actual displacement difference is less than the preset displacement difference value.

5. The method according to claim 4, characterized in that, The step of controlling the drone to land according to the control command includes: when generating a first translation command, controlling the drone to translate the distance of the actual displacement difference at the first linear velocity value; when generating a second translation command and a rotation command, controlling the drone to translate the distance of the actual displacement difference at the second linear velocity value, and controlling the drone to rotate the included angle at the angular velocity value during the translation process; when generating a third translation command, controlling the drone to translate the distance of the actual displacement difference at the third linear velocity value, and controlling the drone to land when the current actual displacement difference is less than the preset displacement difference value.

6. The method according to claim 1, characterized in that, The neural network model is a combination of the YOLOv5s network and the GhostNet network.

7. A drone landing control device for implementing the drone landing control method according to any one of claims 1-6, characterized in that, The device includes: an acquisition module for acquiring video stream information sent by a drone at a preset location, the video stream information containing information of a pre-set landing landmark, the pre-set landing landmark including a nested positioning tag and a directional positioning tag; an input module for inputting video frames corresponding to the video stream information into a neural network model to obtain the output value of the neural network model, the output value including the first center point coordinate value of the nested positioning tag, the second center point coordinate value of the directional positioning tag, the third center point coordinate value of the video frame, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag; a determination module for determining the angle and actual displacement difference between the drone and the landing landmark based on the first center point coordinate value, the second center point coordinate value, the third center point coordinate value, the width pixel value of the nested positioning tag, and the height pixel value of the nested positioning tag; a generation module for generating control commands for the drone based on the drone's height, the angle, and the actual displacement difference; and a control module for controlling the drone to land according to the control commands.

8. A computer-readable storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, the device containing the storage medium is controlled to perform the UAV landing control method as described in any one of claims 1 to 6.

Images