Remote control unmanned aerial vehicle emergency precision landing method and system, and electronic device

Abstract

The present application relates to the technical field of unmanned aerial vehicle emergency accurate landing scheme design, in particular to a remote control unmanned aerial vehicle emergency accurate landing method and system and electronic equipment. When the unmanned aerial vehicle cannot be safely landed due to low power, the operator can find an open and personnel-unreachable place in the unmanned aerial vehicle flight video picture, directly frame the position in the video picture, and the unmanned aerial vehicle will automatically and quickly fly to the place and land. While tracking the framed position by using a target tracking algorithm, the gimbal is adjusted to keep the framed position in the center of the camera picture, and the landing position is updated and returned to the remote platform. The remote control unmanned aerial vehicle emergency accurate landing is realized, the maintenance personnel can quickly and safely find the landed aircraft, and the intelligent degree, accuracy, safety and reliability of the remote control unmanned aerial vehicle landing are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of emergency precision landing scheme design technology for unmanned aerial vehicles (UAVs), specifically to a remote-controlled UAV emergency precision landing method and system, and electronic equipment. Background Technology

[0002] With the increasing prevalence of remote drone hangar monitoring, the application of remote drone control is also growing, making flight safety increasingly important. Although the maturity of drone flight control algorithms has been proven by the market, limitations in battery technology mean that flight time is generally limited to 30-60 minutes. Current technologies face challenges including: Target occlusion and overlap: Tracking algorithms may encounter difficulties when targets occlude or overlap. This can lead to the algorithm failing to correctly identify the target, or generating false detections or incorrect data associations when the target reappears.

[0003] Illumination variations: Illumination variations can affect the appearance of a target, making traditional appearance-based tracking algorithms unstable. Under poor or rapidly changing lighting conditions, algorithms may produce false detections or missed detections.

[0004] Target deformation: When a target undergoes deformation or changes in posture, traditional tracking algorithms based on appearance features may fail to track the target correctly.

[0005] Data association errors: In multi-target tracking, data association errors can lead to incorrect target associations, resulting in erroneous tracking results. This may be due to high similarity in appearance or similar motion patterns between targets.

[0006] Therefore, existing technologies for long-distance flight have four significant drawbacks: 1. Limited airport deployment environments, often lacking suitable locations for setting up alternate landing points. 2. Due to the long-distance nature of the flight, operators cannot accurately ascertain weather conditions at the time of flight, potentially resulting in wind speeds exceeding the drone's required landing speed. 3. Low efficiency of manual intervention during landing due to the long-distance nature of the flight. 4. Due to the algorithm's relative planar limitations, the calculated GPS position corresponding to a drone pixel is the GPS coordinate of that pixel relative to the drone's takeoff calibration plane. This means that when the drone's camera gimbal pitch angle is too large, the GPS will experience significant offset, further hindering accurate landing. Furthermore, upon reaching the landing point, due to the drone's high-altitude flight characteristics, the actual range corresponding to the selected pixel frame is too large, leading to significant offset during landing and rendering the visual tracking algorithm ineffective. These four pain points directly lead to a significant decrease in landing efficiency when the drone has finished its flight and is returning with low battery levels, or when the wind at the landing point is too strong to land stably on the helipad. When the minimum landing battery level is triggered, the drone will land directly, which may result in damage to the drone or, in severe cases, a fall from a high altitude causing unpredictable losses. At the same time, the uncontrollable landing location can easily lead to the serious problem of the drone being lost.

[0007] Therefore, existing technologies still need further development. Summary of the Invention

[0008] The purpose of this invention is to overcome the above-mentioned technical deficiencies and provide a remote-controlled method, system, and electronic device for emergency and precise landing of unmanned aerial vehicles (UAVs) to solve the problems existing in the prior art.

[0009] To achieve the above-mentioned technical objectives, according to a first aspect of the present invention, the present invention provides a method for remotely controlling an unmanned aerial vehicle (UAV) to make an emergency and precise landing, comprising: S100. Obtain the current frame image captured by the drone camera, obtain the pixel coordinates of the four vertices of the first rectangular landing area selected by the user in the current frame image, and then calculate the GPS coordinates of the center point of the first rectangular landing area, which are recorded as the target GPS coordinates. S200: Use a target tracking algorithm to track the first rectangular landing area, and control the gimbal angle to keep the center of the first rectangular landing area always in the center of the camera image. Control the drone to move closer to the target GPS coordinates. At preset intervals, reacquire the current frame image of the drone and the pixel coordinates of the four vertices of the first rectangular landing area, and then recalculate and update the target GPS coordinates of the center point of the first rectangular landing area. S300: Obtain the current coordinates of the drone, determine whether the distance between the current coordinates and the target GPS coordinates is less than or equal to a first preset threshold. If so, control the camera plane to be parallel to the ground, establish a second rectangular landing area of ​​preset size with the target GPS coordinates as the center, use the target tracking algorithm to track the second rectangular landing area, control the drone to keep the center of the second rectangular landing area always in the center position of the camera screen, and then control the drone to reduce the flight altitude.

[0010] Specifically, the method further includes: If the center of the second rectangular landing area shifts relative to the center of the camera view during the drone's descent, stop the drone from descending, adjust the drone so that the center of the second rectangular landing area is in the center of the camera view, and then continue to control the drone to descend until the drone completes the landing.

[0011] Specifically, the method for determining whether the center of the second rectangular landing area has shifted relative to the center of the camera image includes: During the process of the drone descending its flight altitude, it is determined whether the distance between the current coordinates and the target GPS coordinates is less than or equal to a second preset threshold, and based on the determination result, it is determined whether the center of the second rectangular landing area has shifted relative to the center of the camera image. If the distance between the current coordinates and the target GPS coordinates is less than or equal to the second preset threshold, it is determined that the center of the second rectangular landing area has not shifted relative to the center of the camera image. If the distance between the current coordinates and the target GPS coordinates is greater than the second preset threshold, it is determined that the center of the second rectangular landing area has shifted relative to the center of the camera image.

[0012] Specifically, the method further includes: If the distance between the current coordinates and the target GPS coordinates is greater than the first preset threshold, continue to control the movement towards the target GPS coordinates.

[0013] Specifically, the step of using a target tracking algorithm to track the first rectangular landing area includes: The KCF algorithm is used to track the first rectangular landing area. The peak-to-sidelobe ratio is used to determine whether the first rectangular landing area is occluded or lost. After the first rectangular landing area is lost, the target retrieval and matching strategy of SURF-RANSAC is used to re-match and select the first rectangular landing area.

[0014] Specifically, the re-matching of the first rectangular landing area based on the SURF-RANSAC target retrieval matching strategy includes: After the first rectangular landing area is lost, the first rectangular landing area is re-searched and located. During the tracking of the first rectangular landing area by the KCF algorithm, the tracking status is judged, the SURF features of the first rectangular landing area are extracted, and a template of the first rectangular landing area is generated. When the first rectangular landing area is occluded or lost, the SURF features of the next frame image are extracted and matched with the template of the first rectangular landing area. The first rectangular landing area that best matches the features of the template is selected, and then the tracking model is updated to continue tracking the first rectangular landing area.

[0015] Specifically, the re-matching of the first rectangular landing area based on the SURF-RANSAC target retrieval matching strategy further includes: The Random Sampling Consensus Algorithm (RANSAC) and the SURF algorithm are combined to eliminate similar matches and mismatches. Based on the SURF-RANSAC target retrieval matching strategy, the first rectangular landing area is rematched. The matching results are then used to select the new first rectangular landing area, which is then input into the KCF algorithm to continue the tracking task for the first rectangular landing area.

[0016] Specifically, the matching process for re-matching the first rectangular landing area based on the SURF-RANSAC target retrieval matching strategy is as follows: (a) Read in the current frame image and the next frame image respectively; (b) Perform SURF feature point detection on the current frame image and the next frame image respectively to obtain two sets of feature points, point1 and point2; (c) For each point i in points1, find the corresponding point j in points2; (d) For each point k in points2, find the corresponding point l in points1; (e) If point i is matched by point j, and point j is matched by point i, then the match is successful; (f) Use the RANSAC algorithm to remove mismatches in the SURF algorithm matching.

[0017] According to a second aspect of the present invention, a remote-controlled unmanned aerial vehicle (UAV) emergency precision landing system is provided, comprising: The acquisition module is used to acquire the pixel coordinates of the four vertices of the first rectangular landing area selected by the user in the current frame image; it is also used to acquire the current coordinates of the drone. The control module is used to acquire the current frame image captured by the drone camera; to calculate the GPS coordinates of the center point of the first rectangular landing area, denoted as the target GPS coordinates; to track the first rectangular landing area using a target tracking algorithm, and to control the gimbal angle so that the center of the first rectangular landing area is always located in the center of the camera image, controlling the drone to move closer to the target GPS coordinates. At preset time intervals, it reacquires the current frame image of the drone and the pixel coordinates of the four vertices of the first rectangular landing area, and then recalculates and updates the target GPS coordinates of the center point of the first rectangular landing area; and to determine whether the distance between the current coordinates and the target GPS coordinates is less than or equal to a first preset threshold. If so, it controls the camera plane to be parallel to the ground, establishes a second rectangular landing area of ​​a preset size with the target GPS coordinates as the center, tracks the second rectangular landing area using a target tracking algorithm, controls the drone to keep the center of the second rectangular landing area always located in the center of the camera image, and then controls the drone to decrease its flight altitude.

[0018] According to a third aspect of the present invention, an electronic device is provided, comprising: a memory; and a processor, wherein the memory stores computer-readable instructions, which, when executed by the processor, implement the above-described method for remotely controlling an unmanned aerial vehicle to make an emergency and precise landing.

[0019] Beneficial effects: This invention provides a remote-controlled method, system, and electronic device for emergency and precise landing of a drone. When a drone is too low on battery to land safely, the operator can locate an open area that is difficult for people to reach in the drone's flight video feed. The operator can then select the location directly in the video feed, and the drone will automatically and quickly fly to that location and land. While using a target tracking algorithm to track the selected location, the gimbal is adjusted to keep the selected location centered in the camera's view. Simultaneously, the landing position is updated and returned to the remote platform. This enables remote-controlled emergency and precise landing of the drone, allowing maintenance personnel to quickly and safely locate the landed aircraft. It significantly improves the intelligence, accuracy, safety, and reliability of remote-controlled drone landing. Attached Figure Description

[0020] Figure 1 This is a flowchart of a remote-controlled drone emergency precision landing method provided in a specific embodiment of the present invention; Figure 2 This is a schematic diagram of a remote-controlled drone emergency precision landing method provided in a specific embodiment of the present invention; Figure 3 This is a schematic diagram of the system composition of the remote-controlled UAV emergency precision landing system provided in a specific embodiment of the present invention; Figure 4This is a schematic diagram of a drone tracking a first rectangular landing area during flight, provided in a specific embodiment of the present invention; The above figures contain the following reference numerals: 1. Drones. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments in this application, other similar embodiments obtained by those skilled in the art without creative effort should all fall within the scope of protection of this application. Furthermore, directional terms mentioned in the following embodiments, such as "up," "down," "left," and "right," are only for reference to the directions in the accompanying drawings; therefore, the directional terms used are for illustrative purposes and not for limiting the invention.

[0022] The present invention will be further described below with reference to the accompanying drawings and preferred embodiments.

[0023] Please see Figures 1-3 This invention provides a method for remotely controlling an unmanned aerial vehicle (UAV) to make an emergency and precise landing, comprising: S100. Obtain the current frame image captured by the drone camera, obtain the pixel coordinates of the four vertices of the first rectangular landing area selected by the user in the current frame image, and then calculate the GPS coordinates of the center point of the first rectangular landing area, which are recorded as the target GPS coordinates.

[0024] It should be noted here that you should refer to [link / reference]. Figure 4 The attached diagram, labeled 1, indicates a drone. Figure 4 Quadrilateral ABCD is the first rectangular landing region, and point O is the center of the first rectangular landing region. Figure 4 In the diagram, the four sides EFGH form the second rectangular landing area, and O' is the center of the second rectangular landing area.

[0025] It should be noted here that the step of obtaining the pixel coordinates of the four vertices of the first rectangular landing area selected by the user in the current frame image, and then calculating the GPS coordinates of the center point of the first rectangular landing area, includes: The pixel coordinates of the center point of the first rectangular landing area are calculated based on the pixel coordinates of the four vertices of the first rectangular landing area. The GPS coordinates of the center point of the first rectangular landing area are calculated using the homography matrix and the pixel coordinates of the center point of the first rectangular landing area, and are denoted as the target GPS coordinates.

[0026] It should be noted that this invention uses a homography matrix to perform the mutual conversion between pixel coordinates and geographic coordinates. Pixel coordinates can be freely selected in the video frames on the web platform, and are converted into geographic coordinates through the homography matrix. Similarly, when drawing a surface composed of geographic coordinates on the web GIS platform, it can also be converted into pixel coordinates through the homography matrix and sent to a specified topic for use. The application of the homography matrix in the conversion between pixel coordinates and geographic coordinates is existing technology, and this invention will not elaborate on it further.

[0027] It is understandable that, to ensure effective tracking, the length and width of the first rectangular landing area must both be greater than or equal to 50 pixels. In this case, the length and width of the second rectangular landing area are the minimum trackable dimensions of the length and width of the first rectangular landing area, i.e., both the length and width of the second rectangular landing area are 50 pixels. This avoids the problem that if a second rectangular landing area of ​​a fixed size is not established, the first rectangular landing area selected by the user may expand rapidly in the camera's view due to the drone's descent in flight altitude, leading to the loss of landing area tracking. This greatly improves the accuracy of the emergency landing position of the remotely controlled drone, making it easier for maintenance personnel to quickly and safely locate the landing aircraft. It also significantly improves the intelligence, accuracy, safety, and reliability of the remotely controlled drone landing. The above settings were obtained by the inventors through extensive testing, further improving the usability of the invention.

[0028] S200: The first rectangular landing area is tracked using a target tracking algorithm. By controlling the gimbal angle, the center of the first rectangular landing area is always located in the center of the camera image. The drone is controlled to move closer to the target GPS coordinates. At preset intervals, the current frame image of the drone and the pixel coordinates of the four vertices of the first rectangular landing area are re-acquired. Then, the target GPS coordinates of the center point of the first rectangular landing area are recalculated and updated.

[0029] Specifically, the step of using a target tracking algorithm to track the first rectangular landing area includes: The KCF algorithm is used to track the first rectangular landing area. The peak-to-sidelobe ratio is used to determine whether the first rectangular landing area is occluded or lost. After the first rectangular landing area is lost, the target retrieval and matching strategy of SURF-RANSAC is used to re-match and select the first rectangular landing area.

[0030] Specifically, determining whether the first rectangular landing area is obscured or lost based on the peak-to-sidelobe ratio includes: (1) Calculate the correlation coefficient of the first rectangular landing area.

[0031] This invention introduces a strategy of re-searching and locating the first rectangular landing region after it is lost. During KCF tracking of the first rectangular landing region, the tracking status is assessed, and SURF features of the first rectangular landing region are extracted to generate a template. When the first rectangular landing region is occluded or lost, the SURF features of the next frame's first rectangular landing region image are extracted and matched with the template. The first rectangular landing region that best matches the template features is selected, the tracking model is updated, and tracking of the first rectangular landing region continues. To reduce the time consumption caused by additional computation and improve the speed of the tracking algorithm, this invention uses a correlation coefficient response map of the first rectangular landing region's location to represent the reliability of the tracking results. When the first rectangular landing region tracked in the current frame correctly matches the actual first rectangular landing region, the response map has only one peak; the sharper the correlation peak, the higher the positioning accuracy. When the first rectangular landing region is occluded or lost, the entire response map will exhibit peak fluctuations. If the algorithm continues to update the first rectangular landing region model in this case, the tracking result is likely to be incorrect.

[0032] (2) Calculate the peak sidelobe ratio.

[0033] This invention determines whether the tracking of the first rectangular landing area is obstructed or lost by obtaining the Peak Sidelobe Ratio (PSR) of the response map. The formula for calculating the Peak Sidelobe Ratio (PSR) is as follows: In the formula: p is the maximum peak value, μ is the mean of the sidelobe region, and σ is the variance. Two thresholds, T1 and T2, are set, with T1 > T2. When PSR ≥ T1, it is determined that the tracking of the first rectangular landing region is unobstructed and unlost, and the algorithm continues to track the current first rectangular landing region. When T2 ≤ PSR < T1, it is determined that the first rectangular landing region has begun to enter an occlusion state, and the algorithm begins to extract the image within the tracking box of the current frame as the template for the first rectangular landing region, and extracts the SURF features of the first rectangular landing region template. When PSR < T2, it is determined that the first rectangular landing region is severely occluded or lost. At this time, the SURF features of the current frame are extracted and matched with the first rectangular landing region template. After a successful match, the first rectangular landing region is selected, the kernel correlation filter algorithm is updated, and the tracking of the first rectangular landing region continues.

[0034] Specifically, the re-matching of the first rectangular landing area based on the SURF-RANSAC target retrieval matching strategy includes: After the first rectangular landing area is lost, the first rectangular landing area is re-searched and located. During the tracking of the first rectangular landing area by the KCF algorithm, the tracking status is judged, the SURF features of the first rectangular landing area are extracted, and a template of the first rectangular landing area is generated. When the first rectangular landing area is occluded or lost, the SURF features of the next frame image are extracted and matched with the template of the first rectangular landing area. The first rectangular landing area that best matches the features of the template is selected, and then the tracking model is updated to continue tracking the first rectangular landing area.

[0035] It should be noted that Speeded-Up Robust Feature Transform (SURF) is a local feature description algorithm with strong adaptability to image rotation and scaling, and is a comprehensive upgrade of Scale Invariant Feature Transform (SIFT). Compared to the SIFT algorithm, the SURF algorithm has greater efficiency and robustness. In the SURF algorithm, a Hessian matrix is ​​constructed to detect and extract feature values ​​of specific regions in the image. It should be noted that using a Hessian matrix to detect and extract feature values ​​of specific regions in an image is existing technology, and will not be elaborated upon further in this invention.

[0036] Understandably, to ensure rotation invariance of image features, SURF statistically analyzes Haar wavelet features within the neighborhood of each feature point. Using 60-degree sectors as units, the sum of the vertical and horizontal Haar wavelet response values ​​of feature points within each sector is calculated. This sector is then rotated at regular intervals, and the direction with the largest Haar wavelet response value after rotation is taken as the principal direction of the feature point. Next, a 4×4 sub-region is selected centered on the feature point, and Haar wavelet response values ​​are detected for each sub-region. The sum of the x-direction responses, the sum of the absolute values ​​of the x-direction responses, the sum of the y-direction responses, and the sum of the absolute values ​​of the y-direction responses are denoted as ∑dx, ∑dx, ∑dy, and ∑dy, respectively. Connecting the 4-dimensional vectors of each region according to certain rules yields a 64-dimensional feature vector for each feature point. Finally, the similarity between feature vectors is used as the criterion for determining whether image features from different viewpoints have successfully matched.

[0037] Specifically, the re-matching of the first rectangular landing area based on the SURF-RANSAC target retrieval matching strategy further includes: The Random Sampling Consensus Algorithm (RANSAC) and the SURF algorithm are combined to eliminate similar matches and mismatches. Based on the SURF-RANSAC target retrieval matching strategy, the first rectangular landing area is rematched. The matching results are then used to select the new first rectangular landing area, which is then input into the KCF algorithm to continue the tracking task for the first rectangular landing area.

[0038] It should be noted that images are susceptible to influences from factors such as lighting and scene complexity during acquisition, leading to errors in feature point detection and matching, resulting in a certain number of false matches. This invention combines the Random Sample Consensus (RANSAC) algorithm with the SURF algorithm to eliminate similar matches and false matches. Given the known correspondence between feature points in images, the relative changes between images are calculated using the homography matrix.

[0039] Specifically, the matching process for re-matching the first rectangular landing area based on the SURF-RANSAC target retrieval matching strategy is as follows: (a) Read in the current frame image and the next frame image respectively; (b) Perform SURF feature point detection on the current frame image and the next frame image respectively to obtain two sets of feature points, point1 and point2; (c) For each point i in points1, find the corresponding point j in points2; (d) For each point k in points2, find the corresponding point l in points1; (e) If point i is matched by point j, and point j is matched by point i, then the match is successful; (f) Use the RANSAC algorithm to remove mismatches in the SURF algorithm matching.

[0040] It is understandable that this invention adopts a bidirectional matching method, which greatly improves the accuracy of algorithm matching.

[0041] It should be noted that this invention can solve the tracking failure problem caused by target occlusion or loss during long-term target tracking of UAVs. The method uses the KCF target tracking algorithm as a framework and introduces a strategy for re-searching and locating the target after occlusion or loss. It uses the peak-to-sidelobe ratio to determine whether the target is occluded or lost. When the target is occluded or lost, a target retrieval and matching strategy is introduced to re-match and select the target. The new sample after selection is input into the KCF algorithm to continue the target tracking task. Compared with other tracking algorithms, this method has good performance, especially when the target is occluded or lost. It further improves the stability and robustness of the tracking in the first rectangular landing area of ​​this invention, meeting the requirements of high robustness and high stability of the system and the urgency of the actual situation for remote emergency precision landing of UAVs. It greatly improves the safety and reliability, system robustness and stability of remote emergency precision landing of UAVs, and has unexpected technical effects.

[0042] S300: Obtain the current coordinates of the drone, determine whether the distance between the current coordinates and the target GPS coordinates is less than or equal to a first preset threshold. If so, control the camera plane to be parallel to the ground, establish a second rectangular landing area of ​​preset size with the target GPS coordinates as the center, use the target tracking algorithm to track the second rectangular landing area, control the drone to keep the center of the second rectangular landing area always in the center position of the camera screen, and then control the drone to reduce the flight altitude.

[0043] Specifically, the method further includes: If the center of the second rectangular landing area shifts relative to the center of the camera view during the drone's descent, stop the drone from descending, adjust the drone so that the center of the second rectangular landing area is in the center of the camera view, and then continue to control the drone to descend until the drone completes the landing.

[0044] Specifically, the method for determining whether the center of the second rectangular landing area has shifted relative to the center of the camera image includes: During the process of the drone descending its flight altitude, it is determined whether the distance between the current coordinates and the target GPS coordinates is less than or equal to a second preset threshold, and based on the determination result, it is determined whether the center of the second rectangular landing area has shifted relative to the center of the camera image. If the distance between the current coordinates and the target GPS coordinates is less than or equal to the second preset threshold, it is determined that the center of the second rectangular landing area has not shifted relative to the center of the camera image. If the distance between the current coordinates and the target GPS coordinates is greater than the second preset threshold, it is determined that the center of the second rectangular landing area has shifted relative to the center of the camera image.

[0045] Specifically, the method further includes: If the distance between the current coordinates and the target GPS coordinates is greater than the first preset threshold, continue to control the movement towards the target GPS coordinates.

[0046] It should be noted here that step S100 includes the following: The control module pre-sets the preset time, the first preset threshold, the second preset threshold, and the preset size.

[0047] It should be noted that the preset time, first preset threshold, second preset threshold, and preset size can be specifically set according to the actual needs of the user according to the present invention. The present invention does not limit the specific values ​​of the preset time, first preset threshold, second preset threshold, and preset size, as long as they are applicable to the remote control drone emergency precision landing method proposed in the present invention.

[0048] It should be noted that the preset time is set to 0.5 to 1 second, preferably 0.75 seconds. The first preset threshold is preferably 1 meter, the second preset threshold is preferably 0.5 meters, and the preset size is preferably 50 pixels in both length and width. These settings were determined by those skilled in the art through extensive testing and effectively implement the remote-controlled drone emergency precision landing method of this invention, further improving its usability.

[0049] The design concept and design process of this invention are further explained below: The patent shown in the authorization announcement number CN117671543B is a technology for converting drone pixels to GPS coordinates.

[0050] (1) Due to the limitations of the algorithm relative to the plane, the GPS position corresponding to the UAV pixel is the GPS coordinate of the pixel relative to the UAV take-off calibration plane. It can be seen that when the pitch angle of the UAV lens gimbal is too large, the GPS will have a large offset. As the UAV lens gradually probes vertically downward, the correspondence between the pixel and the GPS point gradually becomes more accurate.

[0051] (2) According to (1), when the gimbal is too large, the GPS will shift according to the target object picked up by the drone screen, but the target tracking algorithm remains constant. As the drone flies toward the target point, the GPS of the final landing point gradually becomes more accurate according to the pixel-to-GPS mapping conversion.

[0052] (3) When the UAV arrives at the landing point, due to the high-altitude flight characteristics of the UAV, the actual distance corresponding to the selected pixel box is too large, which will cause a large offset during the landing process, and the visual tracking algorithm will fail. At this time, the GPS to pixel mapping algorithm is used to keep the pixel updated with a box size of 50 pixels in length and width, so as to ensure that the UAV can accurately track the landing.

[0053] The implementation steps of this invention are as follows: (1) Manually select the landing location. Select the landing location in the video frame and center it. The drone usually flies at a high altitude and the image pixel is larger than the actual scale. To ensure effective tracking, select a length and width of ≥50 pixels.

[0054] (2) Control the camera to place the selected position in the center of the screen.

[0055] (3) Based on the selected location in the picture and the current GPS location, altitude, gimbal angle and other information of the UAV, perform pixel-to-position mapping calculation to obtain the GPS location information of the center point of the selected picture.

[0056] (4) Start the KCF target tracking algorithm and control the gimbal to keep the selected position generated by the algorithm in the center of the screen.

[0057] (5) The drone flies to the coordinates according to the GPS position information calculated in step (3), and repeats step (4). With the drone taking pictures of about 30 frames per second, after about 10 to 15 frames of video footage, step (3) is recalculated until the drone position is within 1 meter of the calculated GPS position.

[0058] (6) After step (5), the UAV starts the landing procedure. At this time, according to the GPS and pixel mapping algorithm, the coordinate frame is marked by expanding the length and width of the landing center point by ≥50 pixels. The coordinate frame is passed to the KCF target following algorithm for tracking, so as to keep the UAV always near the landing GPS position point.

[0059] (7) Repeat step (6) and continue adjusting the descent until the drone finally lands.

[0060] It is understood that this invention provides a remote-controlled emergency precise landing method, system, and electronic device for drones. When a drone is low on battery and cannot land safely, the operator can find an open area that is difficult for people to reach in the drone's flight video footage, and directly select the location in the video footage. The drone will automatically and quickly fly to that location and land automatically. While using a target tracking algorithm to track the selected location, the gimbal is adjusted to keep the selected location in the center of the camera's image. At the same time, the landing position is updated and returned to the remote platform, realizing remote-controlled emergency precise landing of drones. This makes it convenient for maintenance personnel to quickly and safely locate the landed drone, and greatly improves the intelligence, accuracy, safety, and reliability of remote-controlled drone landing.

[0061] Please see Figure 2 The present invention provides another embodiment, which provides a remote-controlled drone emergency precision landing system, the remote-controlled drone emergency precision landing system comprising: The acquisition module 100 is used to acquire the pixel coordinates of the four vertices of the first rectangular landing area selected by the user in the current frame image; and to acquire the current coordinates of the drone. The control module 200 is used to acquire the current frame image captured by the drone camera; to calculate the GPS coordinates of the center point of the first rectangular landing area, denoted as the target GPS coordinates; to track the first rectangular landing area using a target tracking algorithm, and to control the gimbal angle so that the center of the first rectangular landing area is always located in the center of the camera image, controlling the drone to move closer to the target GPS coordinates. Every preset time interval, it reacquires the current frame image of the drone and the pixel coordinates of the four vertices of the first rectangular landing area, and then recalculates and updates the target GPS coordinates of the center point of the first rectangular landing area; and to determine whether the distance between the current coordinates and the target GPS coordinates is less than or equal to a first preset threshold. If so, it controls the camera plane to be parallel to the ground, establishes a second rectangular landing area of ​​a preset size with the target GPS coordinates as the center, tracks the second rectangular landing area using a target tracking algorithm, controls the drone to keep the center of the second rectangular landing area always located in the center of the camera image, and then controls the drone to decrease its flight altitude.

[0062] It should be noted that this invention provides a remote-controlled emergency precise landing method, system, and electronic device for drones. When a drone is low on battery and unable to land safely, the operator can find an open area that is difficult for people to reach in the drone's flight video footage, and directly select the location in the video frame. The drone will automatically and quickly fly to that location and land automatically. While using a target tracking algorithm to track the selected location, the gimbal is adjusted to keep the selected location in the center of the camera frame. At the same time, the landing position is updated and returned to the remote platform, realizing remote-controlled emergency precise landing of drones. This makes it convenient for maintenance personnel to quickly and safely locate the landed drone, and greatly improves the intelligence, accuracy, safety, and reliability of remote-controlled drone landing.

[0063] In a preferred embodiment, this application also provides an electronic device, the electronic device comprising: The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions that, when executed by the processor, implement the remote-controlled drone emergency precision landing method. The computer device can be broadly categorized as a server, terminal, or any other electronic device with the necessary computing and / or processing capabilities. In one embodiment, the computer device may include a processor, memory, network interface, communication interface, etc., connected via a system bus. The processor of the computer device can be used to provide the necessary computing, processing, and / or control capabilities. The memory of the computer device may include a non-volatile storage medium and internal memory. The non-volatile storage medium may store an operating system, computer programs, etc. The internal memory can provide an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface and communication interface of the computer device can be used to connect and communicate with external devices via a network. When the computer program is executed by the processor, it performs the steps of the method of the present invention.

[0064] This invention can be implemented as a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the steps of the methods of embodiments of the invention to be performed. In one embodiment, the computer program is distributed across multiple network-coupled computer devices or processors, such that the computer program is stored, accessed, and executed in a distributed manner by one or more computer devices or processors. A single method step / operation, or two or more method steps / operations, may be executed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations may be executed by one or more computer devices or processors, and one or more other method steps / operations may be executed by one or more other computer devices or processors. One or more computer devices or processors may execute a single method step / operation, or execute two or more method steps / operations.

[0065] Those skilled in the art will understand that the method steps of this invention can be performed by a computer program instructing related hardware, such as a computer device or processor, to perform the steps of this invention when executed. Depending on the context, any references herein to memory, storage, databases, or other media may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0066] It is understood that this invention provides a remote-controlled emergency precise landing method, system, and electronic device for drones. When a drone is low on battery and cannot land safely, the operator can find an open area that is difficult for people to reach in the drone's flight video footage, and directly select the location in the video footage. The drone will automatically and quickly fly to that location and land automatically. While using a target tracking algorithm to track the selected location, the gimbal is adjusted to keep the selected location in the center of the camera's image. At the same time, the landing position is updated and returned to the remote platform, realizing remote-controlled emergency precise landing of drones. This makes it convenient for maintenance personnel to quickly and safely locate the landed drone, and greatly improves the intelligence, accuracy, safety, and reliability of remote-controlled drone landing.

[0067] The technical features described above can be combined arbitrarily. Although not all possible combinations of these technical features are described, any combination of these technical features should be considered to be covered by this specification, provided that such combination does not contain contradictions.

[0068] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for remotely controlling an emergency precision landing of a drone, characterized in that, The method includes: S100. Obtain the current frame image captured by the drone camera, obtain the pixel coordinates of the four vertices of the first rectangular landing area selected by the user in the current frame image, and then calculate the GPS coordinates of the center point of the first rectangular landing area, which are recorded as the target GPS coordinates. S200: Use a target tracking algorithm to track the first rectangular landing area, and control the gimbal angle to keep the center of the first rectangular landing area always in the center of the camera image. Control the drone to move closer to the target GPS coordinates. At preset intervals, reacquire the current frame image of the drone and the pixel coordinates of the four vertices of the first rectangular landing area, and then recalculate and update the target GPS coordinates of the center point of the first rectangular landing area. S300: Obtain the current coordinates of the drone, determine whether the distance between the current coordinates and the target GPS coordinates is less than or equal to a first preset threshold. If so, control the camera plane to be parallel to the ground, establish a second rectangular landing area of ​​preset size with the target GPS coordinates as the center, use the target tracking algorithm to track the second rectangular landing area, control the drone to keep the center of the second rectangular landing area always in the center of the camera screen, and then control the drone to reduce the flight altitude. The method further includes: If the distance between the current coordinates and the target GPS coordinates is greater than the first preset threshold, continue to control the movement towards the target GPS coordinates; The method further includes: If the center of the second rectangular landing area shifts relative to the center of the camera view during the drone's descent, stop the drone from descending, adjust the drone so that the center of the second rectangular landing area is located in the center of the camera view, and then continue to control the drone to descend until the drone completes the landing. The methods for determining whether the center of the second rectangular landing area has shifted relative to the center of the camera frame include: During the process of the drone descending its flight altitude, it is determined whether the distance between the current coordinates and the target GPS coordinates is less than or equal to a second preset threshold, and based on the determination result, it is determined whether the center of the second rectangular landing area has shifted relative to the center of the camera image. If the distance between the current coordinates and the target GPS coordinates is less than or equal to the second preset threshold, it is determined that the center of the second rectangular landing area has not shifted relative to the center of the camera image. If the distance between the current coordinates and the target GPS coordinates is greater than the second preset threshold, it is determined that the center of the second rectangular landing area has shifted relative to the center of the camera image. The step of tracking the first rectangular landing area using a target tracking algorithm includes: The KCF algorithm is used to track the first rectangular landing area. The peak sidelobe ratio (PSR) is used to determine whether the first rectangular landing area is occluded or lost. After the first rectangular landing area is lost, the target retrieval matching strategy of SURF-RANSAC is used to re-match and select the first rectangular landing area. The re-matching of the first rectangular landing area based on the SURF-RANSAC target retrieval matching strategy includes: After the first rectangular landing region is lost, it is re-searched and located. During the KCF algorithm's tracking of the first rectangular landing region, the tracking status is assessed, and the SURF features of the first rectangular landing region are extracted to generate a template. When the first rectangular landing region is occluded or lost, the SURF features of the next frame are extracted and matched with the template. The first rectangular landing region that best matches the template features is selected, and the tracking model is updated to continue tracking the first rectangular landing region. Two thresholds, T1 and T2, are set, with T1 > T2. When PSR ≥ T1, the tracking of the first rectangular landing region is considered complete. If the first rectangular landing region is unobstructed and not lost, the algorithm continues to track it. When T2 ≤ PSR < T1, it is determined that the first rectangular landing region has begun to enter an occlusion state. The algorithm then extracts the image within the tracking box of the current frame as the template for the first rectangular landing region and extracts the SURF features of the template. When PSR < T2, it is determined that the first rectangular landing region is severely occluded or lost. At this time, the SURF features of the current frame are extracted and matched with the template of the first rectangular landing region. If the match is successful, the first rectangular landing region is selected, the kernel correlation filter algorithm is updated, and the tracking of the first rectangular landing region continues.

2. The method of claim 1, wherein, The re-matching of the first rectangular landing area based on the SURF-RANSAC target retrieval matching strategy also includes: The Random Sampling Consensus Algorithm (RANSAC) and the SURF algorithm are combined to eliminate similar matches and mismatches. Based on the SURF-RANSAC target retrieval matching strategy, the first rectangular landing area is rematched. The matching results are then used to select the new first rectangular landing area, which is then input into the KCF algorithm to continue the tracking task for the first rectangular landing area.

3. The remote-controlled drone emergency precision landing method according to claim 2, characterized in that, The matching process for re-matching the first rectangular landing area based on the SURF-RANSAC target retrieval matching strategy is as follows: (a) Read in the current frame image and the next frame image respectively; (b) Perform SURF feature point detection on the current frame image and the next frame image respectively to obtain two sets of feature points, point1 and point2; (c) For each point i in points1, find the corresponding point j in points2; (d) For each point k in points2, find the corresponding point l in points1; (e) If point i is matched by point j, and point j is matched by point i, then the match is successful; (f) Use the RANSAC algorithm to remove mismatches in the SURF algorithm matching.

4. A remote-controlled emergency precision landing system for unmanned aerial vehicles, characterized in that, The remote-controlled UAV emergency precision landing method according to any one of claims 1-3 includes: The acquisition module is used to acquire the pixel coordinates of the four vertices of the first rectangular landing area selected by the user in the current frame image; it is also used to acquire the current coordinates of the drone. The control module is used to acquire the current frame image captured by the drone camera; to calculate the GPS coordinates of the center point of the first rectangular landing area, denoted as the target GPS coordinates; to track the first rectangular landing area using a target tracking algorithm, and to control the gimbal angle so that the center of the first rectangular landing area is always located in the center of the camera image, controlling the drone to move closer to the target GPS coordinates. At preset time intervals, it reacquires the current frame image of the drone and the pixel coordinates of the four vertices of the first rectangular landing area, and then recalculates and updates the target GPS coordinates of the center point of the first rectangular landing area; and to determine whether the distance between the current coordinates and the target GPS coordinates is less than or equal to a first preset threshold. If so, it controls the camera plane to be parallel to the ground, establishes a second rectangular landing area of ​​a preset size with the target GPS coordinates as the center, tracks the second rectangular landing area using a target tracking algorithm, controls the drone to keep the center of the second rectangular landing area always located in the center of the camera image, and then controls the drone to decrease its flight altitude.

5. An electronic device, characterized in that, include: Memory; The system includes a processor, wherein the memory stores computer-readable instructions that, when executed by the processor, implement the remote-controlled drone emergency precision landing method according to any one of claims 1 to 3.

Images