Method, device, storage medium and electronic device for controlling landing of unmanned aerial vehicle

Abstract

The present disclosure relates to a method, device, storage medium and electronic device for controlling the landing of a drone, the method comprising: obtaining the preset landing position information of the drone; after the drone detects a first preset marker image set on the ground, determining the first position information corresponding to the first preset marker image; when it is determined that the first position information matches the preset landing position information, obtaining the reference position information determined by the drone through the real-time dynamic differential technology RTK; and controlling the drone to land to the position corresponding to the preset landing position information according to the first position information and the reference position information. That is, the present disclosure can control the landing of the drone according to the first position information obtained by identifying the first preset marker image and the reference position information determined by RTK, so that the problem of abnormal landing of the drone caused by errors in marker image recognition can be avoided, thereby improving the success rate of drone landing.

Description

Technical Field

[0001] The present disclosure relates to the technical field of unmanned aerial vehicles, and in particular to a method, device, storage medium and electronic device for controlling the landing of an unmanned aerial vehicle. Background Art

[0002] Drones have become a research hotspot in the aviation field due to their convenience, maneuverability and ease of control, and are widely used in various occasions. Autonomous landing of drones has high requirements for navigation and control accuracy and reliability, and is an important foundation and key technology for drones to achieve autonomous flight.

[0003] In related technologies, autonomous landing of drones can be achieved based on ground beacons. For example, a marker image is placed on the ground, and the drone adjusts its posture by recognizing the marker image, and finally lands at the center of the marker image. However, when recognizing the marker image, deviations may occur due to shadows, occlusions, overexposure, etc. in the image, resulting in abnormal landing of the drone, which makes the success rate of the drone landing low. Summary of the invention

[0004] In order to solve the above problems, the present disclosure provides a method, device, storage medium and electronic device for controlling the landing of a drone.

[0005] In a first aspect, the present disclosure provides a method for controlling a drone to land, the method comprising:

[0006] Get the preset landing position information of the drone;

[0007] After the drone detects a first preset mark image set on the ground, determining first position information corresponding to the first preset mark image;

[0008] When it is determined that the first position information matches the preset landing position information, obtaining reference position information of the UAV determined by RTK (Real-time kinematic, real-time dynamic differential technology);

[0009] The drone is controlled to land at a position corresponding to the preset landing position information according to the first position information and the reference position information.

[0010] Optionally, controlling the drone to land at a position corresponding to the preset landing position information according to the first position information and the reference position information includes:

[0011] Determine the target landing range according to the reference position information;

[0012] When the first position information is within the target landing range, controlling the drone to land at a position corresponding to the preset landing position information according to the first position information;

[0013] When the first position information is outside the target landing range, the UAV is controlled to land at a position corresponding to the preset landing position information according to the reference position information.

[0014] Optionally, the first preset mark image is covered with a plurality of second preset mark images and a plurality of third preset mark images, the second preset mark image is smaller than the first preset mark image, the third preset mark image is smaller than the second preset mark image, and the third preset mark image is covered at a preset position of the first preset mark image, and the preset position is determined according to a position of a camera device of the drone that collects the third preset mark image; controlling the drone to land to a position corresponding to the preset landing position information according to the first position information includes:

[0015] In the process of controlling the landing of the drone according to the first position information, if a target marking image is detected, determining second position information corresponding to the target marking image, where the target marking image includes the second preset marking image or the third preset marking image;

[0016] The drone is controlled to land at a position corresponding to the preset landing position information according to the second position information.

[0017] Optionally, the determining the first position information corresponding to the first preset marked image includes:

[0018] Using the first preset marked image as an input of a pre-trained edge detection model to obtain a bounding rectangle corresponding to the first preset marked image;

[0019] Determining corner point information corresponding to the first preset marked image according to the circumscribed rectangle;

[0020] According to the corner point information, position information corresponding to the first preset mark image is determined.

[0021] Optionally, determining the corner point information corresponding to the first preset marked image according to the circumscribed rectangle includes:

[0022] Inputting the circumscribed rectangle and the first preset marker image into a pre-trained angle determination model to obtain a deviation angle between a central axis of the first preset marker image and a central axis of the circumscribed rectangle;

[0023] Correcting the first preset mark image according to the deviation angle;

[0024] The vertex information of the corrected first preset mark image is used as the corner point information.

[0025] Optionally, determining, according to the corner point information, position information corresponding to the first preset marked image includes:

[0026] Determine, according to the corner point information, a corner point order of the corner points corresponding to the first preset marked image;

[0027] According to the corner point sequence, regression processing is performed on the corner points to obtain target corner point information corresponding to the first preset marked image;

[0028] According to the target corner point information, position information corresponding to the first preset mark image is determined.

[0029] In a second aspect, the present disclosure provides a device for controlling the landing of a drone, the device comprising:

[0030] A preset position information acquisition module is used to obtain the preset landing position information of the drone;

[0031] a position information determination module, configured to determine first position information corresponding to a first preset mark image after the drone detects a first preset mark image set on the ground;

[0032] A reference position information acquisition module, configured to acquire reference position information determined by the UAV through RTK when it is determined that the first position information matches the preset landing position information;

[0033] A landing control module is used to control the UAV to land at a position corresponding to the preset landing position information according to the first position information and the reference position information.

[0034] Optionally, the landing control module is specifically used to:

[0035] Determine the target landing range according to the reference position information;

[0036] When the first position information is within the target landing range, controlling the drone to land at a position corresponding to the preset landing position information according to the first position information;

[0037] When the first position information is outside the target landing range, the UAV is controlled to land at a position corresponding to the preset landing position information according to the reference position information.

[0038] Optionally, the first preset mark image is covered with a plurality of second preset mark images and a plurality of third preset mark images, the second preset mark image is smaller than the first preset mark image, the third preset mark image is smaller than the second preset mark image, and the third preset mark image is covered at a preset position of the first preset mark image, and the preset position is determined according to a position of a camera device of the drone that collects the third preset mark image on the drone; the landing control module is further used to:

[0039] In the process of controlling the landing of the drone according to the first position information, if a target marking image is detected, determining second position information corresponding to the target marking image, where the target marking image includes the second preset marking image or the third preset marking image;

[0040] The drone is controlled to land at a position corresponding to the preset landing position information according to the second position information.

[0041] Optionally, the location information determination module is specifically used to:

[0042] Using the first preset marked image as an input of a pre-trained edge detection model to obtain a bounding rectangle corresponding to the first preset marked image;

[0043] Determining corner point information corresponding to the first preset marked image according to the circumscribed rectangle;

[0044] According to the corner point information, position information corresponding to the first preset mark image is determined.

[0045] Optionally, the location information determination module is further used to:

[0046] Inputting the circumscribed rectangle and the first preset marker image into a pre-trained angle determination model to obtain a deviation angle between a central axis of the first preset marker image and a central axis of the circumscribed rectangle;

[0047] Correcting the first preset mark image according to the deviation angle;

[0048] The vertex information of the corrected first preset mark image is used as the corner point information.

[0049] Optionally, the location information determination module is further used to:

[0050] Determine, according to the corner point information, a corner point order of the corner points corresponding to the first preset marked image;

[0051] According to the corner point sequence, regression processing is performed on the corner points to obtain target corner point information corresponding to the first preset marked image;

[0052] According to the target corner point information, position information corresponding to the first preset mark image is determined.

[0053] In a third aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect of the present disclosure.

[0054] In a fourth aspect, the present disclosure provides an electronic device, including:

[0055] a memory having a computer program stored thereon;

[0056] A processor is used to execute the computer program in the memory to implement the steps of the method described in the first aspect of the present disclosure.

[0057] Through the above technical solution, the preset landing position information of the drone is obtained; after the drone detects the first preset marker image set on the ground, the first position information corresponding to the first preset marker image is determined; when it is determined that the first position information matches the preset landing position information, the reference position information determined by the drone through RTK is obtained; and the drone is controlled to land to the position corresponding to the preset landing position information according to the first position information and the reference position information. In other words, the present disclosure can control the landing of the drone according to the first position information obtained by identifying the first preset marker image and the reference position information determined by RTK, so that the problem of abnormal landing of the drone caused by errors in marker image recognition can be avoided, thereby improving the success rate of drone landing.

[0058] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0060] Figure 1 is a flow chart of a method for controlling a drone to land according to an exemplary embodiment;

[0061] Figure 2 is a schematic diagram showing a first preset mark image according to an exemplary embodiment;

[0062] Figure 3 is a flow chart of another method for controlling the landing of a drone according to an exemplary embodiment;

[0063] Figure 4is a schematic diagram showing a marking image according to an exemplary embodiment;

[0064] Figure 5 is a schematic diagram showing another marking image according to an exemplary embodiment;

[0065] Figure 6 is a structural schematic diagram of a device for controlling the landing of a drone according to an exemplary embodiment;

[0066] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0067] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0068] In the description below, words such as “first” and “second” are only used for the purpose of distinguishing the description and should not be understood as indicating or implying relative importance or order.

[0069] First, the application scenario of the present disclosure is explained. In drone delivery, landing is a crucial link. Whether it can land accurately and safely at the designated location directly determines whether the delivery fulfillment is successful. At present, there are many technical means used by drones in the process of autonomous landing. Typical technical means include wireless positioning technology, GPS (Global Positioning System) / inertial combined navigation, etc. However, these two methods are easily affected by the surrounding environment. In addition, since GPS relies entirely on navigation satellites, there is a problem of signal monopoly. It is suitable for areas with strong outdoor GPS signals, which limits the range of drone landing points. At the same time, the accuracy of GPS navigation is limited. Generally, the accuracy of GPS is 5-10 meters, which cannot meet the needs of precise landing of drones.

[0070] In the related technology, a solution based on ground beacons can be used to guide the autonomous landing of the drone. This method uses a specially designed marker image and a corresponding image recognition algorithm. By locating the corner points of the marker image, the spatial relative pose information of the drone's onboard camera relative to the marker image is solved. The calculated relative pose information is used to select a suitable control law to generate the flight control quantity, and guide the drone to complete the autonomous landing. However, when identifying the marker image, there may be recognition deviations due to shadows, occlusions, overexposure, etc. in the image, resulting in abnormal landing of the drone, which makes the success rate of the drone landing low.

[0071] In order to solve the above-mentioned problems, the present disclosure provides a method, device, storage medium and electronic device for controlling the landing of a drone, which can control the landing of the drone according to the first position information obtained by identifying the first preset marker image and the reference position information determined by RTK. In this way, the problem of abnormal landing of the drone caused by errors in marker image recognition can be avoided, thereby improving the success rate of drone landing.

[0072] The present disclosure is described below in conjunction with specific embodiments.

[0073] Figure 1 is a flow chart of a method for controlling a drone to land according to an exemplary embodiment. Figure 1 As shown, the method includes:

[0074] S101, obtaining the preset landing position information of the UAV.

[0075] It should be noted that, in the area where the drone flies, a first preset marker image can be laid on the ground in advance. In order to prevent the drone from landing at the wrong location, the first preset marker images at different locations in the same flight area are different. The size of the area can be determined according to the range of the drone's flight. For example, the flight area can be an area covered by a circle with a radius of 50 meters and the first preset marker image as the center, or an area covered by a square with a side length of 100 meters and the first preset marker image as the center. The above-mentioned method of determining the flight area is only an example, and the range corresponding to the area can also be determined by other methods, which is not limited in this disclosure. The drone can land at the location where the center point of any first preset marker image is located.

[0076] For example, Figure 2 is a schematic diagram showing a first preset mark image according to an exemplary embodiment. Figure 2 As shown, the image in the dotted area is the first preset mark image, which is a 5*5 Aruco code, each square in the Aruco code represents a bit, and the first preset mark image has a total of 25 bits. The above first preset mark image is only an example, and the first preset mark image can also be other types of images, which is not limited in the present disclosure.

[0077] Wherein, when the preset landing position is the position of the first target preset mark image, the first target preset mark image may be any first preset mark image, and the preset landing position information may be the coding information corresponding to the first target preset mark image.

[0078] S102: After the drone detects a first preset mark image set on the ground, determine first position information corresponding to the first preset mark image.

[0079] In this step, the drone can detect the first preset mark image laid on the ground in real time during flight, and after detecting the first preset mark image, the first position information corresponding to the first preset mark image can be further determined. For example, in the case where the preset landing position information is the coded information corresponding to the first target preset mark image, the first position information can be the coded information corresponding to the detected first preset mark image, and after detecting the first preset mark image, the coded information corresponding to the first preset mark image can be determined.

[0080] S103: When it is determined that the first position information matches the preset landing position information, obtain reference position information of the UAV determined by RTK.

[0081] In this step, after determining the first position information corresponding to the first preset mark image, the first position information and the preset landing position information can be compared. When the first position information matches the preset landing position information, it is determined that the detected first preset mark image is the first target preset mark image corresponding to the position where the drone will land. For example, when the preset landing position information is the coded information corresponding to the first target preset mark image, if the first position information is the same as the preset landing position information, it means that the first position information matches the preset landing position information. When it is determined that the first position information matches the preset landing position information, the reference position information determined by the drone through RTK can be obtained.

[0082] It should be noted that, since there may be buildings, such as high-rise buildings, around the first target preset mark image, the RTK signal may be interfered, and the reference position information cannot be obtained through RTK. Therefore, the reference position information last obtained by the drone can be stored. For example, after each acquisition of the reference position information, the stored reference position information can be updated to the latest acquired reference position information. In the case that the drone fails to obtain the reference position information through RTK, the stored reference position information can be obtained.

[0083] S104: Control the drone to land at a position corresponding to the preset landing position information according to the first position information and the reference position information.

[0084] In this step, after obtaining the first position information and the reference position information, the deviation between the first position information and the reference position information can be determined. When the deviation is small, the posture of the UAV can be adjusted according to the first position information through the method of the prior art, and the UAV can be controlled to land at the position corresponding to the preset landing position information; when the deviation is large, the posture of the UAV can be adjusted according to the reference position information through the method of the prior art, and the UAV can be controlled to land at the position corresponding to the preset landing position information.

[0085] By adopting the above method, the UAV can be controlled to land based on the first position information obtained by identifying the first preset marker image and the reference position information determined by RTK. In this way, the problem of abnormal landing of the UAV caused by errors in marker image recognition can be avoided, thereby improving the success rate of UAV landing.

[0086] Figure 3 is a flow chart of another method for controlling the landing of a drone according to an exemplary embodiment. Figure 3 As shown, the method includes:

[0087] S301, obtaining the preset landing position information of the UAV.

[0088] It should be noted that, in the area where the drone flies, a first preset marker image can be laid on the ground in advance. In order to prevent the drone from landing at the wrong location, the first preset marker images at different locations in the same flight area are different. The size of the flight area can be determined according to the range of the drone's flight. For example, the flight area can be an area covered by a circle with a radius of 50 meters and the first preset marker image as the center, or it can be an area covered by a square with a side length of 100 meters and the first preset marker image as the center. The above-mentioned method of determining the flight area is only an example, and the range corresponding to the flight area can also be determined by other methods. The present disclosure does not limit this. In the case where the first preset marker image is an Aruco code, considering that the first preset marker image includes multiple bits, the number of first preset marker images that can be represented based on the multiple bits is relatively large. For example, for a 5*5 Aruco code, theoretically, a maximum of 2 25 If a label is set for each first preset mark image, there are too many categories of the first preset mark image, resulting in low recognition efficiency. In this case, a category can be set based on each bit, for example, white is 1 and black is 0. In this way, for the 5*5 Aruco code, there are a total of 25 categories. Figure 2Taking the first preset marked image as an example, the category of the first preset marked image is 1010011100111110111100110. In addition, the drone can land at the location where the center point of any first preset marked image is located.

[0089] Wherein, when the preset landing position is the position of the first target preset mark image, the first target preset mark image may be any first preset mark image, and the preset landing position information may be the category corresponding to the first target preset mark image.

[0090] S302: After the drone detects a first preset marker image set on the ground, the first preset marker image is used as an input of a pre-trained edge detection model to obtain a circumscribed rectangle corresponding to the first preset marker image.

[0091] Among them, the edge detection model can include two types, one is a model based on an anchor-based detection method, such as single-stage YOLO, SSD (Single Shot MultiBox Detector), two-stage FasterRCNN, CascadeRCNN, etc., and the other is a model based on an anchor-free detection method, such as DenseBox, CornerNet, FCOS, etc. In addition, considering that the processing power of the drone onboard chip is limited and it is necessary to detect labeled images of different scales, in the feature extraction part of the detection algorithm of the edge detection model, MobileNet V1 / V2 / V3, ShuffleNet, NASNet, etc. can be selected as the backbone network, and FPN can be selected as Neck. In addition to Cross Entropy Loss and Smooth L1 Loss, the loss function of the edge detection model can also use OHEM or Focal Loss to improve the effect of edge detection. The edge detection model can be based on CNN (Convolutional Neural Networks) and trained by the model training method of the prior art, which will not be repeated here. The circumscribed rectangle may be a detection frame formed by four vertices of the first preset mark image.

[0092] S303: Determine corner point information corresponding to the first preset mark image according to the circumscribed rectangle.

[0093] It should be noted that after obtaining the circumscribed rectangle corresponding to the first preset mark image, since the circumscribed rectangle may not completely enclose the first preset mark image, and there is an angle deviation between the first preset mark image and the circumscribed rectangle, in this case, the first preset mark image can be further corrected.

[0094] In this step, after obtaining the circumscribed rectangle corresponding to the first preset marker image, the circumscribed rectangle and the first preset marker image can be input into a pre-trained angle determination model to obtain the deviation angle between the central axis of the first preset marker image and the central axis of the circumscribed rectangle, and the first preset marker image is corrected according to the deviation angle, and the vertex information of the corrected first preset marker image is used as the corner point information.

[0095] Among them, the central axis can be the central axis in the horizontal direction or the central axis in the vertical direction, and the present disclosure does not limit this. In the case where the central axis is the central axis in the vertical direction, the deviation angle between the central axis of the first preset mark image and the central axis of the circumscribed rectangle can be determined in a clockwise direction. For example, if the central axis of the first preset mark image is on the right side of the central axis of the circumscribed rectangle, and the angle between the central axis of the circumscribed rectangle and the central axis of the circumscribed rectangle is 5 degrees, then it can be determined that the deviation angle between the central axis of the first preset mark image and the central axis of the circumscribed rectangle is 5 degrees; if the central axis of the first preset mark image is on the left side of the central axis of the circumscribed rectangle, and the angle between the central axis of the circumscribed rectangle and the central axis of the circumscribed rectangle is 5 degrees, then it can be determined that the deviation angle between the central axis of the first preset mark image and the central axis of the circumscribed rectangle is 355 degrees. The angle determination model can be trained based on CNN through the model training method of the prior art, which will not be repeated here.

[0096] After obtaining the deviation angle between the central axis of the first preset mark image and the central axis of the circumscribed rectangle, the first preset mark image can be rotated according to the deviation angle to correct the first preset mark image. For example, if the deviation angle between the central axis of the first preset mark image and the central axis of the circumscribed rectangle is 5 degrees, the first preset mark image can be rotated 5 degrees to the left around the central axis of the first preset mark image. Afterwards, the vertex information of the corrected first preset mark image can be used as the corner point information.

[0097] S304: Determine position information corresponding to the first preset mark image according to the corner point information.

[0098] Wherein, in the case where the preset landing position information is the category corresponding to the first target preset mark image, the detected position information corresponding to the first preset mark image may also be the category of the first preset mark image.

[0099] In this step, after obtaining the corner point information of the first preset marked image, the corner point order of the corner points corresponding to the first preset marked image can be determined based on the corner point information, and the corner points can be regressed based on the corner point order to obtain the target corner point information corresponding to the first preset marked image, and the position information corresponding to the first preset marked image can be determined based on the target corner point information.

[0100] Among them, the corner point order can be determined according to the order of the first quadrant to the fourth quadrant. For example, the corner point order of the corner points in the first quadrant is 1, the corner point order of the corner points in the second quadrant is 2, the corner point order of the corner points in the third quadrant is 3, and the corner point order of the corner points in the fourth quadrant is 4.

[0101] After determining the order of the corner points corresponding to the first preset mark image, the first preset mark image can be regressed through a pre-trained regression model, and then the first preset mark image after regression can be cut into blocks. For example, the processed first preset mark image can be evenly divided into four blocks, and then the first preset mark image after cutting can be regressed for the second time through the regression model, and the vertex information of the first preset mark image after the second regression is used as the target corner point information corresponding to the first preset mark image. The regression model can be CascadeCNN or other existing models, which is not limited in the present disclosure.

[0102] After obtaining the target corner point information corresponding to the first preset mark image, the four corner points of the first preset mark image can be mapped to the four vertices of the square according to the target corner point information, and the transformation matrix can be generated by the perspective transformation method of the prior art to transform the first preset mark image into the square area. Afterwards, the category corresponding to each bit in the first preset mark image can be obtained through the pre-trained category recognition model, and finally the category corresponding to the first preset mark image can be obtained. The category recognition model can be trained based on CNN through the model training method of the prior art, which will not be repeated here.

[0103] It should be noted that in order to avoid abnormal recognition of the preset marked image, a tracking algorithm can be added to the network model used in the present disclosure to ensure the continuity of the image recognition. The tracking algorithm can be a KCF (Kernel Correlation Filter) algorithm or a Siamese RPN, which is not limited in the present disclosure. For the tracking result, HOG+SVM can be designed as a verifier of the tracker bounding box to determine whether the image recognition result is lost. The specific use method of the tracking algorithm can refer to the implementation method of the prior art, which will not be repeated here.

[0104] S305: When it is determined that the first position information matches the preset landing position information, obtain reference position information of the UAV determined by RTK.

[0105] In this step, after obtaining the first position information, the first position information can be compared with the preset landing position information. For example, if the preset landing position information is the category corresponding to the first target preset mark image, and the first position information is the category corresponding to the detected first preset mark image, then the first position information and the preset landing position information can be compared. When the category corresponding to the first target preset mark image is the same as the category corresponding to the first preset mark image, it can be determined that the first position information matches the preset landing position information.

[0106] S306: Determine the target landing range according to the reference position information.

[0107] In this step, after obtaining the reference position information, the preset landing radius of the drone can be obtained, and the area covered by a circle with the reference position information as the center and the landing radius as the radius is used as the target landing range. The landing radius can be preset according to an empirical value, for example, the landing radius can be 20 meters or 15 meters, which is not limited in the present disclosure.

[0108] S307, determining whether the first position information is within the target landing range. If the first position information is within the target landing range, executing step S308; if the first position information is outside the target landing range, executing step S309.

[0109] S308: Control the drone to land at a position corresponding to the preset landing position information according to the first position information.

[0110] S309: Control the UAV to land at a position corresponding to the preset landing position information according to the reference position information.

[0111] S310: When controlling the landing of the UAV according to the first position information, if a target mark image is detected, second position information corresponding to the target mark image is determined.

[0112] Among them, the target mark image includes the second preset mark image or the third preset mark image, the first preset mark image is covered with multiple second preset mark images and multiple third preset mark images, the second preset mark image is smaller than the first preset mark image, the third preset mark image is smaller than the second preset mark image, and the third preset mark image is covered at a preset position of the first preset mark image. By way of example, the third preset mark image can be a fractal mark, and the preset position can be determined according to the position of a camera device on the drone that collects the third preset mark image.

[0113] In addition, the sizes of the second preset mark image and the third preset mark image can be determined according to the height at which the drone needs to adjust its posture. For example, the drone can scan the second preset mark image at a height of 10 meters from the ground, and the drone can scan the third preset mark image at a height of 1 meter from the ground. The above-mentioned heights at which the second preset mark image and the third preset mark image are scanned are only examples, and the present disclosure does not limit this.

[0114] The second preset marker image may include multiple sizes, so that the drone can adjust its position according to the second preset marker image multiple times during the landing process, thereby improving the landing accuracy of the drone. For example, the size of the second preset marker image may include three types, such as a second preset marker image of type A, a second preset marker image of type B, and a second preset marker image of type C. The second preset marker image of type A is larger than the second preset marker image of type B, and the second preset marker image of type B is larger than the second preset marker image of type C. In this way, during the landing process, the drone can first scan the second preset marker image of type A and obtain the position information corresponding to the second preset marker image of type A. In the process of controlling the landing of the drone according to the position information corresponding to the second preset marker image of type A, the second preset marker image of type B can be scanned. After obtaining the position information corresponding to the second preset marker image of type B, in the process of controlling the landing of the drone according to the position information corresponding to the second preset marker image of type B, the second preset marker image of type C can be scanned, the position information corresponding to the second preset marker image of type C can be obtained, and the landing of the drone can be controlled according to the position information corresponding to the second preset marker image of type C. The type of the second preset marker image is just an example and is not limited in the present disclosure.

[0115] When the collection device of the drone is installed in the center of the drone or in the front position when looking down, Figure 4 is a schematic diagram showing a marking image according to an exemplary embodiment. Figure 4 As shown, the three larger mark images are the second preset mark images, such as Figure 4The image in the dotted line frame No. 1 in FIG is one of the second preset mark images, and the two smaller mark images are the third preset mark images, such as Figure 4 The image in the dotted line box No. 2 is one of the third preset marked images. Figure 5 is a schematic diagram showing another marking image according to an exemplary embodiment. Figure 5 As shown, the six larger mark images are the second preset mark images, such as Figure 5 The image in the dashed line box No. 1 in FIG is one of the second preset marked images, and the ten smaller marked images are the third preset marked images, such as Figure 5 The image in the dashed box No. 2 in FIG. 1 is one of the third preset marked images. Figure 4 The marked image shown, Figure 5 The number of second preset marker images and third preset marker images in the marker image shown is larger, so that different heading angles of the drone can be better covered, and the drone can scan multiple marker images at the same time, providing more corner points for image recognition, which can effectively improve the problem of unstable PnP solution caused by insufficient number of corner points, thereby further improving the success rate of drone landing.

[0116] It should be noted that, when setting the second preset mark image and the third preset mark image, the second preset mark image and the third preset mark image can be set as coordinate information of the location, and the coordinate information can include longitude information and latitude information. For example, when setting the second preset mark image, the longitude information and latitude information of the location of the second preset mark image can be encoded to generate the second preset mark image.

[0117] In this step, based on Figure 4 According to the marker image shown, when the drone is 10 meters away from the ground, it can detect the second preset marker image, obtain the position information corresponding to the second preset marker image, and control the drone to land according to the position information corresponding to the second preset marker image. When the drone lands to 1 meter from the ground according to the position information corresponding to the second preset marker image, it can detect the third preset marker image, obtain the position information corresponding to the third preset marker image, and control the drone to land according to the position information corresponding to the third preset marker image, wherein the method for detecting the second preset marker image and the third preset marker image can refer to the method for detecting the first preset marker image, and the method for obtaining the position information corresponding to the second preset marker image and the position information corresponding to the third preset marker image can refer to the method for obtaining the position information of the first preset marker image, which will not be repeated here.

[0118] S311. Control the drone to land at a position corresponding to the preset landing position information according to the second position information.

[0119] In this step, if the target preset mark image detected by the drone is the second preset mark image, the drone can first control the drone to land according to the position information corresponding to the second preset mark image. During the landing process, if the drone detects the third preset mark image, it can continue to control the drone to land to the position corresponding to the preset landing position information according to the position information corresponding to the third preset mark image.

[0120] By adopting the above method, the UAV can be controlled to land according to the first position information obtained by identifying the first preset marker image and the reference position information determined by RTK. In this way, the problem of abnormal landing of the UAV caused by errors in marker image recognition can be avoided, thereby improving the success rate of UAV landing; further, in the process of controlling the landing of the UAV according to the first preset marker image, the landing of the UAV can continue to be controlled according to the position information corresponding to the second preset marker image and the position information corresponding to the third preset marker image, which can further improve the success rate of the landing of the UAV. In addition, the preset marker image can be detected and identified by a network model, which improves the accuracy of image recognition, thereby further improving the accuracy of the landing of the UAV.

[0121] Figure 6 FIG. 1 is a schematic diagram of a device for controlling the landing of a drone according to an exemplary embodiment. Figure 6 As shown, the device comprises:

[0122] The preset position information acquisition module 601 is used to obtain the preset landing position information of the drone;

[0123] The position information determination module 602 is used to determine the first position information corresponding to the first preset mark image after the drone detects the first preset mark image set on the ground;

[0124] The reference position information acquisition module 603 is used to acquire the reference position information of the UAV determined by the real-time kinematic differential technology RTK when it is determined that the first position information matches the preset landing position information;

[0125] The landing control module 604 is used to control the UAV to land at a position corresponding to the preset landing position information according to the first position information and the reference position information.

[0126] Optionally, the landing control module 604 is specifically configured to:

[0127] Determine the target landing range according to the reference position information;

[0128] When the first position information is within the target landing range, controlling the drone to land at a position corresponding to the preset landing position information according to the first position information;

[0129] When the first position information is outside the target landing range, the UAV is controlled to land at a position corresponding to the preset landing position information according to the reference position information.

[0130] Optionally, a plurality of second preset mark images and a plurality of third preset mark images are covered on the first preset mark image, the second preset mark image is smaller than the first preset mark image, the third preset mark image is smaller than the second preset mark image, and the third preset mark image is covered at a preset position of the first preset mark image, and the preset position is determined according to a position of a camera device of the drone that collects the third preset mark image on the drone; the landing control module 604 is further used to:

[0131] In the process of controlling the landing of the drone according to the first position information, if a target marking image is detected, determining second position information corresponding to the target marking image, where the target marking image includes the second preset marking image or the third preset marking image;

[0132] The drone is controlled to land at a position corresponding to the preset landing position information according to the second position information.

[0133] Optionally, the location information determining module 602 is specifically configured to:

[0134] Using the first preset marked image as an input of a pre-trained edge detection model to obtain a circumscribed rectangle corresponding to the first preset marked image;

[0135] Determining corner point information corresponding to the first preset mark image according to the circumscribed rectangle;

[0136] According to the corner point information, position information corresponding to the first preset mark image is determined.

[0137] Optionally, the location information determining module 602 is further configured to:

[0138] Inputting the circumscribed rectangle and the first preset marker image into a pre-trained angle determination model to obtain a deviation angle between a central axis of the first preset marker image and a central axis of the circumscribed rectangle;

[0139] Correcting the first preset mark image according to the deviation angle;

[0140] The vertex information of the corrected first preset mark image is used as the corner point information.

[0141] Optionally, the location information determining module 602 is further configured to:

[0142] Determine the corner point order of the corner points corresponding to the first preset marked image according to the corner point information;

[0143] According to the corner point sequence, regression processing is performed on the corner point to obtain target corner point information corresponding to the first preset marked image;

[0144] According to the target corner point information, the position information corresponding to the first preset mark image is determined.

[0145] Through the above-mentioned device, the UAV can be controlled to land according to the first position information obtained by identifying the first preset marker image and the reference position information determined by RTK. In this way, the problem of abnormal landing of the UAV caused by errors in marker image recognition can be avoided, thereby improving the success rate of UAV landing.

[0146] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0147] Figure 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. For example, the electronic device 700 may be provided as a server. Figure 7 The electronic device 700 includes a processor 722, which may be one or more, and a memory 732 for storing a computer program executable by the processor 722. The computer program stored in the memory 732 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 722 may be configured to execute the computer program to perform the above-mentioned method of controlling the landing of the drone.

[0148] In addition, the electronic device 700 may further include a power supply component 726 and a communication component 750. The power supply component 726 may be configured to perform power management of the electronic device 700, and the communication component 750 may be configured to implement communication of the electronic device 700, for example, wired or wireless communication. In addition, the electronic device 700 may further include an input / output (I / O) interface 758. The electronic device 700 may operate based on an operating system stored in the memory 732, such as Windows Server 2000. TM , Mac OSX TM , Unix TM , Linux TM etc.

[0149] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by the processor, the steps of the above-mentioned method for controlling the landing of the drone are implemented. For example, the computer-readable storage medium may be the above-mentioned memory 732 including program instructions, and the above-mentioned program instructions may be executed by the processor 722 of the electronic device 700 to complete the above-mentioned method for controlling the landing of the drone.

[0150] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above-mentioned method of controlling the landing of a drone when executed by the programmable device.

[0151] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, the technical solution of the present disclosure can be subjected to a variety of simple modifications, and these simple modifications all belong to the protection scope of the present disclosure. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0152] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for controlling the landing of an unmanned aerial vehicle, characterized in that: The method comprises: Get the preset landing position information of the drone; After the drone detects a first preset mark image set on the ground, determining first position information corresponding to the first preset mark image; When it is determined that the first position information matches the preset landing position information, obtaining reference position information of the UAV determined by real-time kinematic differential technology RTK; Controlling the UAV to land to a position corresponding to the preset landing position information according to the first position information and the reference position information; Wherein, controlling the drone to land at a position corresponding to the preset landing position information according to the first position information and the reference position information includes: Determine the target landing range according to the reference position information; When the first position information is within the target landing range, controlling the drone to land at a position corresponding to the preset landing position information according to the first position information; When the first position information is outside the target landing range, the UAV is controlled to land at a position corresponding to the preset landing position information according to the reference position information.

2. The method according to claim 1, characterized in that The first preset mark image is covered with a plurality of second preset mark images and a plurality of third preset mark images, the second preset mark image is smaller than the first preset mark image, the third preset mark image is smaller than the second preset mark image, and the third preset mark image is covered at a preset position of the first preset mark image, and the preset position is determined according to the position of a camera device of the drone that collects the third preset mark image on the drone; controlling the drone to land to a position corresponding to the preset landing position information according to the first position information includes: In the process of controlling the landing of the drone according to the first position information, if a target marking image is detected, determining second position information corresponding to the target marking image, where the target marking image includes the second preset marking image or the third preset marking image; The drone is controlled to land at a position corresponding to the preset landing position information according to the second position information.

3. The method according to claim 1, characterized in that The determining the first position information corresponding to the first preset mark image includes: Using the first preset marked image as an input of a pre-trained edge detection model to obtain a bounding rectangle corresponding to the first preset marked image; Determining corner point information corresponding to the first preset marked image according to the circumscribed rectangle; According to the corner point information, position information corresponding to the first preset mark image is determined.

4. The method according to claim 3, characterized in that: The determining, according to the circumscribed rectangle, the corner point information corresponding to the first preset marked image comprises: Inputting the circumscribed rectangle and the first preset marker image into a pre-trained angle determination model to obtain a deviation angle between a central axis of the first preset marker image and a central axis of the circumscribed rectangle; Correcting the first preset mark image according to the deviation angle; The vertex information of the corrected first preset mark image is used as the corner point information.

5. The method according to claim 3, characterized in that: The determining, according to the corner point information, the position information corresponding to the first preset marked image comprises: Determine, according to the corner point information, a corner point order of the corner points corresponding to the first preset marked image; According to the corner point sequence, regression processing is performed on the corner points to obtain target corner point information corresponding to the first preset marked image; According to the target corner point information, position information corresponding to the first preset mark image is determined.

6. A device for controlling the landing of a drone, characterized in that: The device comprises: A preset position information acquisition module is used to obtain the preset landing position information of the drone; a position information determination module, configured to determine first position information corresponding to a first preset mark image after the drone detects a first preset mark image set on the ground; A reference position information acquisition module, configured to acquire reference position information of the UAV determined by real-time kinematic differential technology RTK when it is determined that the first position information matches the preset landing position information; A landing control module, used for controlling the UAV to land at a position corresponding to the preset landing position information according to the first position information and the reference position information; Wherein, the landing control module is specifically used for: Determine the target landing range according to the reference position information; When the first position information is within the target landing range, controlling the drone to land at a position corresponding to the preset landing position information according to the first position information; When the first position information is outside the target landing range, the UAV is controlled to land at a position corresponding to the preset landing position information according to the reference position information.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 5 are implemented.

8. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 5.

Images