Unmanned aerial vehicle assisted landing method based on onboard vision

Abstract

The application relates to the technical field of unmanned aerial vehicle control, in particular to an unmanned aerial vehicle assisted landing method based on airborne vision, which comprises the following steps: firstly, an unmanned aerial vehicle identifies and tracks a cooperative target in a field of view to obtain direction information of the cooperative target and distance information between the unmanned aerial vehicle and the cooperative target; according to the direction information and the distance information, the unmanned aerial vehicle is controlled to fly to above the cooperative target; the unmanned aerial vehicle detects the cooperative target, and calculates a vertical distance between the unmanned aerial vehicle and the cooperative target according to a detection result; and according to the vertical distance, the unmanned aerial vehicle is controlled to land on the cooperative target. According to image features of the cooperative target, target automatic identification and target automatic tracking are utilized to feed back distance information and direction information of the cooperative target in real time, which are then provided to the unmanned aerial vehicle to guide the unmanned aerial vehicle to realize autonomous landing.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of unmanned aerial vehicle control, and particularly relates to an unmanned aerial vehicle assisted landing method based on airborne vision. BACKGROUND

[0002] With the rapid development of science and technology, unmanned aerial vehicles play an increasingly irreplaceable role in military and civilian fields. In the military field, unmanned aerial vehicles are mainly used for enemy reconnaissance, relay guidance, etc.; in the civilian field, unmanned aerial vehicles mainly realize square formation performance, emergency support, etc. Among them, the safe landing of unmanned aerial vehicles is one of the key factors to complete the flight task of unmanned aerial vehicles.

[0003] Currently, unmanned aerial vehicle landing mainly includes three guiding methods: inertial navigation method (INS), global positioning navigation method (GPS) and navigation method based on airborne vision. The inertial navigation method has the disadvantage that the error increases over time. The global positioning navigation method relies on satellite signals and has poor anti-interference ability, and is prone to positioning failure in wartime due to external disturbances. Compared with the above two methods, the navigation method based on airborne vision is a kind of unmanned aerial vehicle landing navigation method developed rapidly in recent years, which has the advantages of safety and reliability, strong anti-interference ability, high navigation precision when approaching the target, and large amount of information, etc., and has attracted widespread attention and research at home and abroad. SUMMARY

[0004] Therefore, the present application aims to provide an unmanned aerial vehicle assisted landing method based on airborne vision, which uses target automatic recognition and target automatic tracking to provide the distance information and direction information of the cooperative target to the unmanned aerial vehicle for guidance, so as to realize the autonomous landing of the unmanned aerial vehicle.

[0005] To achieve the above-mentioned purpose, the technical scheme of the present application is as follows:

[0006] An unmanned aerial vehicle assisted landing method based on airborne vision, comprising:

[0007] S1: When the unmanned aerial vehicle flies to a predetermined airspace, the unmanned aerial vehicle identifies and tracks the cooperative target in the field of view, and obtains the direction information of the cooperative target and the distance information between the unmanned aerial vehicle and the cooperative target;

[0008] S2: According to the direction information and distance information obtained in step S1, the unmanned aerial vehicle is controlled to fly above the cooperative target;

[0009] S3: The unmanned aerial vehicle detects the cooperative target, and calculates the vertical distance between the unmanned aerial vehicle and the cooperative target according to the detection result;

[0010] S4: According to the vertical distance obtained in step S3, the unmanned aerial vehicle is controlled to land on the cooperative target.

[0011] Furthermore, the cooperative target includes a black circle, a white circle and a white rectangular block; wherein, the diameter of the white circle is smaller than the diameter of the black circle, and the white circle and the black circle have the same center; the white rectangular block is located on one side of the white circle, and the midpoint of the long side of the white rectangle is located on the diameter of the black circle.

[0012] Furthermore, step S1 includes:

[0013] S11: Control the UAV's onboard optoelectronic pod to identify the cooperative target and determine the direction information based on the identification results;

[0014] S12: Control the UAV to fly according to the direction information obtained in step S11, and at the same time control the airborne optoelectronic pod to track the cooperative target; during the tracking process, measure the distance between the UAV and the cooperative target in real time.

[0015] Furthermore, in step S11, the YOLOv5 model is used to identify the cooperative targets.

[0016] Furthermore, in step S11:

[0017] Extracting ROI regions from identified cooperative targets;

[0018] Detect the long side in the ROI area and determine the normal direction of the cooperative target based on the long side and the center of the white circle. The normal direction is the direction information.

[0019] Furthermore, the normal angle range is .

[0020] Furthermore, in step S12, the KCF algorithm is used to track the cooperative target.

[0021] Further, in step S2, when the pitch angle of the airborne optoelectronic pod reaches When the UAV reaches the range of the cooperative target, it is determined that the UAV has flown directly above the cooperative target.

[0022] Furthermore, step S3 includes:

[0023] S31: Perform circle detection on the cooperative target to determine whether the field of view of the airborne optoelectronic pod covers the cooperative target; if the field of view covers the cooperative target, execute step S32; if the field of view does not cover the cooperative target, execute step S33;

[0024] S32: Measure the diameter of the black circle imaged in the airborne optoelectronic pod. Based on the focal length of the airborne optoelectronic pod and the actual diameter of the black circle, use the principle of similar triangles to calculate the distance between the cooperative target and the airborne optoelectronic pod. The distance between the two is the vertical distance.

[0025] S33: Measure the diameter of the white circle imaged in the airborne optoelectronic pod. Based on the focal length of the airborne optoelectronic pod and the actual diameter of the white circle, use the principle of similar triangles to calculate the distance between the cooperative target and the airborne optoelectronic pod. The distance between the two is the vertical distance.

[0026] Furthermore, Hough circle transform is used to perform circle detection on cooperative targets.

[0027] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0028] The invention's onboard vision-based drone landing assisted method utilizes a designed cooperative target. The drone acquires image information of the cooperative target during flight. Based on the target's image features, the method utilizes automatic target recognition, automatic target tracking, line segment detection, and circle detection to provide real-time feedback on the target's distance and direction, as well as the azimuth and pitch angles of the onboard optoelectronic pod. This information is then provided to the drone's flight control system to guide the drone through autonomous landing. Experimental results demonstrate that this method can assist drones in achieving safe and highly accurate autonomous landings. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which constitute part of the present invention, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0030] Figure 1 A flow chart of a UAV assisted landing method based on airborne vision according to an embodiment of the present invention;

[0031] Figure 2 A schematic diagram of the cooperative target described in the embodiment of the present invention;

[0032] Figure 3 This is a diagram showing the effect of detecting the long side and determining the normal direction of a white rectangular block according to an embodiment of the present invention;

[0033] Figure 4 A schematic diagram of the normal range described in the embodiment of the present invention;

[0034] Figure 5 Schematic diagram of the similar triangle principle described in the embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0036] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0037] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0038] like Figure 1 As shown, the UAV assisted landing method based on airborne vision described in the embodiment of the present invention includes:

[0039] S1: When the UAV flies to the predetermined airspace, it identifies and tracks the cooperative target in the field of view, and obtains the direction information of the cooperative target and the distance information between the UAV and the cooperative target.

[0040] It should be noted that the cooperative target is an important component of the UAV assisted landing, which directly affects the accuracy of the distance and direction information provided to the UAV. During the landing process of the UAV, the field of view of the onboard optoelectronic pod in the UAV is continuously reduced. It should be ensured that the onboard optoelectronic pod can always calculate effective information from the cooperative target during the landing process. Therefore, the present invention designs a nested cooperative target. Figure 2 As shown, it includes a black circle, a white circle, and a white rectangular block. The diameter of the white circle is smaller than the diameter of the black circle, and the white circle and the black circle have the same center. The white rectangular block is located on one side of the white circle, and the midpoint of the long side of the white rectangular block is located on the diameter of the black circle.

[0041] In one embodiment, the diameter of the black circle is 4 meters, the diameter of the white circle is 0.4 meters, and the white circle and the black circle are concentric. The long side of the white rectangle is 2.8 meters, and the wide side is 0.5 meters. The white rectangular block is located on one side of the white circle, and the white rectangular block is 0.5 meters away from the white circle. The midpoint of the long side of the white rectangular block is located on the diameter of the black circle.

[0042] When using cooperative targets to assist drone landing, the image features of the black circle are preferentially extracted. When the onboard electro-optical pod's field of view cannot cover the entire black circle, the image features of the white circle are extracted. Nested cooperative targets effectively meet the needs of continuously calculating and outputting effective information during the drone's assisted landing process.

[0043] In some embodiments, step S1 includes:

[0044] S11: The onboard optoelectronic pod of the UAV is controlled to identify the cooperative target and determine the direction information based on the identification results.

[0045] In an embodiment, the YOLOv5 model is used to identify the cooperative target, and the YOLOv5 model simultaneously performs regression on the target category and the target position in a recursive manner. The YOLOv5 model has a relatively simple structure, a small model size, a fast calculation processing speed, and a high detection accuracy, and is widely used in embedded systems with real-time processing requirements. In some embodiments, in step S11, an ROI region is extracted from the identified cooperative target; a long side of a white rectangular block in the ROI region is detected, and a normal direction of the cooperative target is determined according to the long side and a center of the white circle, and the normal direction is the direction information.

[0046] In an embodiment, the LSD (Line Segment Detector) algorithm (callable implementation cv: line_descriptor: LSD Detector in the computer vision library Opencv) is used to complete the long side detection of the white rectangular block. The LSD algorithm extracts a straight line segment by using the direction and intensity information of the edge pixels, and has high precision and high robustness. The detection effect is as shown in Figure 3 As can be seen from Figure 3 , the long side of the white rectangular block has been detected and is marked with a red line. At this time, the normal direction of the cooperative target is the direction of the perpendicular line from the center of the white circle to the long side of the white rectangular block, as shown by the blue line in Figure 3 .

[0047] In some embodiments, the angle range of the normal direction is , as shown in Figure 4 .

[0048] S12: Control the unmanned aerial vehicle to fly according to the direction information obtained in step S11, and control the airborne optoelectronic pod to track the cooperative target. In the tracking process, the distance between the unmanned aerial vehicle and the cooperative target is measured in real time.

[0049] In an embodiment, the KCF (Kernel Correlation Filter) method is used to track the cooperative target, so as to achieve continuous locking of the cooperative target. The KCF algorithm (callable implementation cv: TrackerKCF in the computer vision library Opencv) is a target tracking algorithm based on a kernel correlation filter. The KCF algorithm learns the appearance features of the target (i.e., the cooperative target) and uses a kernel correlation filter to locate the target, and has the characteristics of fast operation speed, strong robustness, and high accuracy. In the stable tracking process, the airborne optoelectronic pod measures the distance to the cooperative target in real time by using an internal laser range finder.

[0050] S2: Based on the direction information and distance information obtained in step S1, control the drone to fly above the cooperative target. In some embodiments, when the pitch angle of the airborne optoelectronic pod reaches When the UAV reaches the range of the cooperative target, it is determined that the UAV has flown directly above the cooperative target.

[0051] In some embodiments, the airborne optoelectronic pod provides real-time feedback of the distance and direction information of the cooperative target, as well as the azimuth and pitch angle of the airborne optoelectronic pod itself, and then provides it to the flight control system of the UAV to guide the UAV to gradually fly directly above the cooperative target. When the range is within the range, it is determined that the UAV has flown directly above the cooperative target, and the onboard optoelectronic pod is locked into the vertical downward viewing mode. At this time, the UAV is ready to land.

[0052] S3: The UAV detects the cooperative target and calculates the vertical distance between the UAV and the cooperative target based on the detection results.

[0053] In some embodiments, step S3 includes:

[0054] S31: Perform circle detection on the cooperative target to determine whether the field of view of the airborne optoelectronic pod covers the cooperative target; if the field of view covers the cooperative target, execute step S32; if the field of view cannot cover the cooperative target, execute step S33.

[0055] In one embodiment, circle detection is performed on the cooperative target using the Hough Circle Transform. Specifically, circle detection can be implemented by calling the Hough Circle Transform (cv::HoughCircles) in the computer vision library OpenCV. After detection, the grayscale mean of all pixels within the circle is calculated. If the grayscale mean is less than 110, the circle is considered black; if the grayscale mean is greater than 210, the circle is considered white.

[0056] S32: Measure the diameter of the black circle imaged in the airborne optoelectronic pod. Based on the focal length of the airborne optoelectronic pod and the actual diameter of the black circle, use the principle of similar triangles to calculate the distance between the cooperative target and the airborne optoelectronic pod. The distance between the two is the vertical distance.

[0057] Specifically, according to Figure 5 Calculate perpendicular distances using the principle of similar triangles as shown. Figure 5 Where d represents the diameter of the black circle image in the airborne optoelectronic pod, which is equal to the product of the number of pixels of the black circle image in the airborne optoelectronic pod and the pixel size of the airborne optoelectronic pod. h represents the focal length of the airborne optoelectronic pod. D represents the actual diameter of the black circle. H represents the distance between the cooperative target and the airborne optoelectronic pod, that is, the vertical distance. At this time, the diameter d, focal length f, diameter D, and vertical distance H satisfy:

[0058] .

[0059] S33: Measure the diameter of the white circle imaged by the airborne optoelectronic pod. Based on the focal length of the airborne optoelectronic pod and the actual diameter of the white circle, use the principle of similar triangles to calculate the distance between the cooperative target and the airborne optoelectronic pod. This distance between the two is the vertical distance. The specific calculation process is the same as in step S32 and will not be repeated here.

[0060] S4: According to the vertical distance obtained in step S3, the UAV is controlled to land on the cooperative target.

[0061] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved. This is not limited herein.

[0062] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A UAV assisted landing method based on airborne vision, characterized in that: include: S1: When the UAV flies to a predetermined airspace, the UAV identifies and tracks a cooperative target in the field of view, and obtains direction information of the cooperative target and distance information between the UAV and the cooperative target; The cooperative targets include black circles, white circles and white rectangular blocks; wherein, The diameter of the white circle is smaller than the diameter of the black circle, and the white circle and the black circle have the same center; The white rectangular block is located on one side of the white circle, and the midpoint of the long side of the white rectangle is located on the diameter of the black circle; Step S1 includes: S11: controlling the onboard optoelectronic pod of the UAV to identify the cooperative target, and determining the direction information according to the identification result; In step S11: Extracting ROI regions from identified cooperative targets; Detecting the long side in the ROI area, and determining the normal direction of the cooperative target according to the long side and the center of the white circle, where the normal direction is the direction information; S12: Controlling the UAV to fly according to the direction information obtained in step S11, and controlling the airborne optoelectronic pod to track the cooperative target; during the tracking process, measuring the distance between the UAV and the cooperative target in real time; S2: Controlling the UAV to fly above the cooperative target according to the direction information and distance information obtained in step S1; S3: The UAV detects the cooperative target and calculates the vertical distance between the UAV and the cooperative target according to the detection result; Step S3 includes: S31: performing circle detection on the cooperative target to determine whether the field of view of the airborne optoelectronic pod covers the cooperative target; if the field of view covers the cooperative target, executing step S32; if the field of view does not cover the cooperative target, executing step S33; S32: measuring the diameter of the black circle imaged in the airborne optoelectronic pod, and calculating the distance between the cooperative target and the airborne optoelectronic pod using the principle of similar triangles based on the focal length of the airborne optoelectronic pod and the actual diameter of the black circle. The distance between the two is the vertical distance; S33: measuring the diameter of the white circle imaged in the airborne optoelectronic pod, and calculating the distance between the cooperative target and the airborne optoelectronic pod using the principle of similar triangles based on the focal length of the airborne optoelectronic pod and the actual diameter of the white circle. The distance between the two is the vertical distance; S4: According to the vertical distance obtained in step S3, the UAV is controlled to land on the cooperative target.

2. The UAV assisted landing method based on airborne vision according to claim 1 is characterized in that: In step S11, the cooperative target is identified using the YOLOv5 model.

3. The UAV assisted landing method based on airborne vision according to claim 1 is characterized in that: The angle range of the normal direction is .

4. The UAV assisted landing method based on airborne vision according to claim 1 is characterized in that: In step S12, the cooperative target is tracked using the KCF algorithm.

5. The UAV assisted landing method based on airborne vision according to claim 1 is characterized in that: In step S2, when the pitch angle of the airborne optoelectronic pod reaches When the UAV is within the range of the cooperative target, it is determined that the UAV has flown directly above the cooperative target.

6. The UAV assisted landing method based on airborne vision according to claim 4 is characterized in that: Hough circle transform is used to perform circle detection on the cooperative target.

Images