Unmanned aerial vehicle auxiliary landing method based on airborne vision

Abstract

The invention relates to the technical field of unmanned aerial vehicle control, in particular to an unmanned aerial vehicle auxiliary landing method based on airborne vision, which comprises the following steps that: firstly, an unmanned aerial vehicle identifies and tracks a cooperative target in a field of view to obtain direction information of the cooperative target and distance information between the unmanned aerial vehicle and the cooperative target; controlling the unmanned aerial vehicle to fly above the cooperative target according to the direction information and the distance information; the unmanned aerial vehicle detects the cooperative target and calculates a vertical distance between the unmanned aerial vehicle and the cooperative target according to a detection result; and controlling the unmanned aerial vehicle to land on the cooperative target according to the vertical distance. According to the image features of the cooperative target, the distance information and the direction information of the cooperative target are fed back in real time by means of target automatic identification and target automatic tracking, and then the distance information and the direction information are provided for the unmanned aerial vehicle for guidance, and autonomous landing of the unmanned aerial vehicle is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle control, and in particular relates to an unmanned aerial vehicle assisted landing method based on airborne vision. Background Art

[0002] With the rapid development of science and technology, drones are playing an increasingly irreplaceable role in the military and civilian fields. In the military field, drones are mainly used for enemy reconnaissance, relay guidance, etc.; in the civilian field, drones are mainly used for square performances, emergency support, etc. Among them, the safe landing of drones is one of the key factors in completing drone flight missions.

[0003] At present, there are three main guidance methods for drone landing: inertial navigation (INS), global positioning navigation (GPS) and navigation based on airborne vision. The inertial navigation method has the disadvantage that the error increases over time. The global positioning navigation method relies on satellite signals, has poor anti-interference ability, and is easily disturbed by external factors during wartime, resulting in positioning failure. In comparison, the navigation method based on airborne vision is a drone landing navigation method that has developed rapidly in recent years. This method has the advantages of safety and reliability, strong anti-interference ability, high navigation accuracy when approaching the target, and a large amount of information. It has attracted widespread attention and research at home and abroad. Summary of the invention

[0004] In view of this, the present invention aims to provide a UAV assisted landing method based on airborne vision. According to the image features of the cooperative target, automatic target recognition and automatic target tracking are utilized to provide real-time feedback of the distance information and direction information of the cooperative target, which is then provided to the UAV for guidance to achieve autonomous landing of the UAV.

[0005] To achieve the above object, the technical solution created by the present invention is implemented as follows: An airborne vision-based UAV assisted landing method, comprising: S1: When the UAV flies to the predetermined airspace, it identifies and tracks the cooperative target in the field of view, and obtains the direction information of the cooperative target and the distance information between the UAV and the cooperative target; S2: According to the direction information and distance information obtained in step S1, the UAV is controlled to fly above the cooperative target; S3: The UAV detects the cooperative target and calculates the vertical distance between the UAV and the cooperative target based on the detection results; S4: According to the vertical distance obtained in step S3, the UAV is controlled to land on the cooperative target.

[0006] Furthermore, the cooperative target includes a black circle, a white circle and a white rectangular block; wherein the diameter of the white circle is smaller than the diameter of the black circle, and the white circle and the black circle have the same center; the white rectangular block is located on one side of the white circle, and the midpoint of the long side of the white rectangle is located on the diameter of the black circle.

[0007] Further, step S1 includes: S11: Control the onboard optoelectronic pod of the UAV to identify the cooperative target and determine the direction information based on the identification result; S12: Control the UAV to fly according to the direction information obtained in step S11, and control the airborne optoelectronic pod to track the cooperative target; during the tracking process, measure the distance between the UAV and the cooperative target in real time.

[0008] Furthermore, in step S11, the YOLOv5 model is used to identify the cooperative targets.

[0009] Furthermore, in step S11: Extracting ROI regions from the identified cooperative targets; Detect the long side in the ROI area, and determine the normal direction of the cooperative target based on the long side and the center of the white circle. The normal direction is the direction information.

[0010] Furthermore, the normal angle range is .

[0011] Furthermore, in step S12, the KCF algorithm is used to track the cooperative target.

[0012] Further, in step S2, when the pitch angle of the airborne optoelectronic pod reaches When the UAV is within the range of the cooperative target, it is determined that the UAV has flown directly above the cooperative target.

[0013] Further, step S3 includes: S31: Perform circle detection on the cooperative target to determine whether the field of view of the airborne optoelectronic pod covers the cooperative target; if the field of view covers the cooperative target, execute step S32; if the field of view cannot cover the cooperative target, execute step S33; S32: Measure the diameter of the black circle imaged in the airborne optoelectronic pod, and calculate the distance between the cooperative target and the airborne optoelectronic pod using the principle of similar triangles according to the focal length of the airborne optoelectronic pod and the actual diameter of the black circle. The distance between the two is the vertical distance. S33: Measure the diameter of the white circle imaged in the airborne optoelectronic pod. According to the focal length of the airborne optoelectronic pod and the actual diameter of the white circle, use the principle of similar triangles to calculate the distance between the cooperative target and the airborne optoelectronic pod. The distance between the two is the vertical distance.

[0014] Furthermore, Hough circle transform is used to perform circle detection on cooperative targets.

[0015] Compared with the prior art, the invention can achieve the following beneficial effects: In the UAV assisted landing method based on airborne vision created by the present invention, by designing a cooperative target, the UAV obtains the image information of the cooperative target during flight, and according to the image features of the cooperative target, uses technical means such as automatic target recognition, automatic target tracking, straight line segment detection, circle detection, etc. to provide real-time feedback of the distance information and direction information of the cooperative target, as well as the azimuth and pitch angle of the airborne optoelectronic pod, and then provides them to the flight control system of the UAV to guide the UAV to complete autonomous landing. The test results show that this method can assist the UAV to achieve safe and autonomous landing with high precision. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings: Figure 1 A flowchart of a UAV assisted landing method based on airborne vision according to an embodiment of the present invention; Figure 2 A schematic diagram of the cooperative target described in the embodiment of the present invention; Figure 3 This is a diagram showing the effect of detecting the long side and determining the normal direction of a white rectangular block according to an embodiment of the present invention; Figure 4 A schematic diagram of the normal range described in the embodiment of the present invention; Figure 5 A schematic diagram of the similar triangle principle described in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the invention more clear, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the invention and do not constitute a limitation of the invention.

[0018] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0019] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0020] like Figure 1 As shown, the drone assisted landing method based on airborne vision described in the embodiment of the present invention includes: S1: When the UAV flies to the predetermined airspace, the UAV identifies and tracks the cooperative target in the field of view, and obtains the direction information of the cooperative target and the distance information between the UAV and the cooperative target.

[0021] It should be noted that the cooperative target is an important component of the UAV assisted landing, which directly affects the accuracy of the distance information and direction information provided to the UAV. During the landing process of the UAV, the field of view of the airborne optoelectronic pod in the UAV is continuously reduced. It should be ensured that the airborne optoelectronic pod can always calculate effective information from the cooperative target during the landing process. Therefore, the present invention designs a nested cooperative target. Figure 2 As shown, it includes a black circle, a white circle and a white rectangular block. The diameter of the white circle is smaller than the diameter of the black circle, and the white circle and the black circle have the same center; the white rectangular block is located on one side of the white circle, and the midpoint of the long side of the white rectangular block is located on the diameter of the black circle.

[0022] In one embodiment, the diameter of the black circle is 4 m, the diameter of the white circle is 0.4 m, the white circle and the black circle are concentric, the long side of the white rectangle is 2.8 m, the wide side is 0.5 m, the white rectangular block is located on one side of the white circle, and the white rectangular block is 0.5 m away from the white circle, and the midpoint of the long side of the white rectangular block is located on the diameter of the black circle.

[0023] In the process of using cooperative targets to assist the landing of UAVs, it is preferred to extract the image features of black circles; when the field of view of the airborne optoelectronic pod cannot cover the entire black circle, the image features of white circles are continuously extracted. The nested cooperative targets can well meet the continuous calculation and output of effective information during the UAV assisted landing process.

[0024] In some embodiments, step S1 comprises: S11: The onboard optoelectronic pod of the UAV is controlled to identify the cooperative target and determine the direction information based on the identification result.

[0025] In one embodiment, the cooperative target is identified using the YOLOv5 model, and YOLOv5 recursively regresses the target category and the target position at the same time. YOLOv5 has a relatively simple structure, a small model size, a fast computing speed, and a high detection accuracy, and is currently widely used in embedded systems with real-time processing requirements. In some embodiments, in step S11, the ROI area is extracted from the identified cooperative target; the long side of the white rectangular block in the ROI area is detected, and the normal of the cooperative target is determined according to the long side and the center of the white circle, and the normal is the direction information.

[0026] In one embodiment, the LSD (Line Segment Detector) algorithm (which can be called in the computer vision library Opencv to implement cv::line_descriptor::LSDDetector) is used to complete the long edge detection of the white rectangular block. The algorithm extracts the straight line segment by the direction and intensity information of the edge pixels, and has high accuracy and high robustness. The detection effect is as follows Figure 3 As shown, from Figure 3 It can be seen that the long side of the white rectangle has been detected and marked with a red line. At this time, the normal direction of the cooperative target is the perpendicular direction from the center of the white circle to the long side of the white rectangle, as shown in Figure 3 The blue line in.

[0027] In some embodiments, the normal angle range is ,like Figure 4 shown.

[0028] S12: Control the UAV to fly according to the direction information obtained in step S11, and control the airborne optoelectronic pod to track the cooperative target; during the tracking process, measure the distance between the UAV and the cooperative target in real time.

[0029] In one embodiment, the cooperative target is tracked using the KCF (Kernel Correlation Filter) method to achieve continuous locking of the cooperative target. The KCF algorithm (which can be called in the computer vision library Opencv to implement cv::TrackerKCF) is a target tracking algorithm based on a kernel correlation filter. It learns the appearance features of the target (i.e., the cooperative target) and uses the kernel correlation filter to locate the target. It has the characteristics of fast computing speed, strong robustness, and high accuracy. During the stable tracking process, the airborne optoelectronic pod measures the distance of the cooperative target in real time through the internal laser rangefinder.

[0030] S2: According to the direction information and distance information obtained in step S1, control the UAV to fly above the cooperative target. In some embodiments, when the pitch angle of the airborne optoelectronic pod reaches When the UAV is within the range of the cooperative target, it is determined that the UAV has flown directly above the cooperative target.

[0031] In some embodiments, the airborne optoelectronic pod provides real-time feedback of the distance and direction information of the cooperative target, as well as the azimuth and pitch angle of the airborne optoelectronic pod itself, and then provides them to the flight control system of the UAV to guide the UAV to gradually fly directly above the cooperative target. When the range is within the range, it is determined that the UAV has flown directly above the cooperative target, and the onboard optoelectronic pod is locked into the vertical downward viewing mode, at which time the UAV is ready to land.

[0032] S3: The UAV detects the cooperative target and calculates the vertical distance between the UAV and the cooperative target based on the detection results.

[0033] In some embodiments, step S3 comprises: S31: Perform circle detection on the cooperative target to determine whether the field of view of the airborne optoelectronic pod covers the cooperative target; if the field of view covers the cooperative target, execute step S32; if the field of view cannot cover the cooperative target, execute step S33.

[0034] In one embodiment, the Hough circle transform is used to detect circles on the cooperative target. Specifically, the circle detection can be implemented by calling the Hough circle transform (cv:: HoughCircles) in the computer vision library Opencv. After detection, the grayscale mean of all pixels in the circle is counted. When the grayscale mean is less than 110, the circle is considered to be a black circle; when the grayscale mean is greater than 210, the circle is considered to be a white circle.

[0035] S32: Measure the diameter of the black circle imaged in the airborne optoelectronic pod. According to the focal length of the airborne optoelectronic pod and the actual diameter of the black circle, use the principle of similar triangles to calculate the distance between the cooperative target and the airborne optoelectronic pod. The distance between the two is the vertical distance.

[0036] Specifically, according to Figure 5 Calculate perpendicular distances using the principle of similar triangles as shown. Figure 5 Where d represents the diameter of the black circle imaged in the airborne optoelectronic pod, which is equal to the product of the number of pixels of the black circle imaged in the airborne optoelectronic pod and the pixel size of the airborne optoelectronic pod. h represents the focal length of the airborne optoelectronic pod. D represents the actual diameter of the black circle. H represents the distance between the cooperative target and the airborne optoelectronic pod, that is, the vertical distance. At this time, the diameter d, the focal length f, the diameter D and the vertical distance H satisfy: .

[0037] S33: Measure the diameter of the white circle imaged in the airborne optoelectronic pod, and calculate the distance between the cooperative target and the airborne optoelectronic pod using the principle of similar triangles according to the focal length of the airborne optoelectronic pod and the actual diameter of the white circle. The distance between the two is the vertical distance. The corresponding specific calculation process is consistent with step S32 and will not be repeated here.

[0038] S4: According to the vertical distance obtained in step S3, the UAV is controlled to land on the cooperative target.

[0039] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0040] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A UAV assisted landing method based on airborne vision, characterized in that: include: S1: When the UAV flies to a predetermined airspace, the UAV identifies and tracks the cooperative target in the field of view, and obtains the direction information of the cooperative target and the distance information between the UAV and the cooperative target; S2: According to the direction information and distance information obtained in step S1, the UAV is controlled to fly above the cooperative target; S3: the UAV detects the cooperative target, and calculates the vertical distance between the UAV and the cooperative target according to the detection result; S4: According to the vertical distance obtained in step S3, the UAV is controlled to land on the cooperative target.

2. The UAV assisted landing method based on airborne vision according to claim 1 is characterized in that: The cooperative targets include black circles, white circles and white rectangular blocks; wherein, The diameter of the white circle is smaller than the diameter of the black circle, and the white circle and the black circle have the same center; The white rectangular block is located on one side of the white circle, and the midpoint of the long side of the white rectangle is located on the diameter of the black circle.

3. The UAV assisted landing method based on airborne vision according to claim 2 is characterized in that: Step S1 includes: S11: Controlling the airborne optoelectronic pod of the UAV to identify the cooperative target, and determining the direction information according to the identification result; S12: Control the UAV to fly according to the direction information obtained in step S11, and control the airborne optoelectronic pod to track the cooperative target; during the tracking process, measure the distance between the UAV and the cooperative target in real time.

4. The UAV assisted landing method based on airborne vision according to claim 3 is characterized in that: In step S11, the cooperative target is identified using the YOLOv5 model.

5. The UAV assisted landing method based on airborne vision according to claim 3 is characterized in that: In step S11: Extracting ROI regions from the identified cooperative targets; The long side in the ROI area is detected, and the normal direction of the cooperative target is determined according to the long side and the center of the white circle, where the normal direction is the direction information.

6. The UAV assisted landing method based on airborne vision according to claim 5 is characterized in that: The angle range of the normal direction is .

7. The UAV assisted landing method based on airborne vision according to claim 3 is characterized in that: In step S12, the cooperative target is tracked using the KCF algorithm.

8. The UAV assisted landing method based on airborne vision according to claim 3 is characterized in that: In step S2, when the pitch angle of the airborne optoelectronic pod reaches When the UAV is within the range of the cooperative target, it is determined that the UAV has flown directly above the cooperative target.

9. The UAV assisted landing method based on airborne vision according to claim 3 is characterized in that: Step S3 includes: S31: Perform circle detection on the cooperative target to determine whether the field of view of the airborne optoelectronic pod covers the cooperative target; if the field of view covers the cooperative target, execute step S32; if the field of view cannot cover the cooperative target, execute step S33; S32: measuring the diameter of the black circle imaged in the airborne optoelectronic pod, and calculating the distance between the cooperative target and the airborne optoelectronic pod using the principle of similar triangles according to the focal length of the airborne optoelectronic pod and the actual diameter of the black circle, where the distance between the two is the vertical distance; S33: Measure the diameter of the white circle imaged in the airborne optoelectronic pod, and calculate the distance between the cooperative target and the airborne optoelectronic pod using the principle of similar triangles according to the focal length of the airborne optoelectronic pod and the actual diameter of the white circle. The distance between the two is the vertical distance.

10. The UAV assisted landing method based on airborne vision according to claim 7, characterized in that: The Hough circle transform is used to perform circle detection on the cooperative target.

Images