An unmanned aerial vehicle autonomous take-off and landing guiding system

Abstract

The application discloses an unmanned aerial vehicle autonomous take-off and landing guiding system, which is simple and reasonable in structure, convenient to install and use, and does not need to make large-scale changes to the unmanned aerial vehicle during setting. The data obtained by the camera and the laser ranging radar are simple to process, and the calculation amount is small, so that the calculation ability requirement of the data processing module is not high. Meanwhile, the system is not affected by satellite signals, and does not produce accumulated errors existing in pure inertial navigation, so that the anti-interference performance of the unmanned aerial vehicle autonomous take-off and landing guiding system can be effectively improved.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to an autonomous take-off and landing guidance system for unmanned aerial vehicles (UAVs) based on vision and laser ranging. Background Technology

[0002] The function of the UAV autonomous take-off and landing guidance system is to measure the distance of the UAV from the runway centerline in real time. The flight control system uses this distance as input to adjust the output of the UAV's power system and wing servo system to ensure that the UAV flies or / tails along the runway centerline.

[0003] In the existing technology, the autonomous take-off and landing guidance system for UAVs is mainly based on the combined navigation technology of inertial navigation and satellite navigation. The airborne combined navigation system measures the position of the UAV in real time and compares it with the pre-set runway position to calculate the distance of the UAV from the runway centerline.

[0004] However, in some situations, satellite signals may be blocked, and the airborne integrated navigation system will operate in pure inertial navigation mode. Navigation errors will accumulate continuously, and the distance between the UAV and the runway centerline calculated based on this will deviate from the true value and the error will continue to increase, making it impossible to guide the UAV to land safely and reliably on the runway. Summary of the Invention

[0005] In view of the above problems, the present invention provides an autonomous take-off and landing guidance system for unmanned aerial vehicles (UAVs) to overcome or at least partially solve the above problems.

[0006] This invention provides the following solution:

[0007] An autonomous take-off and landing guidance system for unmanned aerial vehicles (UAVs) includes:

[0008] The system includes a camera, a laser ranging radar, and a data processing module embedded in the camera; both the camera and the laser ranging radar are communicatively connected to the data processing module.

[0009] The optical axis of the camera and the optical axis of the laser ranging radar are both within the same vertical plane as the axis of the UAV.

[0010] The laser ranging radar is mounted below the camera and its optical axis is perpendicular to the axis of the UAV and pointing downwards.

[0011] The data processing module is used to perform the following operations:

[0012] The image information captured by the camera is acquired, and the image information is processed using a deep learning-based target detection method and image processing method to detect and segment the runway edge lines, thereby obtaining a binary image of the runway.

[0013] Obtain the pitch angle of the UAV, and crop or expand the runway binarized image according to the pitch angle so that the object point corresponding to the lower edge of the runway binarized image is directly below the camera;

[0014] Extend the edge of the runway line in the binary image of the runway to the bottom edge of the binary image of the runway;

[0015] Determine the pixel distance from the intersection point of the runway edge and the lower edge of the binary image of the runway to the center line of the image;

[0016] The distance between the camera and the runway centerline is calculated by combining the pixel distance, pixel size, camera focal length, and distance measured by the LiDAR.

[0017] The distance between the UAV and the runway centerline is calculated based on the horizontal distance between the camera's mounting position and the UAV's axis, and the distance between the camera and the runway centerline.

[0018] Preferably, the roll and pitch angles of the camera's coordinate system relative to the UAV's coordinate system are both 0, so that when the UAV's pitch angle is 0, the object point located directly below the camera is at the lower edge of the image.

[0019] Preferably: obtaining the pitch angle of the UAV, and cropping or expanding the runway binarized image based on the pitch angle includes:

[0020] When the pitch angle is indeed upward θ, the lower edge of the binary image of the runway is moved downward by λ pixels;

[0021] When the pitch angle is determined to be downward θ, the lower edge of the binary image of the runway is moved upward by λ pixels;

[0022] When the pitch angle is determined to be 0, the lower edge of the runway binarized image remains unchanged;

[0023] in, f is the focal length, and Q is the pixel size.

[0024] Preferably, the distance between the camera and the runway centerline is calculated by combining the pixel distance, pixel size, camera focal length, and distance measured by the LiDAR, including:

[0025] The formula for calculating the distance between the camera and the runway centerline is expressed by the following equation:

[0026] D(camera, runway centerline) = 0.5 * (W1 - W2) * Q * d / f

[0027] In the formula: W1 and W2 are the pixel distances from the intersection of the lower edges of the runway binarized image to the image center line, Q is the pixel size, f is the camera focal length, and d is the distance measured by the lidar.

[0028] Preferably: the distance between the UAV and the runway centerline is calculated based on the horizontal distance between the camera mounting position and the UAV's axis and the distance between the camera and the runway centerline, including:

[0029] The formula for calculating the distance between the UAV and the runway centerline is expressed by the following formula:

[0030] D(drone, runway centerline) = D(camera, runway centerline) + Δd

[0031] In the formula: Δd is the horizontal distance between the camera mounting position and the drone's axis.

[0032] Preferably, Δd is positive when the camera is on the left side of the camera axis and negative when the camera is on the right side of the camera axis.

[0033] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0034] This application provides an autonomous take-off and landing guidance system for unmanned aerial vehicles (UAVs). The system features a simple and reasonable structure, is easy to install and use, and requires no major modifications to the UAV during setup. The processing of data acquired by the camera and laser ranging radar is simple, with low computational load and minimal requirements on the computing power of the data processing module. Furthermore, this system is unaffected by satellite signals and avoids the accumulated errors present in pure inertial navigation, effectively improving the anti-interference performance of the autonomous take-off and landing guidance system. It is worthy of widespread adoption.

[0035] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0037] Figure 1 This is a connection block diagram of an autonomous take-off and landing guidance system for unmanned aerial vehicles provided in an embodiment of the present invention;

[0038] Figure 2 This is a binarized image of the first runway provided in an embodiment of the present invention;

[0039] Figure 3 This is a binarized image of the second runway provided in an embodiment of the present invention;

[0040] Figure 4This is a binarized image of the third runway provided in an embodiment of the present invention;

[0041] Figure 5 This is the binarized image of the fourth runway provided in this embodiment of the invention;

[0042] Figure 6 This is a schematic diagram of image cropping when the pitch angle is downward θ, provided in an embodiment of the present invention.

[0043] Figure 7 This is an extended image schematic diagram provided by an embodiment of the present invention when the pitch angle is upward θ;

[0044] Figure 8 This is a schematic diagram of the first extended runway edge to the lower edge of the image provided in an embodiment of the present invention;

[0045] Figure 9 This is a schematic diagram of the second extended runway edge to the lower edge of the image provided in an embodiment of the present invention;

[0046] Figure 10 This is a schematic diagram of the third extended runway edge to the lower edge of the image provided in an embodiment of the present invention;

[0047] Figure 11 This is a schematic diagram of the horizontal distance between the runway edge and the camera provided in an embodiment of the present invention.

[0048] In the diagram: Camera 1, Laser ranging radar 2, Data processing module 3. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0050] See Figure 1 This invention provides an autonomous take-off and landing guidance system for unmanned aerial vehicles (UAVs), such as... Figure 1 As shown, the system may include:

[0051] The system includes a camera 1, a laser ranging radar 2, and a data processing module 3 embedded in the camera; both the camera 1 and the laser ranging radar 2 are communicatively connected to the data processing module 3.

[0052] The optical axis of the camera 1 and the optical axis of the laser ranging radar 2 are both within the same vertical plane as the axis of the UAV.

[0053] The laser ranging radar 2 is mounted below the camera 1 and the optical axis of the laser ranging radar 2 is perpendicular to the axis of the UAV and pointing downwards.

[0054] The data processing module 3 is used to perform the following operations:

[0055] The image information captured by the camera 1 is obtained, and the image information is processed using a deep learning-based target detection method and image processing method to detect and segment the runway edge lines, thereby obtaining a binary image of the runway.

[0056] The pitch angle of the UAV is obtained, and the runway binarized image is cropped or expanded according to the pitch angle so that the object point corresponding to the lower edge of the runway binarized image is directly below the camera 1.

[0057] Extend the edge of the runway line in the binary image of the runway to the bottom edge of the binary image of the runway;

[0058] Determine the pixel distance from the intersection point of the runway edge and the lower edge of the binary image of the runway to the center line of the image;

[0059] The distance between camera 1 and the runway centerline is calculated by combining the pixel distance, pixel size, camera 1 focal length, and distance measured by the lidar.

[0060] The distance between the UAV and the runway centerline is calculated based on the horizontal distance between the installation position of camera 1 and the shaft of the UAV, and the distance between camera 1 and the runway centerline.

[0061] The UAV autonomous takeoff and landing guidance system provided in this application uses a camera 1 and a laser ranging radar 2 to acquire image data and accurately measure the UAV's altitude during takeoff and landing. The distance between the UAV and the runway centerline is calculated. This system is unaffected by satellite signals and does not accumulate errors, ensuring accurate distance values. The flight control system then uses this distance as input to adjust the outputs of the UAV's power system and wing servo system, ensuring the UAV flies or / and taxis along the runway centerline.

[0062] In practical applications, the image captured by camera 1 is affected by the pitch angle of the drone, resulting in object points directly below camera 1 not being at the lower edge of the image, which is not conducive to calculation using formulas. To eliminate this problem, this embodiment of the application can provide that the roll angle and pitch angle of the coordinate system of camera 1 relative to the coordinate system of the drone are both 0; so that when the pitch angle of the drone is 0, the object point directly below camera 1 is at the lower edge of the image. When initially setting up camera 1, using the method that the roll angle and pitch angle of the coordinate system of camera 1 are both 0 relative to the coordinate system of the drone can ensure that when the pitch angle is 0, the object point directly below camera 1 is at the lower edge of the image, thereby reducing the amount of calculation.

[0063] Furthermore, obtaining the pitch angle of the UAV, and cropping or expanding the runway binarized image based on the pitch angle includes:

[0064] When the pitch angle is determined to be upward θ, the lower edge of the binary image of the runway is moved downward by λ pixels;

[0065] When the pitch angle is determined to be downward θ, the lower edge of the binary image of the runway is moved upward by λ pixels;

[0066] When the pitch angle is determined to be 0, the lower edge of the runway binarized image remains unchanged;

[0067] in, f is the focal length, and Q is the pixel size.

[0068] By means of the above method, even when the pitch angle of the UAV is not 0, the object point directly below the camera 1 can still be guaranteed to be at the lower edge of the image, so that the distance can be calculated using the formula provided in the embodiments of this application regardless of the pitch angle of the UAV.

[0069] Furthermore, embodiments of this application can provide a method for calculating the distance between camera 1 and the runway centerline by combining the pixel distance, pixel size, camera 1 focal length, and distance measured by LiDAR, including:

[0070] The formula for calculating the distance between camera 1 and the runway centerline is expressed by the following formula:

[0071] D(Camera 1, runway centerline) = 0.5 * (W1 - W2) * Q * d / f

[0072] In the formula: W1 and W2 are the pixel distances from the intersection of the lower edges of the runway binarized image to the image center line, Q is the pixel size, f is the camera focal length, and d is the distance measured by the lidar.

[0073] Furthermore, the distance between the drone and the runway centerline is calculated based on the horizontal distance between the camera 1's mounting position and the drone's axis, and the distance between the camera 1 and the runway centerline.

[0074] The formula for calculating the distance between the UAV and the runway centerline is expressed by the following formula:

[0075] D(UAV, runway centerline) = D(Camera 1, runway centerline) + Δd

[0076] In the formula: Δd is the horizontal distance between the installation position of camera 1 and the axis of the drone.

[0077] Δd is positive when the camera 1 is on the left side of the camera axis, and negative when the camera 1 is on the right side of the camera axis.

[0078] The system provided in the embodiments of this application will be described in detail below.

[0079] The system provided in this application embodiment mainly includes a camera 1 and a laser ranging radar 2. The camera 1 has an embedded data processing module 3, which can process the data of the laser ranging radar 2 (the straight-line distance between the target and the laser ranging radar 2) and the image data of the camera 1 in real time.

[0080] The system installation method provided in this application embodiment is as follows:

[0081] 1. The optical axis of camera 1 and the axis of the drone are in the same vertical plane, and the roll angle and pitch angle of the camera 1 coordinate system relative to the drone coordinate system are both 0. When the drone is flying horizontally, the object point located directly below camera 1 is exactly at the lower edge of the image. In addition, the field of view of camera 1 is not obstructed, and camera 1 can acquire runway images when the drone takes off and lands.

[0082] 2. The optical axis of the laser ranging radar is perpendicular to the axis of the UAV and pointing downwards. When the UAV takes off and lands, the laser radar can accurately measure the height of the UAV above the ground.

[0083] 3. The laser ranging radar 2 should be installed below the camera 1, and the optical axis of the laser ranging radar 2 and the optical axis of the camera 1 should be in the same vertical plane.

[0084] The system provided in this application provides the following specific method for measuring the distance between a UAV and the runway centerline.

[0085] 1. Using deep learning-based object detection and image processing methods, the runway edge lines are detected and segmented to obtain a binary image of the runway, such as... Figure 2 , Figure 3 , Figure 4 , Figure 5 As shown ( Figure 2 , Figure 3 , Figure 4 ,Figure 5 The images shown are the runway binarized images obtained after processing the original images acquired by the UAV under different flight attitudes.

[0086] 2. Crop or expand the image based on the drone's pitch angle: a. When the pitch angle is upward θ, move the bottom edge of the original image downward by λ pixels; b. When the pitch angle is downward θ, move the bottom edge of the original image upward by λ pixels; c. When the pitch angle is 0, the bottom edge of the original image remains unchanged.

[0087] The above three situations f is the focal length, and Q is the pixel size. For example... Figure 6 , Figure 7 The image shown is a schematic diagram of image cropping and image expansion. The bottom edge of the image is not directly below camera 1.

[0088] 3. To Figure 2 , Figure 3 , Figure 4 The runway detection and segmentation results shown are processed by extending the runway edge line to the bottom edge of the image. The result after processing is as follows. Figure 8 , Figure 9 , Figure 10 As shown.

[0089] 4. Measure the pixel distances W1 and W2 from the intersection of the runway edge and the bottom edge of the image to the image center line. Figure 11 As shown. In specific measurements, the length of the lower edge of the image can be obtained first, then the longitudinal coordinates of the midpoint of that length can be determined, and then the longitudinal coordinates of the two intersection points can be obtained. Finally, the pixel distances W1 and W2 between the intersection point of the runway edge and the lower edge of the image and the image centerline can be calculated by subtraction.

[0090] 5. Calculate the distance between camera 1 and the runway centerline. The formula is:

[0091] D(Camera 1, runway centerline) = 0.5 * (W1 - W2) * Q * d / f

[0092] Where Q is the pixel size, f is the camera focal length, and d is the distance measured by the lidar.

[0093] 6. Calculate the distance between the UAV and the runway centerline based on the horizontal distance Δd between the installation position of camera 1 and the UAV axis (Δd is positive when camera 1 is on the left side of the axis, and negative otherwise).

[0094] The calculation formula is:

[0095] D(UAV, runway centerline) = D(Camera 1, runway centerline) + Δd

[0096] In summary, the UAV autonomous takeoff and landing guidance system provided in this application has a simple and reasonable structure, is easy to install and use, and requires no large-scale modifications to the UAV during setup. The processing of data acquired by the camera and laser ranging radar is simple, with low computational load and low requirements for the computing power of the data processing module. Furthermore, this system is unaffected by satellite signals and avoids the accumulated errors present in pure inertial navigation, effectively improving the anti-interference performance of the UAV autonomous takeoff and landing guidance system. It is worthy of widespread adoption.

[0097] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0098] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0099] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0100] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. An autonomous take-off and landing guidance system for unmanned aerial vehicles (UAVs), characterized in that, It includes a camera, a laser ranging radar, and a data processing module embedded in the camera; both the camera and the laser ranging radar are communicatively connected to the data processing module. The optical axis of the camera and the optical axis of the laser ranging radar are both within the same vertical plane as the axis of the UAV. The laser ranging radar is mounted below the camera and its optical axis is perpendicular to the axis of the UAV and pointing downwards. The data processing module is used to perform the following operations: The image information captured by the camera is acquired, and the image information is processed using a deep learning-based target detection method and image processing method to detect and segment the runway edge lines, thereby obtaining a binary image of the runway. Obtain the pitch angle of the UAV, and crop or expand the runway binarized image according to the pitch angle so that the object point corresponding to the lower edge of the runway binarized image is directly below the camera; Extend the edge of the runway line in the binary image of the runway to the bottom edge of the binary image of the runway; Determine the pixel distance from the intersection point of the runway edge and the lower edge of the binary image of the runway to the center line of the image; The distance between the camera and the runway centerline is calculated by combining the pixel distance, pixel size, camera focal length, and distance measured by the LiDAR. The distance between the UAV and the runway centerline is calculated based on the horizontal distance between the camera's mounting position and the UAV's axis, and the distance between the camera and the runway centerline.

2. The UAV autonomous take-off and landing guidance system according to claim 1, characterized in that, The camera's coordinate system has a roll angle and a pitch angle of 0 relative to the UAV's coordinate system, so that when the UAV's pitch angle is 0, the object point located directly below the camera is at the lower edge of the image.

3. The UAV autonomous take-off and landing guidance system according to claim 2, characterized in that, Obtaining the pitch angle of the UAV, and cropping or expanding the runway binarized image based on the pitch angle includes: When the pitch angle is indeed upward θ, the lower edge of the binary image of the runway is moved downward by λ pixels; When the pitch angle is determined to be downward θ, the lower edge of the binary image of the runway is moved upward by λ pixels; When the pitch angle is determined to be 0, the lower edge of the runway binarized image remains unchanged; in, f is the focal length, and Q is the pixel size.

4. The UAV autonomous take-off and landing guidance system according to claim 1, characterized in that, The distance between the camera and the runway centerline is calculated by combining the pixel distance, pixel size, camera focal length, and distance measured by the LiDAR, including: The formula for calculating the distance between the camera and the runway centerline is expressed by the following equation: D(camera, runway centerline) = 0.5 * (W1 - W2) * Q * d / f In the formula: W1 and W2 are the pixel distances from the intersection of the lower edges of the runway binarized image to the image center line, Q is the pixel size, f is the camera focal length, and d is the distance measured by the lidar.

5. The UAV autonomous take-off and landing guidance system according to claim 4, characterized in that, The distance between the drone and the runway centerline is calculated based on the horizontal distance between the camera's mounting position and the drone's axis, and the distance between the camera and the runway centerline. The formula for calculating the distance between the UAV and the runway centerline is expressed by the following formula: D(drone, runway centerline) = D(camera, runway centerline) + Δd In the formula: Δd is the horizontal distance between the camera mounting position and the drone's axis.

6. The UAV autonomous take-off and landing guidance system according to claim 5, characterized in that, Δd is positive when the camera is on the left side of the camera axis, and negative when the camera is on the right side of the camera axis.

Images