Pose solving method and navigation method in landing stage of unmanned aerial vehicle

Through the combination of semantic segmentation, linear edge detection and PnP algorithms of vision cameras and vision computers, the attitude perception problem caused by the failure of traditional drone navigation equipment is solved, and the normal flight and landing of drones under special circumstances is realized, reducing costs and equipment complexity, improving safety and anti-interference capabilities.

CN119048585APending Publication Date: 2024-11-29BEIJING BEIHANG TIANYU ZHANGYING UAV TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411148657.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

During the landing process, traditional drones may fail due to inertial navigation equipment and GPS equipment, resulting in the inability to correctly perceive attitude and control, resulting in the risk of yaw and loss of control. The existing visual navigation solutions are complex and costly.

Method used

Using a method based on vision cameras and vision computers, through semantic segmentation, linear edge detection and PnP positioning algorithms, image information is used to estimate the position of the drone and navigation without relying on inertial navigation devices and GPS to simplify the equipment. , reduce costs.

Benefits of technology

It realizes normal flight and landing of drones under special circumstances, improves positioning accuracy and safety, reduces equipment costs, simplifies equipment configuration, and enhances anti-interference capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048585A_ABST
    Figure CN119048585A_ABST
Patent Text Reader

Abstract

The invention relates to a pose solving method and a navigation method of an unmanned aerial vehicle in a landing stage, the pose solving method is based on a hardware device system mainly comprising a visual camera and a visual computer, an image around the unmanned aerial vehicle is acquired through a camera system and transmitted to the visual computer in real time, and the transmitted video image is processed by the visual computer; the method comprises a semantic segmentation link, a linear edge detection algorithm link and a PnP positioning algorithm pose solving link. The navigation method comprises the following steps: 1) hardware connection and debugging; 2) calibrating internal parameters of the camera; 3) training a target runway template; 4) carrying out semantic segmentation link and semantic segmentation algorithm ground testing; 5) a linear edge detection algorithm link and a linear edge detection algorithm test; 6) a PnP positioning algorithm pose solving link and a pose positioning algorithm ground test; 7) carrying out algorithm joint test and pose output; and 8) verifying the precision. According to the invention, the equipment is simplified, the cost is reduced, and the positioning margin and safety of the unmanned aerial vehicle are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a posture obtaining method and a navigation method for an unmanned aerial vehicle (UAV) during a landing phase, and belongs to the technical field of visual navigation. Background Art

[0002] The position and posture of traditional UAVs depends on the avionics equipment carried by the UAV, including inertial navigation equipment, GPS positioning, atmospheric machine and altimeter, etc. When special situations such as power supply abnormality and signal shielding occur during the landing of the UAV, the above equipment is easy to lose the positioning function, causing the UAV to be unable to correctly perceive the body posture and control the normal flight of the UAV, causing the UAV to yaw, lose control, crash, etc. In this context, the use of visual navigation technology can perceive the body posture and provide it to the flight control system to ensure the normal flight of the UAV. Currently, most visual navigation includes multiple visual devices such as laser, radar and infrared and their corresponding navigation algorithms. This type of solution has complex equipment and is expensive. Summary of the invention

[0003] The purpose of the present invention is to provide a method for estimating the posture of a drone based on visual recognition and a method for performing landing navigation of a drone using the method. Without using an inertial navigation device and a GPS device, after knowing the airport runway information, the posture of the drone can be obtained through camera visual acquisition and algorithm calculation to meet the navigation requirements of normal flight and landing. Compared with the traditional navigation and positioning method, the equipment is simplified, the cost is reduced, and the positioning margin and safety of the drone are improved.

[0004] The present invention is achieved by the following technical solutions:

[0005] A method for obtaining the posture of a UAV during the landing phase is based on a hardware device system mainly composed of a visual camera and a visual computer. The camera system collects images around the UAV and transmits them to the visual computer in real time. The visual computer processes the transmitted video images and calculates the position and posture of the camera at this time. Specifically, it includes three steps:

[0006] 1) Semantic segmentation: Based on deep learning of convolutional neural network (CNN), a large amount of labeled airport runway image data is learned in advance, so that the algorithm has the ability to recognize the runway in the image in real time and segment and display it. During the navigation process, the image information of the airport runway taken by the camera can identify the information of the pixel points including edge, texture, and color. Then, the context information is fused by combining the local and global information of the image, and the fused pixel information is classified and regressed. Finally, the runway in the image is accurately identified and the processed information is passed to the straight edge detection link.

[0007] 2) Straight line edge detection algorithm: The straight line edge detection algorithm receives the airport runway image that has been semantically segmented and annotated, and grayscales the image. Figure 2 Value processing, detect the straight line features of the airport runway through Hough line detection, and after classifying the straight lines, pass the straight line feature parameters to the PnP positioning algorithm posture link;

[0008] 3) PnP positioning algorithm posture determination link: According to the different tasks and function settings, the straight line characteristics of the airport runway transmitted by the straight line edge detection algorithm link, and the difficulty of identifying the target feature signs, different positioning algorithms are selected to calculate the attitude information of the aircraft during the landing phase.

[0009] The straight line parameters described in step 2) refer to the slope k and intercept b of the straight line. First, the image is preprocessed through Gaussian filtering, Canny operator edge extraction, binarization processing and its adaptive parameter adjustment, and morphological processing steps to provide high-quality input for subsequent probabilistic Hough line extraction.

[0010] The positioning algorithm described in step 3) includes algorithm 1 based on the left edge line, right edge line and horizon of the airport runway and algorithm 2 based on the left edge line, right edge line and start line of the runway; during the visual navigation landing process, when the horizon in the camera scene is clear and far away from the start line of the airport runway, algorithm 1 based on the line features of the horizon and the left and right edge lines of the runway is used. First, the roll angle and pitch angle of the UAV flight are calculated based on the identified horizon features, the yaw angle of the camera is calculated based on the edge line features and the focal length f and pixel size d of the camera, and then the camera's posture is calculated by the principle of similar triangles in space; if the horizon in the camera scene is unclear or the start line feature of the runway is obvious, algorithm 2 is selected to position the camera at this time, and the visual positioning system identifies and segments the features of the left edge line, right edge line and start line of the runway and calculates the yaw angle, pitch angle and roll angle of the camera, and calculates the camera's posture through the rotation matrix formula.

[0011] The above two positioning algorithms output six parameters, namely, the longitudinal distance, lateral distance, and height distance of the camera in the world coordinate system, and the roll angle, yaw angle, and pitch angle of the camera in the camera coordinate system.

[0012] The straight line features of an airport runway include the edge line and horizon features of the runway.

[0013] A navigation method for a drone during landing phase, comprising the following steps:

[0014] 1) Hardware connection and debugging: Connect the camera and the visual computer to the 28V onboard DC power supply, and connect the flight control computer, the visual computer, and the camera through data cables. The visual computer and the flight control computer are connected through the RS422 interface. When the system is powered on, the communication system will perform a ground equipment self-check, and the two computers will perform a communication self-test. After the communication connection is successful, the data communication is normal on the ground station software interface;

[0015] 2) Camera intrinsic calibration: Test the camera intrinsic parameter data by taking grid pictures with the camera, including focal length f, unit pixel area d, and number of pixels. Input the relevant camera data into the algorithm of the visual computer through the ground station software.

[0016] 3) Target runway template training: A large number of training set images are collected for the target runway, and the training set is labeled and trained to enable the semantic segmentation algorithm to accurately identify the target runway;

[0017] 4) Semantic segmentation link and semantic segmentation algorithm ground test: The algorithm of this link is tested on the ground using images of airport runways taken by cameras. The trained semantic segmentation algorithm can identify the edge, texture, and color information of pixels in the image, and then fuse the context information by combining the local and global information of the image. The fused pixel information is classified and regressed, and finally the runway in the image is accurately identified and the processed information is passed to the straight line edge detection link;

[0018] Input the test pictures taken by the test camera separately and test the intersection-and-union ratio output by the semantic segmentation algorithm. When the intersection-and-union ratio reaches the index requirement of the corresponding flight mission, it means that the link can work normally.

[0019] 5) Line edge detection algorithm link and line edge detection algorithm test: Based on the image after semantic segmentation, the line parameters of the left edge line, right edge line and start line of the runway and the horizon are extracted. The line parameters refer to the slope k and intercept b of the line. First, the image is preprocessed through Gaussian filtering, Canny operator edge extraction, binarization processing and its adaptive parameter adjustment, and morphological processing steps to provide high-quality input for subsequent probabilistic Hough line extraction. The probabilistic Hough line extraction algorithm first identifies potential straight line segments in the image, and then uses probability distribution to evaluate the possibility of the existence of the line. After extracting all the straight lines contained in the runway image, they are fused and classified. After the lines are clustered and fused by the K-Means clustering algorithm, the runway edge line and the horizon are classified according to their line parameters, and the line feature parameters are transmitted to the PnP positioning algorithm posture link;

[0020] There is no separate measurement indicator for the ground test part of this link. If the subsequent posture positioning algorithm can output the positioning parameters normally, it means that this link can output the straight line eigenvalue normally;

[0021] 6) PnP positioning algorithm posture acquisition link and posture positioning algorithm ground test: According to the different tasks and function settings, the straight line characteristics of the airport runway transmitted by the straight line edge detection algorithm link, and the difficulty of identifying the target feature mark, different positioning algorithms are selected: Algorithm 1 based on the left edge line, right edge line and horizon of the airport runway, and Algorithm 2 based on the left edge line, right edge line and start line of the runway; During the visual navigation landing process, when the horizon in the camera scene is clear and the distance to the start line of the airport runway is far, the line composed of the horizon and the left and right edge lines of the runway is used. The first algorithm of the feature first calculates the roll angle and pitch angle of the UAV flight according to the recognized horizon feature, and calculates the camera's yaw angle according to the edge line feature and the camera's focal length f and pixel size d, and then calculates the camera's posture through the principle of similar triangles in space; if the horizon in the camera scene is not clear or the start line feature of the runway is obvious, then select algorithm 2 to locate the camera. The visual positioning system identifies and segments the features of the left edge line, right edge line and start line of the runway and calculates the camera's yaw angle, pitch angle and roll angle, and calculates the camera's posture through the rotation matrix formula;

[0022] The above two positioning algorithms output six parameters, namely, the longitudinal distance, lateral distance, and height distance of the camera in the world coordinate system, and the roll angle, yaw angle, and pitch angle of the camera in the camera coordinate system. When the system is working, a set of straight line feature data output by the straight line edge detection algorithm link is input separately. If the six sets of parameter indicators output by this link meet the accuracy required for the current flight, it means that the posture positioning algorithm is working normally.

[0023] 7) Algorithm joint test and posture output: Test the connectivity of the algorithms in the above three links and the connectivity of the camera and visual computer communication. The camera captures image information, displays the camera's posture information in real time and transmits it to the flight control computer, indicating that the system software and hardware are working properly;

[0024] 8) Accuracy verification: After test flight, the algorithm meets the navigation accuracy requirements in certain scenarios.

[0025] This invention aims at the situation that the drone's inertial navigation and atmospheric altimeter cannot be used normally in some special scenes. The drone can navigate and fly along the normal route and land safely. Compared with the use of inertial navigation equipment and GPS navigation, the cost is lower and the anti-interference ability is stronger; compared with the visual navigation using the traditional SLAM (Simultaneous Localization and Mapping) algorithm, there is no need to use laser, radar and other equipment, and the equipment is simpler to use.

[0026] This invention has a certain promoting effect on the combination of novel artificial intelligence visual recognition field and traditional UAV navigation and flight control field. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a structural flow chart of the visual navigation system of the present invention;

[0028] Figure 2 It is a flow chart of the positioning algorithm of the present invention;

[0029] Figure 3 is a comparison diagram between the visual navigation system of the present invention and the traditional navigation system;

[0030] Figure 4 is an input and output information diagram of the edge detection module of the present invention;

[0031] Figure 5 It is a flow chart of the navigation method of the present invention. DETAILED DESCRIPTION

[0032] The implementation of the pose obtaining method of the present invention is based on a hardware device system mainly composed of a visual camera and a visual computer. The camera system collects images around the drone and transmits them to the visual computer in real time. The visual computer processes the transmitted video images and calculates the position and pose of the camera at this time. This joint pose obtaining method combines a variety of visual recognition technologies and combines the PnP (Perspective-n-Point) algorithm. It is different from the traditional positioning solution in that it does not use traditional SLAM (Simultaneous Localization and Mapping) point cloud scanning and laser ranging scanning equipment. It combines a variety of visual recognition technologies to achieve pose estimation and positioning completely based on image information and replace traditional positioning equipment such as inertial navigation equipment and atmospheric machines. The structure flow of the visual navigation system is as follows: Figure 1 shown.

[0033] The implementation of the algorithm layer of visual navigation technology is mainly divided into three steps: semantic segmentation, straight line edge detection and PnP (Perspective-n-Point) algorithm pose calculation.

[0034] Semantic segmentation: This link is mainly based on the deep learning of Convolutional Neural Networks (CNN). By learning a large amount of labeled airport runway image data in advance, the algorithm has the ability to recognize the runway in the image in real time and segment it for display. During the navigation process, the algorithm can identify the edge, texture, color and other information of the pixel points through the image information of the airport runway taken by the camera, and then fuse the context information by combining the local and global information of the image, and classify and regress the fused pixel information, and finally accurately identify the runway in the image and pass the processed information to the straight edge detection link.

[0035] Line edge detection: The line edge detection algorithm receives the airport runway image identified and annotated by semantic segmentation and grayscales the image. Figure 2 The straight line features of the airport runway are detected through Hough line detection, and the straight lines are classified and the straight line feature parameters are passed to the PnP positioning algorithm.

[0036] PnP (Perspective-n-Point) algorithm pose acquisition: The PnP algorithm selects different positioning algorithms based on different task and function settings, as well as the difficulty of identifying target feature marks, and based on features such as the edge line and horizon of the airport runway.

[0037] After going through the above steps, the visual navigation system has realized the function of completely identifying the target by image and positioning itself. By transmitting data with the flight control computer of the UAV, the UAV can achieve normal flight and landing without relying on inertial navigation, atmospheric machine and other attitude equipment. The above positioning algorithm process and functions are as follows Figure 2 shown.

[0038] The schematic diagram of the innovative features of the visual navigation system of the present invention is as follows: Figure 3 The system replaces the traditional inertial navigation equipment and measurement sensors, and only uses visual cameras and visual computers. The various algorithms inside the visual computer jointly replace the functions of auxiliary sensor equipment such as lasers and radars that are different from other visual systems.

[0039] The basic parameters required by the visual navigation algorithm include the intercept and slope of the edge line of the runway, the horizon and other straight line features in the captured image coordinate system, as well as the internal parameters of the camera such as focal length, unit pixel area, and number of pixels. After the positioning calculation by the visual navigation computer, the output camera pose includes six parameters: the longitudinal distance, lateral distance, height distance of the camera in the world coordinate system, and the roll angle, yaw angle, and pitch angle of the camera in the camera coordinate system. Since the camera is rigidly connected to the drone body, the output parameters of the visual computer are the current pose parameters of the drone in the world coordinate system.

[0040] The specific implementation process of this technical solution is mainly as follows:

[0041] 1. Hardware connection and debugging: Connect the camera and vision computer to the 28V onboard DC power supply, and connect the flight control computer, vision computer and camera through data cables. The vision computer and flight control computer are connected through the RS422 interface. When the system is powered on, the communication system will perform a ground equipment self-check, and the two computers will perform a communication self-test. After the communication connection is successful, the data communication is normal on the ground station software interface.

[0042] 2. Camera intrinsic calibration: The camera intrinsic parameter data (focal length f, unit pixel area d, number of pixels, etc.) is tested by using the camera to take grid pictures, and the relevant camera data is input into the algorithm of the visual computer through the ground station software.

[0043] 3. Target runway template training: The prerequisite for the implementation of the semantic segmentation algorithm is to collect a large number of training set images of the cooperative target runway, and to annotate and train the training set so that the algorithm has the ability to accurately identify the target runway.

[0044] 4. Semantic segmentation algorithm ground test: The image of the airport runway taken by the camera is used to test the algorithm of this link on the ground. The trained semantic segmentation algorithm can identify the edge, texture, color and other information of the pixel points in the image, and then fuse the context information by combining the local and global information of the image, and classify and regress the fused pixel information, and finally accurately identify the runway in the image and pass the processed information to the straight edge detection link. The test image taken by the test camera is input separately to test the intersection and union ratio of the output of the semantic segmentation algorithm (the overlapped part of the airport runway area of ​​the original image and the airport runway area identified by the semantic segmentation link is the intersection, and the non-overlapping part is the union. The larger the intersection and union ratio parameter, the higher the recognition accuracy). When the intersection and union ratio meets the index requirements of the corresponding flight mission, it means that this link can work normally.

[0045] 5. Ground test of line edge detection algorithm: The core function of the edge detection module is to extract the line parameters of the three edge lines of the runway (the left edge line, the right edge line and the start line of the runway) and the horizon based on the semantically segmented image. The line parameters refer to the slope k and intercept b of the line. This module first pre-processes the image through Gaussian filtering, Canny operator edge extraction, binarization processing and its adaptive parameter adjustment, morphological processing and other steps to provide high-quality input for subsequent line extraction.

[0046] Probabilistic Hough line extraction is a key part of runway edge extraction. Probabilistic Hough line detection is an efficient line extraction method. The algorithm first identifies potential straight line segments in the image and then uses probability distribution to evaluate the possibility of the existence of the line. By randomly selecting edge point pairs and specific thresholds, probabilistic Hough line detection can effectively reduce false detection and noise interference while maintaining high accuracy in identifying straight lines. After extracting all the straight lines contained in the runway image, they need to be fused and classified. After clustering and fusion of the lines using the K-Means clustering algorithm, the runway edge line and horizon are classified according to their line parameters.

[0047] like Figure 4 As shown in the figure, the input information of the edge detection module is an RGB image, in which the runway part is displayed normally and the rest of the area is represented by black pixel blocks. The output information is the specific category information and line parameters of the line. There is no separate measurement indicator for the ground test part of this link. If the subsequent posture positioning algorithm can output the positioning parameters normally, it means that this link can output the line feature value normally.

[0048] 6. Ground test of posture positioning algorithm: The patent contains different posture positioning algorithms, among which different positioning algorithms can be selected according to different tasks and function settings and the difficulty of identifying target feature marks: Algorithm 1 based on the left edge line, right edge line and horizon of the airport runway, and Algorithm 2 based on the left edge line, right edge line and start line of the airport runway. The positioning flow chart of the invention is as follows: Figure 5 shown.

[0049] During the visual navigation landing process, when the horizon in the camera scene is clear and the distance from the starting line of the airport runway is far, Algorithm 1 based on the line features of the horizon and the left and right edge lines of the runway is used. First, the roll angle and pitch angle of the UAV flight are calculated based on the recognized horizon features, and the yaw angle of the camera is calculated based on the edge line features and the focal length f and pixel size d of the camera. Then, the camera's posture is calculated based on the principle of similar triangles in space. If the horizon in the camera scene is not clear or the starting line features of the runway are obvious, Algorithm 2 is selected to position the camera. The visual positioning system identifies and segments the features of the left edge line, right edge line and starting line of the runway and calculates the yaw angle, pitch angle and roll angle of the camera, and calculates the camera's posture through the rotation matrix formula.

[0050] The above two positioning algorithms output six parameters, namely, the longitudinal distance, lateral distance, and height distance of the camera in the world coordinate system, and the roll angle, yaw angle, and pitch angle of the camera in the camera coordinate system. When the system is working, a set of straight line feature data output by the test edge detection link is input separately. If the six sets of parameter indicators output by this link meet the accuracy required for the current flight, it means that the posture positioning algorithm is working properly.

[0051] 7. Algorithm joint testing and posture output: The main purpose is to test the connectivity of the algorithms in the above three links and the connectivity of the camera and visual computer communication. The camera captures image information, displays the camera's posture information in real time and transmits it to the flight control computer, indicating that the system software and hardware are working normally.

[0052] 8. Accuracy verification: After test flight, the algorithm meets the navigation accuracy requirements in certain scenarios.

Claims

1. A method for obtaining the posture of a UAV during landing, characterized by: Based on a hardware device system mainly composed of visual cameras and visual computers, the camera system collects images around the drone and transmits them to the visual computer in real time. The visual computer processes the transmitted video images and calculates the position and posture of the camera at this time. Specifically, it includes three steps: 1) Semantic segmentation: Based on deep learning of convolutional neural network (CNN), a large amount of labeled airport runway image data is learned in advance, so that the algorithm has the ability to recognize the runway in the image in real time and segment and display it. During the navigation process, the image information of the airport runway taken by the camera can identify the information of the pixel points including edge, texture, and color. Then, the context information is fused by combining the local and global information of the image, and the fused pixel information is classified and regressed. Finally, the runway in the image is accurately identified and the processed information is passed to the straight edge detection link. 2) Line edge detection algorithm: The line edge detection algorithm receives the airport runway image identified and annotated by semantic segmentation, performs grayscale binarization on the image, detects the straight line features of the airport runway through Hough line detection, and passes the straight line feature parameters to the PnP positioning algorithm posture link after classifying the straight lines; 3) PnP positioning algorithm posture determination link: According to the different tasks and function settings, the straight line characteristics of the airport runway transmitted by the straight line edge detection algorithm link, and the difficulty of identifying the target feature signs, different positioning algorithms are selected to calculate the attitude information of the aircraft during the landing phase.

2. The method for obtaining the posture of an unmanned aerial vehicle during the landing phase according to claim 1, characterized in that: The straight line parameters described in step 2) refer to the slope k and intercept b of the straight line. First, the image is preprocessed through Gaussian filtering, Canny operator edge extraction, binarization processing and its adaptive parameter adjustment, and morphological processing steps to provide high-quality input for subsequent probabilistic Hough line extraction.

3. The method for obtaining the posture of an unmanned aerial vehicle during landing according to claim 1, characterized in that: The positioning algorithm described in step 3) includes algorithm 1 based on the left edge line, right edge line and horizon of the airport runway and algorithm 2 based on the left edge line, right edge line and start line of the runway; during the visual navigation landing process, when the horizon in the camera scene is clear and far away from the start line of the airport runway, algorithm 1 based on the line features of the horizon and the left and right edge lines of the runway is used. First, the roll angle and pitch angle of the UAV flight are calculated based on the identified horizon features, the yaw angle of the camera is calculated based on the edge line features and the focal length f and pixel size d of the camera, and then the camera's posture is calculated by the principle of similar triangles in space; if the horizon in the camera scene is unclear or the start line feature of the runway is obvious, algorithm 2 is selected to position the camera at this time, and the visual positioning system identifies and segments the features of the left edge line, right edge line and start line of the runway and calculates the yaw angle, pitch angle and roll angle of the camera, and calculates the camera's posture through the rotation matrix formula.

4. The method for obtaining the posture of an unmanned aerial vehicle during landing according to claim 3, characterized in that: The two positioning algorithms output six parameters, namely, the longitudinal distance, lateral distance, and height distance of the camera in the world coordinate system, and the camera roll angle, yaw angle, and pitch angle in the camera coordinate system.

5. A navigation method for a drone during landing, characterized in that: The following steps are involved: 1) Hardware connection and debugging: Connect the camera and the visual computer to the 28V onboard DC power supply, and connect the flight control computer, the visual computer, and the camera through data cables. The visual computer and the flight control computer are connected through the RS422 interface. When the system is powered on, the communication system will perform a ground equipment self-check, and the two computers will perform a communication self-test. After the communication connection is successful, the data communication is normal on the ground station software interface; 2) Camera intrinsic calibration: Test the camera intrinsic parameter data by taking grid pictures with the camera, including focal length f, unit pixel area d, and number of pixels. Input the relevant camera data into the algorithm of the visual computer through the ground station software. 3) Target runway template training: A large number of training set images of the target runway are collected, and the training set is labeled and trained to enable the semantic segmentation algorithm to accurately identify the target runway; 4) Semantic segmentation link and semantic segmentation algorithm ground test: The algorithm of this link is tested on the ground using images of airport runways taken by cameras. The trained semantic segmentation algorithm can identify the edge, texture, and color information of pixels in the image, and then fuse the context information by combining the local and global information of the image. The fused pixel information is classified and regressed, and finally the runway in the image is accurately identified and the processed information is passed to the straight line edge detection link; Input the test pictures taken by the test camera separately and test the intersection-and-union ratio output by the semantic segmentation algorithm. When the intersection-and-union ratio reaches the index requirement of the corresponding flight mission, it means that the link can work normally. 5) Line edge detection algorithm link and line edge detection algorithm test: Based on the image after semantic segmentation, the line parameters of the left edge line, right edge line and start line of the runway and the horizon are extracted. The line parameters refer to the slope k and intercept b of the line. First, the image is preprocessed through Gaussian filtering, Canny operator edge extraction, binarization processing and its adaptive parameter adjustment, and morphological processing steps to provide high-quality input for subsequent probabilistic Hough line extraction. The probabilistic Hough line extraction algorithm first identifies potential straight line segments in the image, and then uses probability distribution to evaluate the possibility of the existence of the line. After extracting all the straight lines contained in the runway image, they are fused and classified. After the lines are clustered and fused by the K-Means clustering algorithm, the runway edge line and the horizon are classified according to their line parameters, and the line feature parameters are transmitted to the PnP positioning algorithm posture link; There is no separate measurement indicator for the ground test part of this link. If the subsequent posture positioning algorithm can output the positioning parameters normally, it means that this link can output the straight line eigenvalue normally; 6) PnP positioning algorithm posture acquisition link and posture positioning algorithm ground test: According to the different tasks and function settings, the straight line characteristics of the airport runway transmitted by the straight line edge detection algorithm link, and the difficulty of identifying the target feature mark, different positioning algorithms are selected: Algorithm 1 based on the left edge line, right edge line and horizon of the airport runway, and Algorithm 2 based on the left edge line, right edge line and start line of the runway; During the visual navigation landing process, when the horizon in the camera scene is clear and the distance to the start line of the airport runway is far, the line composed of the horizon and the left and right edge lines of the runway is used. The first algorithm of the feature first calculates the roll angle and pitch angle of the UAV flight according to the recognized horizon feature, and calculates the camera's yaw angle according to the edge line feature and the camera's focal length f and pixel size d, and then calculates the camera's posture through the principle of similar triangles in space; if the horizon in the camera scene is not clear or the start line feature of the runway is obvious, then select algorithm 2 to locate the camera. The visual positioning system identifies and segments the features of the left edge line, right edge line and start line of the runway and calculates the camera's yaw angle, pitch angle and roll angle, and calculates the camera's posture through the rotation matrix formula; The above two positioning algorithms output six parameters, namely, the longitudinal distance, lateral distance, and height distance of the camera in the world coordinate system, and the roll angle, yaw angle, and pitch angle of the camera in the camera coordinate system. When the system is working, a set of straight line feature data output by the straight line edge detection algorithm link is input separately. If the six sets of parameter indicators output by this link meet the accuracy required for the current flight, it means that the posture positioning algorithm is working normally. 7) Algorithm joint test and posture output: Test the connectivity of the algorithms in the above three links and the connectivity of the camera and visual computer communication. The camera captures image information, displays the camera's posture information in real time and transmits it to the flight control computer, indicating that the system software and hardware are working properly; 8) Accuracy verification: After test flight, the algorithm meets the navigation accuracy requirements in certain scenarios.

Citation Information

Patent Citations

  • Visual positioning method for fixed-wing unmanned aerial vehicle in landing stage

    CN115272458A

  • Unmanned aerial vehicle autonomous landing method and device based on visual image guidance

    CN117636284A

  • Runway center line side deviation detection method and device based on semantic segmentation, equipment and medium

    CN118097472A

  • Determining a current pose estimate of an aircraft relative to a runway to support the aircraft on approach

    US20220198703A1