Multi-rotor UAV autonomous tracking and landing control system and control method

By carrying a monocular camera and a visual positioning camera on a multi-rotor drone, combined with QR code landmarks, high-precision and stable autonomous tracking and landing are achieved, and the problem of insufficient control accuracy and stability in the existing technology is solved. In various application scenarios.

CN115237158BActive Publication Date: 2024-11-19JILIN UNIVERSITY

Patent Information

Application Number
CN202210987758.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-08-17
Publication Date
2024-11-19
Estimated Expiration
2042-08-17

AI Technical Summary

Technical Problem

In the prior art, multi-rotor drones based on visual navigation have low accuracy and poor stability, making it difficult to meet the application needs in agriculture, aerial photography, power line patrol and other fields.

Method used

A system consisting of ground computers, remote controls, flight controllers, on-board processors and visual navigation modules is used to estimate positions and postures through QR code landmarks, and control commands for tracking or landing modes are generated. , realize high-precision and stable autonomous tracking and landing.

Benefits of technology

The drone can independently find landmarks through visual information and achieve high-precision tracking and landing without relying on the global positioning system, which improves the robustness and stability of the system and reduces cost and external signal interference.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115237158B_ABST
    Figure CN115237158B_ABST
Patent Text Reader

Abstract

The present invention discloses a multi-rotor unmanned aerial vehicle autonomous tracking and landing control system and control method. A landmark composed of two layers of two-dimensional codes is collected by a monocular camera and transmitted to an onboard processor to realize the estimation of the position and posture of the unmanned aerial vehicle relative to the landmark. The unmanned aerial vehicle's own posture is collected by a visual positioning camera and output to the onboard processor. The onboard processor generates a control instruction and transmits it to the flight controller, thereby controlling the unmanned aerial vehicle to perform tracking or landing tasks. The multi-rotor unmanned aerial vehicle autonomous tracking and landing control method of the present invention enables the unmanned aerial vehicle to autonomously find landmarks, track landmarks and land on landmarks with high precision and good stability without human guidance, only relying on visual information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle visual navigation, and in particular relates to a multi-rotor unmanned aerial vehicle autonomous tracking and landing control system and a control method. Background Art

[0002] In recent years, multi-rotor drones have been widely used in the civilian field, making them a research hotspot in the field of drones. At the same time, with the rapid development of image processing technology and related hardware, obtaining the desired image information by equipping multi-rotor drones with visual sensors has gradually become a research field that has attracted much attention from technicians in the field of drones in recent years. In addition, many drone manufacturers have designed and released many multi-rotor drone products with new functions by combining visual sensors with drone systems.

[0003] In the design and development process of drones, especially multi-rotor drones, the tracking of moving targets and autonomous landing are two extremely important control stages for achieving autonomous flight of drones. Among them, the tracking technology of moving targets is the basis for drones to carry visual sensors to perform related tasks. Therefore, if drones can be accurately and stably controlled to achieve autonomous tracking and autonomous landing, the intelligence of drones will be greatly enhanced. At present, autonomous navigation capabilities are required as a prerequisite for autonomous tracking and landing of drones.

[0004] With the deepening of image processing and machine vision algorithm research and the improvement of GPU and DSP hardware processing speed, visual navigation technology has received more and more attention. Especially in complex environments, vision-based autonomous landing and navigation systems can detect targets or obstacles earlier than other sensors to ensure flight safety. The premise of realizing autonomous navigation of drones is to detect the surrounding environment of the aircraft and estimate its own position. Due to the limitations of GPS use, how to achieve autonomous navigation that does not rely on global positioning system information has become an important research goal in the field of drone research, and image-based visual navigation is a key technology.

[0005] Compared with other traditional navigation methods, visual navigation has the advantages of lower cost, richer recorded information, and less interference from external signals.

[0006] In the existing technology, the tracking and landing control solutions based on visual navigation have low control accuracy and poor control stability. With the continuous expansion of the application of drones in agriculture, aerial photography, power line inspection, unmanned express delivery and other fields, the autonomous tracking and landing control technology of multi-rotor drones needs to be further improved. Summary of the invention

[0007] In view of the defects existing in the above-mentioned prior art, the present invention discloses a multi-rotor UAV autonomous tracking and landing control system and control method, which control the multi-rotor UAV to achieve high-precision and good stability autonomous tracking and landing.

[0008] Combined with the accompanying drawings, the technical solution of the present invention is as follows:

[0009] The multi-rotor UAV autonomous tracking and landing control system includes: a ground terminal module composed of a ground computer and a remote controller, a flight controller, an airborne processor, and a power supply module, and is characterized by:

[0010] Also includes: landmark, visual navigation module;

[0011] The landmark is composed of two layers of QR codes, the center points of the two layers of QR codes coincide with each other;

[0012] The visual navigation module is composed of a monocular camera and a visual positioning camera carried on the multi-rotor drone;

[0013] The monocular camera is used to collect video stream information including landmarks around the multi-rotor drone. The signal output end of the monocular camera is connected to an onboard processor. The onboard processor detects and identifies the landmarks, and calculates and analyzes to obtain the position and attitude estimation results of the multi-rotor drone relative to the landmarks.

[0014] The visual positioning camera is used to estimate the state information of the multi-rotor drone itself, including the position and attitude, and the signal output end of the visual positioning camera is connected to the onboard processor;

[0015] The onboard processor generates control instructions corresponding to the tracking mode or the landing mode based on the state information estimation results of the multi-rotor UAV itself, including the position and attitude, and the position and attitude estimation results of the multi-rotor UAV relative to the landmark, and transmits them to the flight controller to control the multi-rotor UAV to fly to the desired position in the tracking mode or the landing mode.

[0016] Furthermore, the outer dimension of the landmark is 150 mm×150 mm, and the inner dimension of the landmark is 30 mm×30 mm.

[0017] Furthermore, the flight controller:

[0018] Connected to the onboard processor signal, it is used to receive the control instructions generated by the onboard processor to control the flight status of the multi-rotor UAV, and to feed back the flight status of the multi-rotor UAV to the onboard processor in real time to achieve closed-loop control.

[0019] Connected to the remote control signal to receive the remote control manual multi-rotor drone flight status control instructions;

[0020] Connected to the ground computer signal, it is used to feed back the flight status of the multi-rotor drone to the ground computer in real time.

[0021] Furthermore, the onboard processor:

[0022] Connected with the ground computer signal, used for remote login of the ground computer, and receiving control instructions sent by the ground computer signal, including: starting the airborne processor, tracking or landing mode switching;

[0023] It is connected to the flight controller signal and is used to send the analyzed flight control command signal to the flight controller, thereby controlling the flight state of the multi-rotor UAV through the flight controller, and controlling the UAV to fly to the desired position to execute tracking or landing commands.

[0024] Furthermore, the power supply module is electrically connected to the visual positioning camera, the monocular camera, the airborne processor and the flight controller respectively.

[0025] The autonomous tracking and landing control method of a multi-rotor UAV adopts the control system described above. The specific steps of the autonomous tracking and landing control method are as follows:

[0026] S1: Control the multi-rotor drone to take off to the designated position and then hover;

[0027] S2: Collect video stream information around the multi-rotor drone. If a landmark is identified within a preset time, proceed to step S3. Otherwise, the multi-rotor drone rises to a preset height and repeats step S2.

[0028] S3: Analyze the transformation relationship between the pixel coordinate system, image coordinate system, monocular camera coordinate system, and the inertial coordinate system where the landmark is located, and calculate the position and attitude estimation of the multi-rotor drone relative to the landmark;

[0029] S4: taking the estimated pose of the multi-rotor UAV relative to the landmark at the current moment as input to the onboard processor, and converting the onboard processor into a control command for controlling the maneuver of the multi-rotor UAV to the flight controller;

[0030] S5: Determine the current mode of the multi-rotor drone. If it is a tracking mode, proceed to step S6; if it is a landing mode, proceed to step S7;

[0031] S6: The onboard processor generates a tracking mode control instruction based on the estimated position of the multi-rotor drone relative to the landmark at the current moment and combines it with the "tracking" program control instruction, and transmits it to the flight controller, which controls the multi-rotor drone to fly to the desired position in the tracking mode through the flight controller;

[0032] S7: Based on the estimated position of the multi-rotor drone relative to the landmark at the current moment and combined with the "landing" program control instruction, the onboard processor generates a landing mode control instruction and transmits it to the flight controller. The flight controller controls the multi-rotor drone to fly to the desired position in the landing mode.

[0033] Furthermore, in step S3, the process of obtaining the position and posture information of the monocular camera relative to the landmark is specifically as follows:

[0034] First: pixel coordinate system AO(u,v) and image coordinate system BO s The (x,y) conversion relationship is as follows:

[0035]

[0036]

[0037] In the above conversion relationship:

[0038] d x ,d y The photosensitive size occupied by the pixel coordinate point;

[0039] (u0,v0) is the coordinate of pixel O0 of the pixel coordinate origin.

[0040] Then: Figure 5 As shown, the monocular camera coordinate system CO is obtained by using the imaging principle c (x c ,y c ,z c ) and the image coordinate system BO s The conversion relationship of (x,y) is:

[0041]

[0042]

[0043] In the above conversion relationship:

[0044] f is the focal length of the monocular camera.

[0045] Next: Monocular camera coordinate system CO c (x c ,y c ,z c ) and the inertial coordinate system DO where the landmark is located e (x e ,y e ,z e ) is:

[0046]

[0047] Finally: Through the above conversion relationship, we can obtain the pixel coordinate system AO(u,v) and the inertial coordinate system DO where the landmark is located e (x e ,y e ,z e )’s conversion relationship:

[0048]

[0049]

[0050] In the above conversion relationship:

[0051] Parameter f u ,f v ,u0,v0 are obtained through the calibration of the monocular camera. The pixel coordinates and inertial coordinates of the three spatial object points are known. The equation group is solved to obtain t x , t y , t z And six unknown parameters θ, φ, ψ, we can get the pixel coordinate system AO(u,v) and the inertial coordinate system DO where the landmark is located e (x e ,y e ,z e ), thereby realizing the position and attitude estimation of the multi-rotor UAV equipped with a monocular camera relative to the landmark.

[0052] Furthermore, in step S6, controlling the multi-rotor drone to fly to the desired position in the tracking mode through the flight controller means: the multi-rotor drone approaches the landmark in the horizontal direction to achieve autonomous tracking of the landmark by the multi-rotor drone, and maintaining the flight altitude of the multi-rotor drone to ensure that the target is in the field of view of the monocular camera video stream acquisition.

[0053] Furthermore, in step S7, the multi-rotor drone is controlled by the flight controller to fly to a desired position in the following landing mode, and the specific process is as follows:

[0054] S71: Control the multi-rotor drone to approach the landmark horizontally while descending its own altitude;

[0055] S72: Determine whether the landmark is lost. If the landmark is lost, ascend to a preset flight altitude H2 to expand the field of view so that the landmark re-enters the field of view of the monocular camera video stream. If the landmark is not lost, calculate the straight-line distance between the multi-rotor drone and the center of the landmark.

[0056] S73: Determine whether the straight-line distance between the multi-rotor drone and the center of the landmark is less than a preset distance L. If so, execute step S74; otherwise, continue to maintain the current landing mode, and the multi-rotor drone approaches the landmark horizontally while descending its own altitude until the straight-line distance between the multi-rotor drone and the center of the landmark is less than the preset distance L.

[0057] S74: The onboard processor sends a drive motor stop control command to the flight controller, and the flight controller controls the drive motor of the multi-rotor drone to stop rotating, the wings of the multi-rotor drone stop rotating, and the multi-rotor drone lands according to its own gravity.

[0058] Furthermore, in step S2, the preset time T is 10s, and the preset height H1 is 0.3m;

[0059] In step S72, the preset flight altitude H2 is 0.3 m;

[0060] In step S73, the preset distance L is 0.3 m.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. The multi-rotor UAV autonomous tracking and landing control method of the present invention enables the UAV to autonomously find, track and land on landmarks by relying on visual information without human guidance when performing tasks.

[0063] 2. In the multi-rotor UAV autonomous tracking and landing control system of the present invention, a self-designed two-layer nested ArUco code is used as a landmark, which has higher accuracy and robustness when the UAV is autonomously tracking or landing.

[0064] 3. The multi-rotor UAV autonomous tracking and landing control method of the present invention enables the UAV to have higher stability during operation through multiple target recognition and judgment steps.

[0065] 4. In the multi-rotor UAV autonomous tracking and landing control system of the present invention, only a camera is used as a sensor to identify the surrounding environment and landmarks of the UAV, and the system structure is simple and the manufacturing cost is low. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is the structural block diagram of the autonomous tracking and landing control system of a multi-rotor UAV;

[0067] Figure 2 This is a flowchart of the autonomous tracking and landing control method of the multi-rotor UAV of the present invention;

[0068] Figure 3It is a schematic diagram of the pixel coordinate system A, the image coordinate system B, the monocular camera coordinate system C, and the inertial coordinate system D where the landmark is located in the multi-rotor drone autonomous tracking and landing control method of the present invention;

[0069] Figure 4 A schematic diagram of a landmark in the multi-rotor UAV control system and the autonomous tracking and landing control method of the present invention;

[0070] Figure 5 The figure is a schematic diagram of the imaging principle used in the autonomous tracking and landing control method of the multi-rotor UAV of the present invention. DETAILED DESCRIPTION

[0071] In order to clearly and completely describe the technical solution and its specific working process of the present invention, the specific implementation methods of the present invention are as follows in conjunction with the accompanying drawings of the specification:

[0072] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0073] Embodiment 1:

[0074] This embodiment 1 discloses a multi-rotor drone control system. Figure 1 As shown, the control system includes: landmarks, visual navigation module, ground terminal module, flight controller, airborne processor, power supply module. Among them:

[0075] Said landmarks:

[0076] like Figure 3As shown, the landmark adopts a secondary structure landmark composed of an inner and outer layer of two-dimensional codes, that is, a two-layer nested ArUco code, the outer layer size of the landmark is 150mm×150mm, the inner layer size of the landmark is 30mm×30mm, the outer layer uses a two-dimensional code with id number 31, and the inner layer uses a two-dimensional code with id number 21. The center points of the inner and outer layers of the two-dimensional codes coincide, ensuring that the UAV system can directly take the inner and outer layers of the two-dimensional codes to estimate the position of the multi-rotor UAV relative to the landmark while simultaneously identifying the secondary structure landmark, and take the average of the position of the multi-rotor UAV relative to the landmark estimated by taking the inner layer of the two-dimensional code as the landmark information and the position of the multi-rotor UAV relative to the landmark estimated by taking the outer layer of the two-dimensional code as the landmark information as the average of the two as the estimated value of the position of the multi-rotor UAV relative to the landmark at that moment and then as the input value of the next control node.

[0077] The visual navigation module:

[0078] It consists of a monocular camera and a visual positioning camera mounted on a multi-rotor drone, including:

[0079] The monocular camera is used to collect video stream information including landmarks around the multi-rotor drone, and the signal output end of the monocular camera is connected to the onboard processor for the onboard processor to check and identify the landmarks;

[0080] In the first embodiment, the monocular camera adopts a high-definition camera produced by Shenzhen Weixin World Technology Co., Ltd. The camera model is SY011HD. This camera has 2 million pixels, a focal length of 3mm, a resolution of 1920×1080, a viewing angle of 130°, an operating voltage of 5V, and can output images in MJPG, YUV, and YUV2 formats;

[0081] The visual positioning camera is used to estimate the state information of the multi-rotor drone itself, including the position and attitude, and the signal output end of the visual positioning camera is connected to the onboard processor;

[0082] In the first embodiment, the camera selected in this article is the T265 camera produced by Intel Corporation. The visual positioning camera consists of two fisheye lenses and has a built-in V-SLAM algorithm, which can directly output the estimation result of the multi-rotor drone's own state.

[0083] During the test of the multi-rotor drone control system described in the first embodiment, a protective net was built around the site to ensure the safety of the test personnel. Since the drone positioning data is entirely dependent on visual information, a number of QR codes are fixed on the protective net. The visual positioning camera in front of the multi-rotor drone integrates the V-SLAM positioning algorithm during flight, and the position and attitude information of the multi-rotor drone can be directly output through the visual positioning camera. Therefore, by setting relatively obvious feature point markers around the multi-rotor drone, the positioning accuracy of the multi-rotor drone can be effectively improved.

[0084] The ground terminal module:

[0085] It consists of a ground computer and a remote controller, including:

[0086] The ground computer:

[0087] Connected with the onboard processor signal, the ground computer remotely logs into the onboard processor via WIFI to input discrete control instructions to make the tracking or landing system work, and switch the tracking mode or landing mode;

[0088] Connect with the flight controller signal to obtain the flight status of the multi-rotor drone in real time;

[0089] The remote control:

[0090] Connected to the flight controller signal to manually send flight control commands to the multi-rotor drone;

[0091] In the first embodiment, the remote controller adopts a FS-i6s model UAV remote controller, which has the function of switching between manual flight mode and Offboard mode. The Offboard mode means that the current flight control command of the UAV is issued by the onboard processor, and the manual mode means that the flight command of the UAV is issued by the remote controller. Therefore, the ground-side remote controller has the highest control over the UAV to avoid the situation where the UAV loses control due to system errors.

[0092] The flight controller:

[0093] Connected to the onboard processor signal, it is used to receive the control instructions generated by the onboard processor to control the flight status of the multi-rotor UAV, and to feed back the flight status of the multi-rotor UAV to the onboard processor in real time to achieve closed-loop control.

[0094] Connected to the remote control signal to receive the remote control manual multi-rotor drone flight status control instructions;

[0095] Connected to the ground computer signal to provide real-time feedback of the multi-rotor UAV flight status to the ground computer;

[0096] In the first embodiment of the present invention, the flight controller is Pixhawk4 hardware jointly developed by Shenzhen Heli Company and the PX4 team. It is an automatic driving device that can control the drone by outputting multiple control instructions through the built-in control algorithm based on the data information collected by the onboard barometer, gyroscope, GPS and other sensors. The latest PX4 native firmware is pre-installed. Since the PX4 project is open source, it has been developed and maintained by most scientific research enthusiasts. Currently, PX4 firmware has become the universal standard for automatic driving software.

[0097] The onboard processor:

[0098] Connected to the visual positioning camera signal to receive the multi-rotor drone's own status information output by the visual positioning camera;

[0099] Connected to the monocular camera signal, it is used to receive the video stream information including landmark information collected and output by the monocular camera, and then detect and identify the landmarks, and calculate and analyze the position and attitude estimation results of the multi-rotor drone relative to the landmarks based on the V-SLAM algorithm;

[0100] Connected with the ground computer signal, used for remote login of the ground computer, and receiving control instructions sent by the ground computer signal, including: starting the airborne processor, tracking / landing mode switching, etc.;

[0101] Connected to the flight controller signal, used to send the analyzed flight control command signal to the flight controller. Then the flight controller controls the flight state of the multi-rotor drone and controls the drone to fly to the desired position to perform tracking or landing tasks;

[0102] In the first embodiment, the airborne processor uses the Jetson Xavier NX board developed by NVIDIA as the airborne processor of the system. The board has 40 pins and provides expansion interfaces such as I2C, UART and SPI, and also provides 4 USB 3.1A ports to meet the needs of most users.

[0103] In the first embodiment, the onboard processor publishes a MAVLink type message to the flight controller through the MAVROS interface of the ROS system.

[0104] The power supply module:

[0105] They are respectively connected to the visual positioning camera, monocular camera, airborne processor and flight controller by electrical signals to supply power to individual electrical components.

[0106] Embodiment 2:

[0107] The second embodiment discloses a multi-rotor UAV autonomous tracking and landing control method, which is based on the multi-rotor UAV control system described in the first embodiment.

[0108] like Figure 2 As shown, the specific steps of the autonomous tracking and landing control method are as follows:

[0109] S1: Control the multi-rotor drone to take off to the designated position and then hover;

[0110] The multi-rotor UAV is powered on, the control system of the multi-rotor UAV is powered on, and after the onboard processor of the control system is turned on, the operator remotely logs in to the computer end of the onboard processor on the ground PC of the ground terminal module and inputs a take-off command. The onboard processor receives the take-off command and sends a control command to the flight controller to control the multi-rotor UAV to take off to a specified position and then hover.

[0111] S2: Collect video stream information around the multi-rotor drone. If a landmark is identified within a preset time, proceed to step S3. Otherwise, the multi-rotor drone rises to a preset height and repeats step S2.

[0112] The monocular camera in the visual navigation module is mounted on the multi-rotor drone. The monocular camera collects video stream information around the multi-rotor drone and sends it to the airborne processor. The airborne processor detects and identifies the received video stream information. If a landmark is detected within the preset time T, the process proceeds to the following step S3. Otherwise, the airborne processor sends a control instruction to the flight controller to control the multi-rotor drone to automatically continue to rise from the current altitude to the preset altitude H1 to expand the field of view of the monocular camera, and repeats this step S2 to detect and identify the landmark again.

[0113] In this step S2, the preset time T is 10s, and the preset height H1 is 0.3m.

[0114] S3: Based on the V-SLAM algorithm, the transformation relationship between the pixel coordinate system A, the image coordinate system B, the monocular camera coordinate system C, and the inertial coordinate system D where the landmark is located is analyzed to estimate the position and attitude of the multi-rotor drone relative to the landmark;

[0115] like Figure 3 As shown, the process of obtaining the position and posture information of the monocular camera relative to the landmark is as follows:

[0116] First: pixel coordinate system AO(u,v) and image coordinate system BO s The (x,y) conversion relationship is as follows:

[0117]

[0118]

[0119] In the above conversion relationship:

[0120] d x ,d y The photosensitive size occupied by the pixel coordinate point;

[0121] (u0,v0) is the coordinate of pixel O0 of the pixel coordinate origin.

[0122] Then: Figure 5 As shown, the monocular camera coordinate system CO is obtained by using the imaging principle c (x c ,y c ,z c ) and the image coordinate system BO s The conversion relationship of (x,y) is:

[0123]

[0124]

[0125] In the above conversion relationship:

[0126] f is the focal length of the monocular camera.

[0127] Next: Monocular camera coordinate system CO c (x c ,y c ,z c ) and the inertial coordinate system DO where the landmark is located e (x e ,y e ,z e ) is:

[0128]

[0129] Finally: Through the above conversion relationship, we can obtain the pixel coordinate system AO(u,v) and the inertial coordinate system DO where the landmark is located e (x e ,y e ,z e )’s conversion relationship:

[0130]

[0131]

[0132] In the above conversion relationship:

[0133] Parameter f u ,f v ,u0,v0 are obtained by monocular camera calibration, and the only unknown parameter is t x , t y , t z The pixel coordinates and inertial coordinates of the three spatial object points are known. By solving the above six unknown parameters through the equations, we can obtain the pixel coordinate system AO(u,v) and the inertial coordinate system DO where the landmark is located. e (x e ,y e ,z e), thereby realizing the position and attitude estimation of the multi-rotor UAV equipped with a monocular camera relative to the landmark;

[0134] like Figure 3 As shown, the landmark adopts a secondary structure landmark composed of an inner and outer layer of two-dimensional codes, that is, two layers of nested ArUco codes, the outer layer uses a two-dimensional code with id number 31, and the inner layer uses a two-dimensional code with id number 21. The center points of the inner and outer layers of two-dimensional codes coincide, ensuring that the UAV system can directly take the inner and outer layers of two-dimensional codes to estimate the position of the multi-rotor UAV relative to the landmark while simultaneously identifying the secondary structure landmark, and take the average of the position of the multi-rotor UAV relative to the landmark estimated by taking the inner layer of two-dimensional code as the landmark information and the position of the multi-rotor UAV relative to the landmark estimated by taking the outer layer of two-dimensional code as the landmark information as the average of the two as the estimated value of the position of the multi-rotor UAV relative to the landmark at that moment and then as the input value of the next control node.

[0135] S4: After the estimated value of the multi-rotor drone's current position relative to the landmark is calculated, it is used as an input to the onboard processor, which converts it into a control command for controlling the multi-rotor drone's maneuvers and sends it to the flight controller;

[0136] S5: Determine the current mode of the multi-rotor drone. If it is a tracking mode, proceed to step S6; if it is a landing mode, proceed to step S7;

[0137] In this step S5, the onboard processor determines the current mode of the multi-rotor UAV by receiving the mode program instruction issued by the ground-end module. If the onboard processor receives the program instruction of the tracking mode issued by the ground-end module, it determines that the multi-rotor UAV is currently in the tracking mode and enters step S6. If the onboard processor receives the program instruction of the landing mode issued by the ground-end module, it determines that the multi-rotor UAV is currently in the landing mode and enters step S7.

[0138] S6: Based on the estimated position of the multi-rotor drone relative to the landmark at the current moment and combined with the "tracking" program control instruction, the onboard processor generates a tracking mode control instruction and transmits it to the flight controller. The flight controller controls the multi-rotor drone to fly to the desired position in the tracking mode, that is, the multi-rotor drone approaches the landmark in the horizontal direction to achieve autonomous tracking of the landmark by the multi-rotor drone, and maintains the flight altitude of the multi-rotor drone to ensure that the target is in the field of view of the monocular camera video stream acquisition.

[0139] S7: Based on the estimated position of the multi-rotor drone relative to the landmark at the current moment and combined with the "landing" program control instruction, the onboard processor generates a landing mode control instruction and transmits it to the flight controller. The flight controller controls the multi-rotor drone to fly to the desired position in the landing mode. The specific process is as follows:

[0140] S71: Control the multi-rotor drone to approach the landmark horizontally while descending its own altitude;

[0141] S72: Determine whether the landmark is lost. If the landmark is lost, ascend to a preset flight altitude H2 to expand the field of view so that the landmark re-enters the field of view of the monocular camera video stream. If the landmark is not lost, calculate the straight-line distance between the multi-rotor drone and the center of the landmark.

[0142] In this step S72, the preset flight altitude H2 is 0.3m;

[0143] S73: Determine whether the straight-line distance between the multi-rotor drone and the center of the landmark is less than a preset distance L. If so, execute step S74; otherwise, continue to maintain the current landing mode, and the multi-rotor drone approaches the landmark horizontally while descending its own altitude until the straight-line distance between the multi-rotor drone and the center of the landmark is less than the preset distance L.

[0144] In this step S73, the preset distance L is 0.3m;

[0145] S74: The onboard processor sends a drive motor stop control command to the flight controller, and the flight controller controls the drive motor of the multi-rotor drone to stop rotating, the wings of the multi-rotor drone stop rotating, and the multi-rotor drone lands according to its own gravity.

[0146] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

[0147] The specific implementation of the present invention described above does not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. Multi-rotor UAV autonomous tracking and landing control system, including: The ground terminal module, flight controller, airborne processor and power supply module, which are composed of ground computer and remote controller, are characterized by: Also includes: landmark, visual navigation module; The landmark is composed of two layers of QR codes, the center points of the two layers of QR codes coincide with each other; The visual navigation module is composed of a monocular camera and a visual positioning camera carried on the multi-rotor drone; The monocular camera is used to collect video stream information including landmarks around the multi-rotor drone. The signal output end of the monocular camera is connected to an onboard processor. The onboard processor detects and identifies the landmarks, and calculates and analyzes to obtain the position and attitude estimation results of the multi-rotor drone relative to the landmarks. The visual positioning camera is used to estimate the state information of the multi-rotor drone itself, including the position and attitude, and the signal output end of the visual positioning camera is connected to the onboard processor; The onboard processor generates a control instruction corresponding to the tracking mode or the landing mode based on the state information estimation result including the position and attitude of the multi-rotor UAV itself, and the position and attitude estimation result of the multi-rotor UAV relative to the landmark, and transmits it to the flight controller to control the multi-rotor UAV to fly to the desired position in the tracking mode or the landing mode; The outer dimension of the landmark is 150mm×150mm, and the inner dimension of the landmark is 30mm×30mm.

2. The multi-rotor UAV autonomous tracking and landing control system as claimed in claim 1, characterized in that: The flight controller: Connected to the onboard processor signal, it is used to receive the control instructions generated by the onboard processor to control the flight status of the multi-rotor UAV, and to feed back the flight status of the multi-rotor UAV to the onboard processor in real time to achieve closed-loop control. Connected to the remote control signal to receive the remote control manual multi-rotor drone flight status control instructions; Connected to the ground computer signal, it is used to feed back the flight status of the multi-rotor drone to the ground computer in real time.

3. The multi-rotor UAV autonomous tracking and landing control system as claimed in claim 1, characterized in that: The onboard processor: Connected with the ground computer signal, used for remote login of the ground computer, and receiving control instructions sent by the ground computer signal, including: starting the airborne processor, tracking or landing mode switching; It is connected to the flight controller signal and is used to send the analyzed flight control command signal to the flight controller, thereby controlling the flight state of the multi-rotor UAV through the flight controller, and controlling the UAV to fly to the desired position to execute tracking or landing commands.

4. The multi-rotor UAV autonomous tracking and landing control system as claimed in claim 1, characterized in that: The power supply module is electrically connected to the visual positioning camera, the monocular camera, the airborne processor and the flight controller respectively.

5. The autonomous tracking and landing control method of a multi-rotor UAV is characterized by: Using the control system described in any one of claims 1 to 4, the specific steps of the autonomous tracking and landing control method are as follows: S1: Control the multi-rotor drone to take off to the designated position and then hover; S2: Collect video stream information around the multi-rotor drone. If a landmark is identified within a preset time, proceed to step S3. Otherwise, the multi-rotor drone rises to a preset height and repeats step S2. S3: Analyze the transformation relationship between the pixel coordinate system, image coordinate system, monocular camera coordinate system, and the inertial coordinate system where the landmark is located, and calculate the position and attitude estimation of the multi-rotor drone relative to the landmark; S4: taking the estimated pose of the multi-rotor UAV relative to the landmark at the current moment as input to the onboard processor, and converting the onboard processor into a control command for controlling the maneuver of the multi-rotor UAV to the flight controller; S5: Determine the current mode of the multi-rotor drone. If it is a tracking mode, proceed to step S6; if it is a landing mode, proceed to step S7; S6: The onboard processor generates a tracking mode control instruction based on the estimated position of the multi-rotor drone relative to the landmark at the current moment and combines it with the "tracking" program control instruction, and transmits it to the flight controller, which controls the multi-rotor drone to fly to the desired position in the tracking mode through the flight controller; S7: Based on the estimated position of the multi-rotor drone relative to the landmark at the current moment and combined with the "landing" program control instruction, the onboard processor generates a landing mode control instruction and transmits it to the flight controller. The flight controller controls the multi-rotor drone to fly to the desired position in the landing mode.

6. The multi-rotor UAV autonomous tracking and landing control method as claimed in claim 5, characterized in that: In step S3, the process of obtaining the position and posture information of the monocular camera relative to the landmark is as follows: First: pixel coordinate system AO(u,v) and image coordinate system BO s The (x,y) conversion relationship is as follows: In the above conversion relationship: d x ,d y The photosensitive size occupied by the pixel coordinate point; (u0, v0) is the pixel O0 coordinate of the pixel coordinate origin; Using the imaging principle to obtain the monocular camera coordinate system CO c (x c ,y c ,z c ) and the image coordinate system BO s The conversion relationship of (x,y) is: In the above conversion relationship: f is the focal length of the monocular camera; Next: Monocular camera coordinate system CO c (x c ,y c ,z c ) and the inertial coordinate system DO where the landmark is located e (x e ,y e ,z e ) is: Finally: Through the above conversion relationship, we can obtain the pixel coordinate system AO(u,v) and the inertial coordinate system DO where the landmark is located e (x e ,y e ,z e )’s conversion relationship: In the above conversion relationship: Parameter f u ,f v ,u0,v0 are obtained through the calibration of the monocular camera. The pixel coordinates and inertial coordinates of the three spatial object points are known. The equation group is solved to obtain t x , t y , t z And six unknown parameters θ, φ, ψ, we can get the pixel coordinate system AO(u,v) and the inertial coordinate system DO where the landmark is located e (x e ,y e ,z e ), thereby realizing the position and attitude estimation of the multi-rotor UAV equipped with a monocular camera relative to the landmark.

7. The multi-rotor UAV autonomous tracking and landing control method as claimed in claim 6, characterized in that: In step S6, controlling the multi-rotor drone to fly to the desired position in the tracking mode through the flight controller means that the multi-rotor drone approaches the landmark in the horizontal direction to achieve autonomous tracking of the landmark by the multi-rotor drone, and maintains the flight altitude of the multi-rotor drone to ensure that the target is in the field of view of the monocular camera video stream acquisition.

8. The multi-rotor UAV autonomous tracking and landing control method as claimed in claim 6, characterized in that: In step S7, the multi-rotor drone is controlled by the flight controller to fly to the desired position in the follow-and-land mode. The specific process is as follows: S71: Control the multi-rotor drone to approach the landmark horizontally while descending its own altitude; S72: Determine whether the landmark is lost. If the landmark is lost, ascend to a preset flight altitude H2 to expand the field of view so that the landmark re-enters the field of view of the monocular camera video stream. If the landmark is not lost, calculate the straight-line distance between the multi-rotor drone and the center of the landmark. S73: Determine whether the straight-line distance between the multi-rotor drone and the center of the landmark is less than a preset distance L. If so, execute step S74; otherwise, continue to maintain the current landing mode, and the multi-rotor drone approaches the landmark horizontally while descending its own altitude until the straight-line distance between the multi-rotor drone and the center of the landmark is less than the preset distance L. S74: The onboard processor sends a drive motor stop control command to the flight controller, and the flight controller controls the drive motor of the multi-rotor drone to stop rotating, the wings of the multi-rotor drone stop rotating, and the multi-rotor drone lands according to its own gravity.

9. The multi-rotor UAV autonomous tracking and landing control method as claimed in claim 8, characterized in that: In step S2, the preset time T is 10s, and the preset height H1 is 0.3m; In step S72, the preset flight altitude H2 is 0.3 m; In step S73, the preset distance L is 0.3 m.

Citation Information

Patent Citations

  • Autonomous precision landing system of unmanned aerial vehicle on motion platform and landing method

    CN106527487A

  • Control method for rotor-wing unmanned aerial vehicle to autonomously land on mobile target

    CN107515622A

  • Natural landmark identification and autonomous landing method of drone

    CN109460046A

  • Sea-air collaborative visual tracking and autonomous recovery method for unmanned ship-mounted unmanned aerial vehicle

    CN113657256A

  • Method for accurate landing of an unmanned aerial vehicle and device for implementing the method

    RU2773978C1

Cited By

  • Unmanned aerial vehicle accurate take-off and landing control system based on RTK and vision fusion

    CN121594880A

  • Unmanned aerial vehicle precision take-off and landing control system based on fusion of RTK and vision

    CN121594880B