A flight control method and system for unmanned aerial vehicle tracking technology

Abstract

The present invention discloses a flight control method and system of unmanned aerial vehicle tracking technology, which relates to the field of unmanned aerial vehicle control technology. The method comprises the following steps: before the unmanned aerial vehicle takes off, the airborne sensor is pre-started, parameter initialization and calibration operations are performed, and magnetic markers are installed on the tracking target at the same time; the body position of the unmanned aerial vehicle during flight is obtained, and a multi-sensor data fusion algorithm is used to sense the target position of the tracking target in real time; obstacles existing in the flight path are detected in real time, integrated into a three-dimensional environmental model, and combined with a follow-up visual tracking algorithm to capture the motion state of the tracking target; target tracking data is saved and fed back in real time, and the battery status and energy consumption rate of the unmanned aerial vehicle are monitored. By combining magnetic guidance technology, multi-sensor data fusion and a follow-up visual tracking algorithm, the present invention can achieve high-precision target tracking in a dynamic and complex environment, while effectively avoiding obstacles and ensuring the flight stability and tracking accuracy of the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle control technology, and in particular to a flight control method and system for unmanned aerial vehicle tracking technology. Background Art

[0002] A drone, also known as an unmanned aerial vehicle (UAV), is an aircraft that flies through a remote control or autonomous control system. In recent years, drone technology has been widely used in many fields such as military, industry, agriculture, express delivery, and rescue. They are usually equipped with various sensors and cameras, which can perform tasks such as aerial photography, environmental monitoring, and object tracking. The flight control of drones is one of its core technologies, mainly including stable flight, navigation, obstacle avoidance, path planning, and other tasks. The flight control system usually integrates a variety of sensors, such as GPS, inertial measurement unit (IMU), barometer, magnetometer, camera, etc. By fusing the data of these sensors, the attitude, speed, and position of the drone are perceived in real time, thereby ensuring that the drone can fly autonomously or semi-autonomously in various complex environments. The flight control system must not only ensure the stability of the drone in the air, but also adapt to different flight environments and make dynamic adjustments. Especially when tracking moving targets, flight control is particularly critical.

[0003] The tracking technology of drones is an important part of realizing intelligent autonomous flight. Its main purpose is to enable drones to automatically identify and follow a specific target. Common target tracking technologies include vision-based target tracking and signal source-based target positioning. In visual tracking, a camera is used to capture the image information of the target, and combined with deep learning or traditional tracking algorithms such as KCF, MOSSE, YOLO, etc., the position and status of the target are updated in real time. Signal source-based tracking, such as RFID, Wi-Fi, etc., calculates the position of the target through the signal strength and direction emitted by the target. For drones, target tracking is not just about tracking the static position of the target, but also involves real-time adjustments in dynamic scenes, including speed adjustment, flight path planning, and the ability to avoid obstacles.

[0004] However, the existing tracking and flight control technologies still have some shortcomings. The first is the reliability in complex environments. In the face of dynamic obstacles, signal interference or light changes, the performance of existing technologies is not ideal, and it is easy to lose the tracking target or the flight path deviation. Secondly, the real-time problem also needs to be improved. Especially in high-dynamic scenarios, the response time of the tracking algorithm and the flight control system may be long, resulting in the drone being unable to adjust the flight status in time. Furthermore, in terms of energy consumption, tracking tasks often require drones to make frequent attitude adjustments and accelerations, which will accelerate battery consumption and limit the drone's endurance. Although drone tracking technology and tracking-based flight control have made great progress, there is still much room for improvement when facing challenges such as complex environments, dynamic changes in targets, and energy consumption. Summary of the invention

[0005] Based on this, it is necessary to provide a flight control method and system for drone tracking technology to address the above technical problems.

[0006] In a first aspect, the present invention provides a flight control method of a UAV tracking technology, the method comprising the following steps:

[0007] S1. Before the drone takes off, start the onboard sensor in advance, perform parameter initialization and calibration operations, and install magnetic markers on the tracking target; after the drone takes off, the onboard sensor is used to capture the magnetic field signal and motion image of the tracking target in real time;

[0008] S2. Obtain the position of the drone during flight, use a multi-sensor data fusion algorithm to perceive the target position of the tracking target in real time, and use magnetic guidance technology to plan the flight path of the drone based on the relative position relationship between the tracking target and the drone;

[0009] S3, real-time detection of obstacles in the flight path, integration into the three-dimensional environment model, combined with the follow-up visual tracking algorithm, capture the motion state of the tracking target, and dynamically adjust the control parameters and flight attitude of the drone based on the obstacle distribution and motion state changes;

[0010] S4: Save and feedback target tracking data in real time, monitor the battery status and energy consumption rate of the drone, control the remaining flight time according to the drone's navigable mileage, and return to the preset safe area before the energy is completely consumed to end the target tracking mission.

[0011] Furthermore, obtaining the body position of the UAV during flight, using a multi-sensor data fusion algorithm to perceive the target position of the tracking target in real time, and using magnetic guidance technology to plan the flight path of the UAV based on the relative position relationship between the tracking target and the UAV includes the following steps:

[0012] S21, obtaining the current geographical coordinates of the drone through a positioning sensor in a built-in airborne sensor of the drone, and then obtaining the acceleration and angular velocity of the drone using an inertial sensor;

[0013] S22, setting an update interval for the drone, recording the geographic location coordinates, acceleration and angular velocity at each update interval, and then calculating the drone's body position at the current coordinate update interval by combining the body position at the previous update interval and the acceleration and angular velocity at the current update interval;

[0014] S23, obtaining the magnetic field signal of the tracking target, identifying the interval distance between the UAV and the tracking target through magnetic field gradient analysis, and identifying the spatial coordinates of the tracking target in combination with three-dimensional space detection, and obtaining the target position of the tracking target through data fusion and position correction;

[0015] S24. Continuously and periodically update the body position of the UAV during flight and the target position of the tracking target, and monitor the change of the magnetic field gradient between the UAV and the tracking target in real time, and plan the flight path of the UAV by predicting the future target position.

[0016] Furthermore, obtaining the magnetic field signal of the tracking target, identifying the interval distance between the UAV and the tracking target through magnetic field gradient analysis, and identifying the spatial coordinates of the tracking target in combination with three-dimensional space detection, and obtaining the target position of the tracking target through data fusion and position correction includes the following steps:

[0017] S231, using the magnetometer of the drone's built-in airborne sensor to read the magnetic field signal generated by the magnetic marker configured for the tracking target in real time, and by analyzing the magnetic field strength and magnetic field direction of the magnetic field signal, detecting the change in the magnetic field gradient, and identifying the magnetic field direction and magnetic field distance of the tracking target;

[0018] S232, constructing a three-dimensional environment model through the three-dimensional point cloud data detected by the drone, obtaining the motion image collected by the visual sensor, using the target detection algorithm, capturing and tracking the visual marker points of the target in the motion image in real time, and then identifying the spatial coordinates of the visual marker points in the three-dimensional environment model;

[0019] S233. Based on the body position and spatial coordinates in the three-dimensional environment model, the spatial distance, heading angle and pitch angle between the tracking target and the body position are calculated, and the known magnetic field azimuth and magnetic field distance are integrated. Through spatial and magnetic field position correction, the target position of the tracking target is obtained.

[0020] Furthermore, the drone's body position and the target position of the tracking target are continuously and periodically updated during flight, and the magnetic field gradient changes between the drone and the tracking target are monitored in real time. By predicting the future target position, the flight path of the drone is planned, including the following steps:

[0021] S241, presetting target tracking conditions for the UAV to perform a tracking task, the target tracking conditions including a tracking distance interval, a flight altitude interval, a flight speed interval, and a cruising time;

[0022] S242, in the tracking distance interval, the flight altitude interval and the flight speed interval, select the initial distance, the initial altitude and the initial speed, and set the tracking distance as the first priority, the flight altitude as the second priority, and the flight speed as the third priority, with the initial distance being kept unchanged first;

[0023] S243, record the target position of the tracking target at each update interval in real time, predict the target position of the tracking target at the next update interval, mark the position space point in the three-dimensional environment model, and plan the flight path of the UAV according to the initial distance and initial height.

[0024] Furthermore, the target position of the tracking target at each update interval is recorded in real time, the target position of the tracking target at the next update interval is predicted, and the position space point is marked in the three-dimensional environment model. According to the initial distance and initial height, the flight path of the UAV is planned, including the following steps:

[0025] S2431, record the historical target position of the tracking target in the previous n update intervals, form a position sequence, establish a state transfer matrix, and discretize the movement state of the tracking target, each movement state is represented as a combination of speed and direction of the tracking target at a specific position;

[0026] S2432, calculating the movement state of the tracking target at the current update interval according to the flight speed and body position of the UAV at the current update interval, and then using the state transfer matrix to predict the movement state of the tracking target at the next update interval;

[0027] S2433, based on the predicted movement state, estimating the target position of the tracking target at the next update interval, marking the spatial point corresponding to the target position in the three-dimensional environment model, and calibrating the flight point of the UAV at the next update interval in the three-dimensional environment model according to the initial distance and initial height;

[0028] S2434, sequentially connecting the flight points of the current update interval and the next update interval, and using the route formed by the flight points as the flight path for the UAV to perform the tracking task.

[0029] Furthermore, the calculation formula for the target position at the next update interval is:

[0030]

[0031] In the formula, Indicates the tracking target at the next update intervalx Axis coordinates; Indicates the tracking target at the next update interval y Axis coordinates; Indicates the tracking target at the next update interval z Axis coordinates; x t Indicates the tracking target at the current update interval x Axis coordinates; y t Indicates the tracking target at the current update interval y Axis coordinates; z t Indicates the tracking target at the current update interval z Axis coordinates; Indicates that the tracking target is in the prediction state x Velocity components on the axis; Indicates that the tracking target is in the prediction state y Velocity components on the axis; Indicates that the tracking target is in the prediction state z Velocity components on the axis; Indicates the update interval.

[0032] Furthermore, real-time detection of obstacles existing in the flight path, integration into the three-dimensional environment model, combining with the follow-up visual tracking algorithm, capturing the motion state of the tracking target, and dynamically adjusting the control parameters and flight attitude of the UAV based on the obstacle distribution and motion state changes include the following steps: S31, real-time monitoring of environmental data around the flight path through airborne sensors, identifying existing obstacles, and calculating the size, distance and position of the obstacles, generating a three-dimensional virtual model of the obstacles, and then importing the three-dimensional environment model, and dynamically updating the three-dimensional environment model;

[0033] S32, based on the spatial distribution of obstacles, dividing the flyable area and the obstacle area in the three-dimensional environment model, and detecting the spatial area where the flight path is located. If there is an obstacle area, the flight path is optimized by adjusting the control parameters of the UAV;

[0034] S33. Detect the tracking target in the acquired motion image, and use the follow-up visual tracking algorithm to capture the motion state of the tracking target, and then lock the tracking target in the core area of ​​the motion image by adjusting the flight attitude of the drone.

[0035] Furthermore, based on the spatial distribution of obstacles, the flyable area and the obstacle area are divided in the three-dimensional environment model, and the spatial area where the flight path is located is detected. If there is an obstacle area, the flight path is optimized by adjusting the control parameters of the UAV, including the following steps:

[0036] S321, according to the distribution of obstacles in the three-dimensional environment model, the three-dimensional model is divided into a flyable area and an obstacle area, and whether there is an obstacle area in the current flight path is detected. If there is no obstacle area, the original flight path is maintained; if there is an obstacle area, a path adjustment instruction is issued;

[0037] S322. Based on a pre-set priority principle, the initial distance between the UAV and the tracking target is maintained unchanged. The flight speed of the UAV is first adjusted. If the speed adjustment cannot avoid the obstacle, the flight altitude of the UAV is then adjusted to move to a passable flight altitude. If the altitude adjustment cannot avoid the obstacle, the horizontal position of the UAV is adjusted to move to a flight position that can be circumvented.

[0038] S323. According to the UAV path adjustment result, the flight path is reconstructed using the path planning algorithm, and the control parameters of the UAV are adjusted to achieve the purpose of optimizing the path.

[0039] Furthermore, in the acquired motion image, detecting the tracking target, and using the follow-up visual tracking algorithm to capture the motion state of the tracking target, and then adjusting the flight attitude of the drone to lock the tracking target in the core area of ​​the motion image includes the following steps:

[0040] S331, setting a core area of ​​the motion image, selecting a visual tracking algorithm, tracking the position change of the tracking target in the motion image frame by frame, and updating the motion state of the tracking target in real time according to the position change of the tracking target, the motion state including pixel position, speed and acceleration;

[0041] S332. The difference between the current pixel position of the tracking target and the center point of the core area is taken as an error. According to the error value, the flight attitude of the UAV is adjusted using a proportional-integral-differential control algorithm. The flight attitude includes the roll, pitch angle and yaw angle of the UAV.

[0042] In a second aspect, the present invention further provides a flight control system for unmanned aerial vehicle tracking technology, the system comprising:

[0043] The mission configuration module is used to start the onboard sensors before the drone takes off, perform parameter initialization and calibration operations, and install magnetic markers on the tracking target; after the drone takes off, the onboard sensors are used to capture the magnetic field signals and motion images of the tracking target in real time;

[0044] The tracking planning module is used to obtain the position of the drone during flight, use a multi-sensor data fusion algorithm to perceive the target position of the tracking target in real time, and use magnetic guidance technology to plan the flight path of the drone based on the relative position relationship between the tracking target and the drone;

[0045] The attitude control module is used to detect obstacles in the flight path in real time, integrate the three-dimensional environment model, and combine the follow-up visual tracking algorithm to capture the motion state of the tracking target. Based on the obstacle distribution and motion state changes, the control parameters and flight attitude of the UAV are dynamically adjusted;

[0046] The monitoring return module is used to save and feedback target tracking data in real time, monitor the battery status and energy consumption rate of the drone, control the remaining flight time according to the drone's navigable mileage, and return to the preset safe area before the energy is completely consumed to end the target tracking mission.

[0047] The beneficial effects of the present invention are:

[0048] 1. By combining magnetic guidance technology, multi-sensor data fusion and follow-up visual tracking algorithm, high-precision target tracking can be achieved in dynamic and complex environments, while effectively avoiding obstacles and ensuring the flight stability and tracking accuracy of the UAV; compared with traditional solutions, the present invention optimizes the accuracy of target recognition and position perception, especially in changing scenarios. The flight path can be dynamically adjusted, which significantly improves the real-time and intelligence of the tracking task; in addition, the present invention also optimizes the energy management strategy, effectively prolongs the flight time, and reduces the risk of the UAV running out of power during the mission.

[0049] 2. By adopting a multi-sensor data fusion algorithm, accurate real-time perception of the position of the UAV and the tracking target is achieved; the relative distance between the UAV and the tracking target is obtained through magnetic field gradient analysis, and combined with spatial detection technology and position correction mechanism, the tracking accuracy and path planning flexibility are effectively improved; in addition, the use of future target position prediction for path planning enables the UAV to adjust the flight route more intelligently, thereby maintaining stable tracking in complex environments, and improving the system's response speed and tracking efficiency; the real-time positioning and adaptability in dynamic environments are improved, the position error is reduced, and the system's robustness and endurance efficiency are improved.

[0050] 3. By real-time monitoring of obstacles in the flight path, combined with a three-dimensional environmental model and a follow-up visual tracking algorithm, it is possible to automatically identify obstacles in complex environments and dynamically adjust the flight path to ensure the safety and efficiency of the drone. The present invention improves the drone's autonomous obstacle avoidance capability through three-dimensional space modeling and division, accurate identification and dynamic adjustment of the flyable area and obstacle area; at the same time, the follow-up visual tracking algorithm further improves the accuracy of target capture and tracking, ensuring that the tracked target is always within the drone's line of sight, thereby improving the reliability and intelligence of the flight mission. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0052] Figure 1 is a flow chart of a flight control method of a drone tracking technology according to an embodiment of the present invention;

[0053] Figure 2 1 is a system principle structure diagram of a flight control system of a UAV tracking technology according to an embodiment of the present invention.

[0054] Figure numbers: 1. Mission configuration module; 2. Tracking planning module; 3. Attitude control module; 4. Monitoring return module. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0056] See also Figure 1 , a flight control method of unmanned aerial vehicle tracking technology, the method comprising the following steps:

[0057] S1. Before the UAV takes off, start the onboard sensor in advance, perform parameter initialization and calibration operations, and install magnetic markers on the tracking target; after the UAV takes off, the onboard sensor is used to capture the magnetic field signal and motion image of the tracking target in real time.

[0058] Specifically, drones are equipped with a variety of sensors, including GPS, inertial measurement unit (IMU), magnetometer, barometer, camera, etc. Before the drone takes off, these onboard sensors are activated.

[0059] Here are the functions of different sensors:GPS (positioning sensor): used to provide geographic location information.IMU (inertial sensor): including accelerometers and gyroscopes, used to detect the acceleration, angular velocity and other information of the drone to help determine the drone's attitude.Magnetometer: used to sense the direction of the magnetic field and assist in the direction of the drone, especially in magnetic guidance technology.Barometer: used to detect altitude changes.Camera (visual sensor): used to capture real-time motion images of the target to assist visual tracking.

[0060] After the sensors are started, they need to be initialized and calibrated to ensure accurate environmental data and flight parameters can be obtained during flight:

[0061] 1. IMU calibration: Inertial measurement units usually require gyro bias calibration and acceleration calibration to ensure the accuracy of measurement data.

[0062] 2. Magnetometer calibration: Drones are usually calibrated according to the magnetic field strength in the surrounding environment to eliminate interference from the drone's own equipment and environment, ensuring that the target's magnetic field signal can be accurately sensed when using magnetic guidance technology.

[0063] 3. Camera calibration: In order to ensure the accuracy of image capture, operations such as lens distortion correction are usually required, which is especially important in high-precision visual tracking.

[0064] 4. GPS signal detection: Before taking off, you need to ensure that the GPS signal strength is good to ensure the reliability of the location data.

[0065] Installing magnetic markers on the target is an important step in tracking the target using magnetic guidance technology. Magnetic markers generate unique magnetic field signals, allowing the drone to sense the target's position through the magnetometer. Magnetic markers should be lightweight, low power, and not affect the normal movement of the target. When installing, it is necessary to ensure that the position of the magnetic marker does not affect the target's motion state, and the magnetic field strength of the magnetic marker is identifiable within the detection range of the drone.

[0066] After the drone takes off, the onboard magnetometer will continue to capture the magnetic field signal from the tracking target in real time. The strength and direction of the magnetic field signal will change with the distance and relative position between the target and the drone. The core of magnetic guidance technology is to analyze these changes and then infer the relative position of the tracking target. At the same time, the onboard camera will capture the moving image of the target in real time and use visual tracking algorithms (such as KCF, YOLO, etc.) to lock the target.

[0067] S2. Obtain the body position of the UAV during flight, use a multi-sensor data fusion algorithm to perceive the target position of the tracking target in real time, and use magnetic guidance technology to plan the flight path of the UAV based on the relative position relationship between the tracking target and the UAV.

[0068] In the description of the present invention, obtaining the body position of the UAV during flight, using a multi-sensor data fusion algorithm to perceive the target position of the tracking target in real time, and planning the flight path of the UAV using magnetic guidance technology based on the relative position relationship between the tracking target and the UAV includes the following steps:

[0069] S21. Obtain the current geographic location coordinates of the drone through the positioning sensor in the drone's built-in airborne sensor, and then use the inertial sensor to obtain the acceleration and angular velocity of the drone.

[0070] S22. Set the update interval of the drone, record the geographic location coordinates, acceleration and angular velocity at each update interval, and then calculate the drone's body position at the current coordinate update interval by combining the body position at the previous update interval and the acceleration and angular velocity at the current update interval.

[0071] Specifically, at each update interval, the current geographic coordinates of the drone are obtained, including latitude, longitude, and altitude information. The accelerometer is then used to obtain the acceleration data of the drone on three axes (X, Y, Z), and the gyroscope is used to obtain the angular velocity of the drone on three axes.

[0072] Set a fixed update interval (Δt), for example, obtain the position and motion parameters every 0.1 seconds, so that the drone periodically updates its status at regular intervals. The acceleration and angular velocity of the drone will change continuously in each update interval, so the current position of the drone needs to be calculated based on these changes. According to the kinematic equation of acceleration and velocity, the change in the velocity of the drone can be derived from the acceleration, and the change in position can be further derived from the velocity.

[0073] Since the geographic location information provided by GPS is subject to noise and signal interference, and IMU is prone to cumulative errors (drift), algorithms such as Kalman filtering can be used to fuse the data of the two. Kalman filtering combines the location data provided by GPS with the motion state information calculated by IMU to continuously correct the drone's position estimate and obtain a more accurate real-time position.

[0074] S23, obtaining the magnetic field signal of the tracking target, identifying the interval distance between the UAV and the tracking target through magnetic field gradient analysis, and combining three-dimensional space detection to identify the spatial coordinates of the tracking target, and obtaining the target position of the tracking target through data fusion and position correction.

[0075] In the description of the present invention, obtaining the magnetic field signal of the tracking target, identifying the interval distance between the UAV and the tracking target through magnetic field gradient analysis, and identifying the spatial coordinates of the tracking target in combination with three-dimensional space detection, and obtaining the target position of the tracking target through data fusion and position correction includes the following steps:

[0076] S231. The magnetometer of the drone's built-in airborne sensor is used to read the magnetic field signal generated by the magnetic marker configured for the tracking target in real time. By analyzing the magnetic field strength and direction of the magnetic field signal, the magnetic field gradient change is detected, and the magnetic field direction and magnetic field distance of the tracking target are identified.

[0077] Specifically, the magnetic field vector of the current position of the drone is recorded, and its magnetic field signal is sensed through the magnetic markers on the target. The magnetometer provides a value representing the magnetic field strength, which changes as the distance between the drone and the tracking target changes. Generally speaking, the closer the distance, the greater the magnetic field strength; the farther the distance, the smaller the magnetic field strength. The magnetometer also provides the direction information of the magnetic field, which is used to determine the direction of the tracking target relative to the drone.

[0078] The magnetic field gradient refers to the rate of change of the magnetic field strength in space. The gradient is calculated by comparing the magnetic field strength at different locations or at different points in time. By analyzing the changes in the magnetic field gradient, the relative movement state between the tracking target and the drone can be inferred. For example, if the magnetic field gradient gradually increases, it may mean that the drone is approaching the tracking target; if the magnetic field gradient gradually decreases, it may mean that the drone is moving away from the tracking target. Combining the magnetic field direction information and the drone's own attitude information (such as obtained through the gyroscope and accelerometer), the specific position of the tracking target relative to the drone can be determined. Then, based on the attenuation law of the magnetic field strength, the distance between the drone and the tracking target can be estimated.

[0079] S232. Build a three-dimensional environment model through the three-dimensional point cloud data detected by the drone, obtain the motion image collected by the visual sensor, use the target detection algorithm to capture and track the visual marker points of the target in the motion image in real time, and then identify the spatial coordinates of the visual marker points in the three-dimensional environment model.

[0080] Specifically, when the drone is flying in the air, the sensors such as the 3D laser scanner or depth camera will continuously scan the surrounding environment and generate a large amount of point cloud data, which contains the location, shape and other information of objects in the environment. The collected point cloud data is preprocessed, including denoising, filtering, registration and other steps to improve the accuracy and reliability of the data. The preprocessed point cloud data is clearer and more accurate, which is conducive to the subsequent construction of the 3D environment model.

[0081] Use professional 3D modeling software or geographic information system (GIS) software to import the pre-processed point cloud data into the software. Through the algorithm in the software, the point cloud data is converted into a 3D environment model, including surface reconstruction, texture mapping and other steps, and finally a realistic 3D environment model is generated.

[0082] The generated three-dimensional environment model is optimized to improve the accuracy and fluency of the model.

[0083] Next, select a suitable target detection algorithm (such as YOLO, SSD, Faster R-CNN, etc.) to capture the visual markers of the tracking target in real time in the moving image. For example, the deep learning algorithm needs to be trained in advance. During the training process, a large amount of labeled data is needed to teach the algorithm how to identify and locate the visual markers of the tracking target. During the flight of the drone, the collected moving images are input into the target detection algorithm. The algorithm will analyze the image in real time and output the position information of the visual markers of the tracking target. The visual marker position information output by the target detection algorithm is mapped to the three-dimensional environment model. Through the mapping process, the spatial coordinates of the visual markers of the tracking target in the three-dimensional environment model are determined.

[0084] S233. Based on the body position and spatial coordinates in the three-dimensional environment model, the spatial distance, heading angle and pitch angle between the tracking target and the body position are calculated, and the known magnetic field azimuth and magnetic field distance are integrated. Through spatial and magnetic field position correction, the target position of the tracking target is obtained.

[0085] Specifically, in the three-dimensional environment model, the spatial coordinates of the aircraft position and the tracking target are known, and the Euclidean distance formula is used to calculate the spatial distance between the two points. The heading angle refers to the angle between the projection of the line connecting the aircraft and the tracking target on the horizontal plane and the front of the aircraft, and the pitch angle refers to the angle between the line connecting the aircraft and the tracking target and the horizontal plane. The heading angle and pitch angle are calculated by operations such as dot product and cross product of vectors.

[0086] After fusing the spatial coordinates and magnetic field information, a consistency check is required to ensure that the target position obtained is reasonable in both spatial and magnetic field dimensions. It is necessary to ensure that the spatial position data and magnetic field data are represented in the same coordinate system and that the spatial position data and magnetic field data are acquired at the same time or at a similar time.

[0087] Combined with the known magnetic field orientation and distance, the magnetic field strength and the relative position of the target are used for correction. If the magnetic field orientation gives the direction of the tracked target, this direction is compared with the calculated heading angle and adjusted. By fusing the heading angle and pitch angle with the magnetic field orientation information, the calculation of the target position is corrected, and a weighted average or filtering algorithm (such as Kalman filtering) is used to integrate the spatial position and magnetic field information to improve accuracy. After correction, the final position of the tracked target is obtained, which can be used for subsequent navigation and control decisions.

[0088] S24. Continuously and periodically update the body position of the UAV during flight and the target position of the tracking target, and monitor the change of the magnetic field gradient between the UAV and the tracking target in real time, and plan the flight path of the UAV by predicting the future target position.

[0089] In the description of the present invention, the body position of the drone during flight and the target position of the tracking target are continuously and periodically updated, and the magnetic field gradient change between the drone and the tracking target is monitored in real time. By predicting the future target position, the flight path of the drone is planned, including the following steps:

[0090] S241. Preset target tracking conditions for the UAV to perform a tracking task. The target tracking conditions include a tracking distance range, a flight altitude range, a flight speed range, and a cruising time.

[0091] S242. In the tracking distance interval, the flight altitude interval and the flight speed interval, select the initial distance, the initial altitude and the initial speed, and set the tracking distance as the first priority, the flight altitude as the second priority and the flight speed as the third priority, giving priority to keeping the initial distance unchanged.

[0092] S243, record the target position of the tracking target at each update interval in real time, predict the target position of the tracking target at the next update interval, mark the position space point in the three-dimensional environment model, and plan the flight path of the UAV according to the initial distance and initial height.

[0093] In the description of the present invention, the target position of the tracking target at each update interval is recorded in real time, the target position of the tracking target at the next update interval is predicted, and the position space point is marked in the three-dimensional environment model. According to the initial distance and the initial height, the flight path of the UAV navigation is planned, including the following steps:

[0094] S2431. Record the historical target positions of the tracking target in the previous n update intervals to form a position sequence, establish a state transfer matrix, and discretize the movement state of the tracking target. Each movement state is represented as a combination of the speed and direction of the tracking target at a specific position.

[0095] S2432. According to the flight speed and body position of the UAV during the current update interval, the movement state of the tracking target during the current update interval is calculated, and then the movement state of the tracking target during the next update interval is predicted using the state transfer matrix.

[0096] S2433. Based on the predicted movement state, estimate the target position of the tracking target at the next update interval, mark the spatial point corresponding to the target position in the three-dimensional environment model, and calibrate the flight point of the UAV at the next update interval in the three-dimensional environment model according to the initial distance and initial height.

[0097] In the description of the present invention, the calculation formula for the target position at the next update interval is:

[0098]

[0099] In the formula, Indicates the tracking target at the next update interval x Axis coordinates; Indicates the tracking target at the next update interval y Axis coordinates; Indicates the tracking target at the next update interval z Axis coordinates; x t Indicates the tracking target at the current update interval x Axis coordinates; y t Indicates the tracking target at the current update interval y Axis coordinates; z t Indicates the tracking target at the current update interval z Axis coordinates; Indicates that the tracking target is in the prediction state x Velocity components on the axis; Indicates that the tracking target is in the prediction state y Velocity components on the axis; Indicates that the tracking target is in the prediction state z Velocity components on the axis; Indicates the update interval.

[0100] S2434, sequentially connecting the flight points of the current update interval and the next update interval, and using the route formed by the flight points as the flight path for the UAV to perform the tracking task.

[0101] S3. Real-time detection of obstacles in the flight path, integration into the three-dimensional environment model, combined with the follow-up visual tracking algorithm, capture the motion state of the tracking target, and dynamically adjust the control parameters and flight attitude of the UAV based on the obstacle distribution and motion state changes.

[0102] In the description of the present invention, real-time detection of obstacles in the flight path, integration into the three-dimensional environment model, combined with the follow-up visual tracking algorithm, capturing the motion state of the tracking target, and dynamically adjusting the control parameters and flight attitude of the drone based on the obstacle distribution and motion state changes include the following steps:

[0103] S31. Monitor the environmental data around the flight path in real time through airborne sensors, identify existing obstacles, and calculate the size, distance and position of the obstacles, generate a three-dimensional virtual model of the obstacles, then import the three-dimensional environmental model and dynamically update the three-dimensional environmental model.

[0104] S32. Based on the spatial distribution of obstacles, the flyable area and the obstacle area are divided in the three-dimensional environment model, and the spatial area where the flight path is located is detected. If there is an obstacle area, the flight path is optimized by adjusting the control parameters of the UAV.

[0105] In the description of the present invention, based on the spatial distribution of obstacles, the flyable area and the obstacle area are divided in the three-dimensional environment model, and the spatial area where the flight path is located is detected. If there is an obstacle area, the flight path is optimized by adjusting the control parameters of the drone, including the following steps:

[0106] S321, according to the distribution of obstacles in the three-dimensional environment model, the three-dimensional model is divided into a flyable area and an obstacle area, and whether there is an obstacle area in the current flight path is detected, if there is no obstacle area, the original flight path is maintained. If there is an obstacle area, a path adjustment instruction is issued.

[0107] S322. Based on the pre-set priority principle, the initial distance between the UAV and the tracking target is maintained unchanged. The flight speed of the UAV is first adjusted. If the speed cannot avoid the obstacle, the flight altitude of the UAV is then adjusted to move to a passable flight altitude. If the altitude cannot avoid the obstacle, the horizontal position of the UAV is adjusted to move to a flight position that can be circumvented.

[0108] Specifically, by setting the priority principle, the distance between the drone and the tracking target can be ensured to remain unchanged to maintain effective tracking. In path planning, the current distance to the target is calculated and a distance threshold is set. If an obstacle is detected, the flight parameters are adjusted.

[0109] 1. Avoid obstacles by changing flight speed

[0110] Implementation:

[0111] Reduce speed: If the current flight speed is high, try to slow down, for example, reduce the speed from 10 m / s to 5 m / s to increase the reaction time to obstacles.

[0112] Example: Assume that the drone detects a high voltage power line ahead and its current speed is 10 m / s. The adjusted speed is 5 m / s, giving the drone more time to evaluate and adjust its flight path.

[0113] 2. Adjusting the flight altitude

[0114] Goal: If speed adjustments are not sufficient to avoid an obstacle, try changing flight altitude.

[0115] Implementation:

[0116] Raise or lower the altitude: Adjust the flight altitude based on the altitude information of obstacles. For example, adjust the altitude from 50 meters to 70 meters to avoid trees or buildings.

[0117] Example: If after reducing speed, the drone still cannot avoid high-voltage power lines, adjust the flight altitude to ensure that the drone rises to 70 meters so that it can pass safely.

[0118] 3. Adjust the horizontal position

[0119] Goal: If the altitude adjustment still cannot avoid the obstacle, finally ensure safe flight by adjusting the horizontal position.

[0120] Implementation:

[0121] Lateral movement: Based on the new altitude, select a new heading or flight path to bypass obstacles.

[0122] Example: When a drone cannot avoid an obstacle at an altitude of 70 meters, adjusting its horizontal position may offset it by 5 meters from the target path and fly it in a safe direction.

[0123] S323. According to the UAV path adjustment result, the flight path is reconstructed using the path planning algorithm, and the control parameters of the UAV are adjusted to achieve the purpose of optimizing the path.

[0124] S33. Detect the tracking target in the acquired motion image, and use the follow-up visual tracking algorithm to capture the motion state of the tracking target, and then lock the tracking target in the core area of ​​the motion image by adjusting the flight attitude of the drone.

[0125] In the description of the present invention, in the captured motion image, the tracking target is detected, and the motion state of the tracking target is captured by using a follow-up visual tracking algorithm, and then the tracking target is locked in the core area of ​​the motion image by adjusting the flight attitude of the drone, including the following steps: S331, setting the core area of ​​the motion image, selecting a visual tracking algorithm, tracking the position change of the tracking target in the motion image frame by frame, and updating the motion state of the tracking target in real time according to the position change of the tracking target, the motion state including pixel position, speed and acceleration.

[0126] S332. The difference between the current pixel position of the tracking target and the center point of the core area is taken as an error. According to the error value, the flight attitude of the UAV is adjusted using a proportional-integral-differential control algorithm. The flight attitude includes the roll, pitch angle and yaw angle of the UAV.

[0127] Among them, the calculation formula of the proportional-integral-differential control algorithm is:

[0128]

[0129] In the formula, u ( t ) represents control input; e ( t) represents the error value of the target position relative to the center of the core area; t Indicates time; K p , K i , K d Represent the proportional, integral, and differential control gains respectively.

[0130] Specifically, according to the error value (Δ x , Δ y ), adjust the drone's attitude (roll, pitch and yaw) and position (forward, backward, left and right):

[0131] If Δ x If the value is positive, the target is on the right side of the image, and the flight direction of the drone needs to be adjusted to the left.

[0132] If Δ x If it is negative, the target is on the left side of the image, and the flight direction needs to be adjusted to the right.

[0133] If Δ y If it is positive, the target is above the image, and the drone needs to lower its flight altitude.

[0134] If Δ y If it is negative, the target is below the image and the drone needs to be raised.

[0135] By adjusting the flight attitude and position of the drone, the target gradually returns to the center area of ​​the image.

[0136] In addition, while adjusting the drone's attitude, dynamic path planning is performed in combination with the target's movement trend to ensure that the drone can not only keep the target in the center area, but also fly along the target's movement trajectory.

[0137] S4: Save and feedback target tracking data in real time, monitor the battery status and energy consumption rate of the drone, control the remaining flight time according to the drone's navigable mileage, and return to the preset safe area before the energy is completely consumed to end the target tracking mission.

[0138] Specifically, the battery monitoring module is used to track parameters such as battery voltage, remaining power, and temperature in real time to evaluate the battery's health status and remaining energy. According to the flight mode, speed, and operating conditions of the drone, the current energy consumption rate is calculated to predict the remaining flight time under the current state. Then, based on the remaining battery power and energy consumption rate, the drone's flight range is estimated to determine whether the target tracking task can be completed without running out of power.

[0139] If the battery power reaches the preset safety threshold, or the remaining flight time is insufficient to support continuous tracking, the flight path will be adjusted immediately to plan a safe return route to ensure that the drone can return to the preset safety area before the energy is completely consumed. In addition, multiple safety areas can be set, and the drone will give priority to the nearest safety area for return and landing during navigation.

[0140] See also Figure 2 , and also provides a flight control system for UAV tracking technology, the system comprising:

[0141] Mission configuration module 1 is used to start the onboard sensor before the drone takes off, perform parameter initialization and calibration operations, and install magnetic markers on the tracking target. After the drone takes off, the onboard sensor is used to capture the magnetic field signal and motion image of the tracking target in real time.

[0142] Tracking planning module 2 is used to obtain the body position of the UAV during flight, adopt a multi-sensor data fusion algorithm to perceive the target position of the tracking target in real time, and plan the flight path of the UAV using magnetic guidance technology based on the relative position relationship between the tracking target and the UAV.

[0143] The attitude control module 3 is used to detect obstacles in the flight path in real time, integrate into the three-dimensional environment model, and combine with the follow-up visual tracking algorithm to capture the motion state of the tracking target, and dynamically adjust the control parameters and flight attitude of the UAV based on the obstacle distribution and motion state changes.

[0144] The monitoring return module 4 is used to save and feedback the target tracking data in real time, monitor the battery status and energy consumption rate of the drone, control the remaining flight time according to the drone's navigable mileage, and return to the preset safe area before the energy is completely consumed to end the target tracking task.

[0145] In summary, with the help of the above technical solutions of the present invention, by combining magnetic guidance technology, multi-sensor data fusion and follow-up visual tracking algorithm, high-precision target tracking can be achieved in dynamic and complex environments, while effectively avoiding obstacles and ensuring the flight stability and tracking accuracy of the drone; compared with the traditional solution, the present invention optimizes the accuracy of target recognition and position perception, especially in changing scenes, the flight path can be dynamically adjusted, significantly improving the real-time and intelligence of the tracking task; in addition, the present invention also optimizes the energy management strategy, effectively prolongs the endurance time, and reduces the risk of the drone running out of power during the task. By adopting a multi-sensor data fusion algorithm, accurate real-time perception of the position of the drone and the tracking target is achieved; the relative distance between the drone and the tracking target is obtained by magnetic field gradient analysis, and combined with spatial detection technology and position correction mechanism, the tracking accuracy and path planning flexibility are effectively improved; in addition, the future target position prediction is used for path planning, so that the drone can adjust the flight route more intelligently, thereby maintaining stable tracking in a complex environment, and improving the response speed and tracking efficiency of the system; the real-time positioning and adaptability in a dynamic environment are improved, the position error is reduced, and the robustness and endurance efficiency of the system are improved.

[0146] By real-time monitoring of obstacles in the flight path, combined with a three-dimensional environmental model and a follow-up visual tracking algorithm, it is possible to automatically identify obstacles in complex environments and dynamically adjust the flight path to ensure the safety and efficiency of the drone. The present invention improves the autonomous obstacle avoidance capability of the drone through three-dimensional space modeling and division, accurate identification and dynamic adjustment of the flyable area and obstacle area; at the same time, the follow-up visual tracking algorithm further improves the accuracy of target capture and tracking, ensuring that the tracked target is always within the drone's line of sight, thereby improving the reliability and intelligence of the flight mission.

[0147] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

Claims

1. A flight control method for drone tracking technology, characterized in that: The method comprises the following steps: S1. Before the drone takes off, start the onboard sensor in advance, perform parameter initialization and calibration operations, and install magnetic markers on the tracking target; after the drone takes off, the onboard sensor is used to capture the magnetic field signal and motion image of the tracking target in real time; S2. Obtain the position of the drone during flight, use a multi-sensor data fusion algorithm to perceive the target position of the tracking target in real time, and use magnetic guidance technology to plan the flight path of the drone based on the relative position relationship between the tracking target and the drone; specifically include: S21, obtaining the current geographical coordinates of the drone through a positioning sensor in a built-in airborne sensor of the drone, and then obtaining the acceleration and angular velocity of the drone using an inertial sensor; S22, setting an update interval for the drone, recording the geographic location coordinates, acceleration and angular velocity at each update interval, and then calculating the drone's body position at the current coordinate update interval by combining the body position at the previous update interval and the acceleration and angular velocity at the current update interval; S23, obtaining the magnetic field signal of the tracking target, identifying the interval distance between the UAV and the tracking target through magnetic field gradient analysis, and identifying the spatial coordinates of the tracking target in combination with three-dimensional space detection, and obtaining the target position of the tracking target through data fusion and position correction; S24, continuously and periodically updating the body position of the UAV during flight and the target position of the tracking target, and monitoring the magnetic field gradient change between the UAV and the tracking target in real time, and planning the flight path of the UAV by predicting the future target position; comprising the following steps: S241, presetting target tracking conditions for the UAV to perform a tracking task, wherein the target tracking conditions include a tracking distance interval, a flight altitude interval, a flight speed interval, and a cruising time; S242, selecting an initial distance, an initial altitude, and an initial speed from among the tracking distance interval, the flight altitude interval, and the flight speed interval, and setting the tracking distance as the first priority, the flight altitude as the second priority, and the flight speed as the third priority, with the initial distance being kept unchanged first; S243, recording the target position of the tracking target at each update interval in real time, predicting the target position of the tracking target at the next update interval, marking the position space point in the three-dimensional environment model, and planning the flight path of the UAV according to the initial distance and initial height; comprising the following steps: S2431, record the historical target position of the tracking target in the previous n update intervals, form a position sequence, establish a state transfer matrix, and discretize the movement state of the tracking target, each movement state is represented as a combination of the speed and direction of the tracking target at a specific position; S2432, calculating the movement state of the tracking target at the current update interval according to the flight speed and body position of the UAV at the current update interval, and then using the state transfer matrix to predict the movement state of the tracking target at the next update interval; S2433, based on the predicted movement state, estimating the target position of the tracking target at the next update interval, marking the spatial point corresponding to the target position in the three-dimensional environment model, and calibrating the flight point of the UAV at the next update interval in the three-dimensional environment model according to the initial distance and initial height; S2434, sequentially connect the flight points of the current update interval and the next update interval, and use the route formed by the flight points as the flight path for the UAV to perform the tracking task; The calculation formula for the target position at the next update interval is: ； In the formula, Indicates the tracking target at the next update interval x Axis coordinates; Indicates the tracking target at the next update interval y Axis coordinates; Indicates the tracking target at the next update interval z Axis coordinates; x t Indicates the tracking target at the current update interval x Axis coordinates; y t Indicates the tracking target at the current update interval y Axis coordinates; z t Indicates the tracking target at the current update interval z Axis coordinates; Indicates that the tracking target is in the prediction state x Velocity components on the axis; Indicates that the tracking target is in the prediction state y Velocity components on the axis; Indicates that the tracking target is in the prediction state z Velocity components on the axis; Indicates the update interval; S3, real-time detection of obstacles in the flight path, integration into the three-dimensional environment model, combined with the follow-up visual tracking algorithm, capture the motion state of the tracking target, and dynamically adjust the control parameters and flight attitude of the drone based on the obstacle distribution and motion state changes; S4: Save and feedback target tracking data in real time, monitor the battery status and energy consumption rate of the drone, control the remaining flight time according to the drone's navigable mileage, and return to the preset safe area before the energy is completely consumed to end the target tracking mission.

2. The flight control method of the drone tracking technology according to claim 1, characterized in that: The method of obtaining the magnetic field signal of the tracking target, identifying the interval distance between the UAV and the tracking target through magnetic field gradient analysis, identifying the spatial coordinates of the tracking target through three-dimensional space detection, and obtaining the target position of the tracking target through data fusion and position correction includes the following steps: S231, using the magnetometer of the drone's built-in airborne sensor to read the magnetic field signal generated by the magnetic marker configured for the tracking target in real time, and by analyzing the magnetic field strength and magnetic field direction of the magnetic field signal, detecting the change in the magnetic field gradient, and identifying the magnetic field direction and magnetic field distance of the tracking target; S232, constructing a three-dimensional environment model through the three-dimensional point cloud data detected by the drone, obtaining the motion image collected by the visual sensor, using the target detection algorithm, capturing and tracking the visual marker points of the target in the motion image in real time, and then identifying the spatial coordinates of the visual marker points in the three-dimensional environment model; S233. Based on the body position and spatial coordinates in the three-dimensional environment model, the spatial distance, heading angle and pitch angle between the tracking target and the body position are calculated, and the known magnetic field azimuth and magnetic field distance are integrated. Through spatial and magnetic field position correction, the target position of the tracking target is obtained.

3. The flight control method of the drone tracking technology according to claim 2 is characterized in that: The real-time detection of obstacles in the flight path, integration into the three-dimensional environment model, combined with the follow-up visual tracking algorithm, capturing the motion state of the tracking target, and dynamically adjusting the control parameters and flight attitude of the drone based on the obstacle distribution and motion state changes include the following steps: S31, monitoring the environmental data around the flight path in real time through the airborne sensor, identifying the existing obstacles, and calculating the size, distance and position of the obstacles, generating a three-dimensional virtual model of the obstacles, and then importing the three-dimensional environment model to dynamically update the three-dimensional environment model; S32, based on the spatial distribution of obstacles, dividing the flyable area and the obstacle area in the three-dimensional environment model, and detecting the spatial area where the flight path is located. If there is an obstacle area, the flight path is optimized by adjusting the control parameters of the UAV; S33, detecting the tracking target in the acquired motion image, and using a follow-up visual tracking algorithm to capture the motion state of the tracking target, and then locking the tracking target in the core area of ​​the motion image by adjusting the flight attitude of the drone.

4. The flight control method of the drone tracking technology according to claim 3 is characterized in that: The method of dividing the flyable area and the obstacle area in the three-dimensional environment model based on the spatial distribution of obstacles and detecting the spatial area where the flight path is located, and optimizing the flight path by adjusting the control parameters of the drone if an obstacle area exists, includes the following steps: S321, according to the distribution of obstacles in the three-dimensional environment model, the three-dimensional model is divided into a flyable area and an obstacle area, and whether there is an obstacle area in the current flight path is detected. If there is no obstacle area, the original flight path is maintained; if there is an obstacle area, a path adjustment instruction is issued; S322. Based on a pre-set priority principle, the initial distance between the UAV and the tracking target is maintained unchanged. The flight speed of the UAV is first adjusted. If the speed adjustment cannot avoid the obstacle, the flight altitude of the UAV is then adjusted to move to a passable flight altitude. If the altitude adjustment cannot avoid the obstacle, the horizontal position of the UAV is adjusted to move to a flight position that can be circumvented. S323. According to the UAV path adjustment result, the flight path is reconstructed using the path planning algorithm, and the control parameters of the UAV are adjusted to achieve the purpose of optimizing the path.

5. The flight control method of the drone tracking technology according to claim 4 is characterized in that: The method of detecting the tracking target in the acquired motion image, capturing the motion state of the tracking target by using a follow-up visual tracking algorithm, and then locking the tracking target in the core area of ​​the motion image by adjusting the flight attitude of the drone includes the following steps: S331, setting the core area of ​​the motion image, selecting a visual tracking algorithm, tracking the position change of the tracking target in the motion image frame by frame, and updating the motion state of the tracking target in real time according to the position change of the tracking target, wherein the motion state includes pixel position, speed and acceleration; S332. The difference between the current pixel position of the tracking target and the center point of the core area is taken as an error. According to the error value, the flight attitude of the UAV is adjusted using a proportional-integral-differential control algorithm. The flight attitude includes the roll, pitch angle and yaw angle of the UAV.

6. A flight control system for drone tracking technology, used to implement the flight control method for drone tracking technology according to any one of claims 1 to 5, characterized in that: The system includes: The mission configuration module is used to start the onboard sensors before the drone takes off, perform parameter initialization and calibration operations, and install magnetic markers on the tracking target; after the drone takes off, the onboard sensors are used to capture the magnetic field signals and motion images of the tracking target in real time; The tracking planning module is used to obtain the position of the drone during flight, use a multi-sensor data fusion algorithm to perceive the target position of the tracking target in real time, and use magnetic guidance technology to plan the flight path of the drone based on the relative position relationship between the tracking target and the drone; The attitude control module is used to detect obstacles in the flight path in real time, integrate the three-dimensional environment model, and combine the follow-up visual tracking algorithm to capture the motion state of the tracking target. Based on the obstacle distribution and motion state changes, the control parameters and flight attitude of the UAV are dynamically adjusted; The monitoring return module is used to save and feedback target tracking data in real time, monitor the battery status and energy consumption rate of the drone, control the remaining flight time according to the drone's navigable mileage, and return to the preset safe area before the energy is completely consumed to end the target tracking mission.

Images