A three-stage unmanned aerial vehicle end flight guidance tracking system and method

Abstract

This invention discloses a three-stage UAV terminal flight guidance and tracking system and method, relating to the field of UAV technology. It includes an analog camera module, a digital-to-analog converter chip module, a computing chip module, an OSD chip module, a flight controller module, a data transmission module, an analog image transmission module, a remote controller module, and an FPV glasses module. Combining current technological and human resources conditions, this invention uses an FPV UAV target detection and tracking module as an anchor point to construct an independent and controllable integrated hardware and software UAV flight guidance system. This system possesses high robustness, high precision, and high synergy with the flight control system, effectively filling existing technological gaps, meeting the advanced needs of various industries for intelligent UAV flight guidance operations, promoting the popularization and upgrading of UAV flight guidance technology, and laying a solid foundation for subsequent intelligent UAV flight guidance product projects. It has significant technological value and promising industrial application prospects.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, specifically to a three-stage UAV terminal flight guidance and tracking system and method. Background Technology

[0002] With the rapid integration of drone technology and computer vision technology, drones are being used more and more widely in fields such as security monitoring, emergency rescue, power line inspection, and military reconnaissance. As a core component of intelligent drone operations, the performance of the drone flight guidance system directly determines the efficiency, reliability, and safety of drone operations. Target recognition and tracking are the core functions of the drone flight guidance system. Its recognition accuracy, tracking stability, and coordination with the flight control system directly affect the accuracy of flight guidance and the effectiveness of operations. It is a key prerequisite for achieving precise drone flight guidance and completing various complex tasks. Currently, existing drone flight guidance products on the market generally suffer from numerous technical pain points in their core target recognition and tracking functions, making it difficult to meet the high-level requirements of various industries for high-precision and high-reliability drone flight guidance. Specifically: Weak system coordination and insufficient guidance reliability: In existing flight guidance systems, the coordination between visual recognition and tracking algorithms and UAV flight control systems is weak, and data interaction is delayed, which can easily lead to problems such as tracking lag and target loss. As a result, flight guidance commands cannot be transmitted in a timely and accurate manner, affecting the reliability of flight guidance and failing to guarantee the continuity of operational tasks. High product costs limit large-scale promotion: Drone flight guidance products with target recognition and tracking functions are expensive, making it difficult to promote and apply them on a large scale. This restricts the popularization of drone flight guidance technology in various fields such as civilian and industrial applications, and fails to fully unleash the value of intelligent drone operations. Summary of the Invention

[0003] This invention provides a three-stage unmanned aerial vehicle (UAV) terminal flight guidance and tracking system and method, which can effectively solve the problems of weak system coordination, insufficient guidance reliability, high product cost, and limited large-scale promotion mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a three-stage UAV terminal flight guidance and tracking system, comprising an analog camera module, a digital-to-analog converter chip module, a computing chip module, an OSD chip module, a flight controller module, a data transmission module, an analog image transmission module, a remote controller module, and an FPV glasses module; The analog camera module is responsible for capturing real-time images in front of the drone, the digital-to-analog converter chip module is responsible for converting analog signals to digital signals, the computing chip module is responsible for running target detection algorithms, target tracking algorithms and attitude control algorithms, and the OSD chip module is responsible for superimposing the position information of the target tracking results onto the analog video signal. The flight controller module is the core control unit of the UAV, the data transmission module is responsible for establishing wireless communication, the analog image transmission module is responsible for transmitting real-time video signals collected by the analog camera, the remote controller module is used by the pilot to control the UAV, and the FPV glasses module is the ground station display device.

[0005] According to the above technical solution, the analog camera module is adapted to a traditional FPV drone analog camera, and the camera's analog signal is divided into two paths. One path converts the image signal into a digital signal through an analog-to-digital converter chip and transmits it to a computing chip for image signal processing. The other path directly transmits the analog signal to an analog image transmission chip, which then transmits the real-time drone image to the FPV glasses on the ground, ensuring the real-time performance of the image in the glasses. The analog-to-digital converter chip module converts the analog video stream output by the analog camera into a digital signal that can be processed by the computing chip. The analog-to-digital converter chip module receives one signal from the analog camera, completes the digital conversion, and outputs it to the computing chip.

[0006] According to the above technical solution, the computing chip module integrates target detection algorithm, target tracking algorithm and attitude data derivation algorithm; The target detection algorithm uses the YOLO series algorithm, the target tracking algorithm uses the KCF algorithm, and the attitude data inference algorithm is used in Auto mode to derive attitude data by combining the UAV's own sensor data and through the collaborative inference of multiple algorithms.

[0007] According to the above technical solution, the OSD chip module converts the target coordinate information output by the target tracking algorithm into display instructions that the OSD chip can recognize. After achieving time synchronization with the original video signal, visual markers are superimposed at the specified screen position, and finally the video with target position markers is output. The original video data is not destroyed throughout the process, and the superposition is only completed in the display stage. The core process includes four key steps: position information reception and parsing, video time synchronization, marker generation and pixel mixing, and superimposed video output.

[0008] According to the above technical solution, the flight controller module is the core control unit of the UAV, responsible for the attitude stabilization and flight control of the UAV, and undertakes two key functions: First, the target detection and tracking commands transmitted from the remote controller are captured through the data transmission module and transmitted to the computing chip. Second, upon receiving the Auto command, control is immediately switched to the computing chip, and the drone's own attitude data is sent to the computing chip. The computing chip calculates the drone's attitude data for the next moment based on the position of the target object in the image and the drone's attitude data at this time, and sends this attitude data to the drone to realize the drone's tracking and approach of the target object. The data transmission module is responsible for communication between the remote controller module and the flight controller module, transmitting key control information; The analog image transmission module is responsible for transmitting the real-time video signal collected by the analog camera to the FPV glasses module on the ground. It works in conjunction with the OSD chip. The video signal with the target position mark superimposed on it by the OSD chip is transmitted to the ground equipment through the analog image transmission link.

[0009] According to the above technical solution, the remote controller module is the operating device for the pilot to control the drone. It controls the drone and selects targets by sending control commands. The pilot selects the target to be tracked in sequence through remote sensing operation, or moves and adjusts the target frame position to select the target object by using the joystick and dial operation. After confirming the target, the remote controller sends a confirmation signal and an Auto command to trigger the target tracking algorithm to start and the flight mode to switch, respectively. The FPV glasses module is a ground station display device responsible for receiving and displaying real-time images of the UAV and target location information superimposed on the OSD. The pilot observes the UAV flight footage, target detection results, and target tracking markers through the FPV glasses, enabling real-time monitoring of the target selection and tracking process.

[0010] According to the above technical solution, a three-stage UAV terminal flight guidance and tracking method includes the following steps: S1, Unmanned Aerial Vehicle System Power-On Self-Test Initialization; S2, Target Detection and Target Selection; S3, automatic tracking and approach; The specific steps of S1 are as follows: S11, Flight control system initialization; S12, FPV system initialization; S13, Computing chip initialization; S14, OSD chip initialization; S2 includes target detection manual selection adsorption and manual box selection adsorption; The specific steps for manually selecting adsorption for target detection are as follows: S211, UAV takeoff and target observation; S212, Activate target detection function; S213, Predefined target location calculation; S214, test results are displayed in overlay; S215, Manually select the target; S216, Entering the tracking phase; The specific steps for manually selecting and adsorbing are as follows: S221, Unmanned Aerial Vehicle Takeoff and Target Observation; S222, activate manual selection function; S223, Adjust the target bounding box; S224, Confirm the selected target; S225, Target Verification and Tracking Initiated; S226, Entering the tracking phase; The specific steps of S3 are as follows: S31, activate Auto mode; S32, Control Transfer; S33, acquire multi-source data; S34, Coordinate Transformation and Direction Angle Calculation; S35, Attitude error PI correction; S36, PID attitude control; S37, control command output and attitude adjustment.

[0011] According to the above technical solution, in step S11, after the UAV system is powered on, the flight control system is initialized and the remote controller displays that it is ready for takeoff. In step S12, after completing the flight control initialization, the FPV system initialization is started, and the real-time image of the UAV camera can be seen. In step S13, after the FPV system initialization is completed, the computing chip initialization is started, and the computing chip initialization outputs an initialization success signal to the OSD chip. In step S14, after receiving the computing chip initialization success signal, the OSD chip initialization is started. After the OSD chip initialization is completed, the computing chip initialization success signal is loaded onto the video stream, and the computing chip initialization success is displayed in the FPV glasses.

[0012] According to the above technical solution, in step S211, the pilot operates the drone to take off and observes the target object through FPV goggles; In S212, the remote controller sends a target detection signal to the computing chip through the flight control board, and the computing chip starts the target detection algorithm. In step S213, the target detection algorithm is run by the computing chip to calculate the position of the predefined target in the image; In step S214, the target position coordinates are added to the analog signal through the OSD chip, and the final detection result is displayed in the FPV glasses. In S215, the pilot selects the targets to be tracked sequentially via remote sensing. In step S216, after the target selection is completed, the process proceeds to step S3, where the selected target is locked and automatically tracked. In S221, the pilot takes off the drone and observes the target object through FPV goggles; In S222, the remote controller sends a manual box selection signal to the computing chip through the flight control board, and the computing chip controls the OSD to directly draw a square target box. In step S223, the size of the target frame is adjusted using the remote control's pulsator, and the remote control's joystick can move the position of the target frame. In S224, the pilot selects the target object by operating the joystick and the dial, and the remote controller sends a confirmation signal to the computing chip. In step S225, after receiving the data, the computing chip starts the target detection algorithm to detect whether the selected target is a predefined target. In step S226, after starting target tracking, the process proceeds to step S3 to perform automatic tracking.

[0013] According to the above technical solution, in S31, after the UAV achieves stable tracking during flight, the pilot initiates the Auto command, and the remote controller sends the Auto mode to the flight controller. In step S32, after receiving the Auto command, the flight controller forwards it to the computing chip. S33, the calculation chip acquires flight control attitude and tracking result data; S34 executes a coordinate transformation algorithm through a computing chip to convert the target pixel coordinates into a target orientation angle in the body coordinate system; In S35, the computing chip uses the drone camera pointing at the target as a reference to calculate the difference between the current attitude angle and the target's required attitude angle as the attitude error, eliminates the attitude error, and enables the drone's attitude to quickly converge to the target's required attitude. The S36 uses a PID control law to convert the corrected attitude angle into an angular velocity command, thereby achieving closed-loop attitude control. In S37, the computing chip sends the angular velocity command output by the PID control law to the UAV motor controller. The motor controller adjusts the speed of each motor according to the command, so that the UAV flies towards the target direction and controls the UAV to track and approach the target object.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention, combining current technological and human resources, uses an FPV drone target detection and tracking module as an anchor point to construct an independent and controllable integrated hardware and software drone flight guidance system. It involves the fusion of drone technology and computer vision technology, possessing high robustness, high precision, and high synergy with the flight control system. It effectively fills the gaps in existing technologies, solves the defects of existing flight guidance products, meets the advanced needs of various industries for intelligent drone flight guidance operations, promotes the popularization and upgrading of drone flight guidance technology, and lays a solid foundation for subsequent intelligent drone flight guidance product projects. It has significant technological value and promising industrial application prospects.

[0015] 2. This invention is compatible with the analog camera of traditional FPV drones. It splits the analog signal of the camera into two paths. One path is converted into a digital signal by an analog-to-digital converter chip and transmitted to a computing chip for image signal processing. The other path directly transmits the analog signal to an analog image transmission chip via an image transmission chip. The analog image transmission chip then transmits the real-time drone image to the FPV glasses on the ground, ensuring the real-time performance of the image in the glasses. At the same time, the flight controller captures the target detection and tracking commands from the remote controller through the data transmission module and transmits the commands to the computing chip. The computing chip starts the target detection and tracking algorithm and outputs the real-time position of the target object in the image. The position signal is sent to the OSD chip via a serial port. The OSD chip marks the target position on the analog image transmission signal, which is then finally displayed in the FPV glasses. After the pilot controls and fine-tunes the detected target object using the remote controller, the pilot sends an Auto command to the flight controller. The flight controller immediately switches control to the computing chip and simultaneously sends its own attitude data to the computing chip. Based on the target object's position in the image and the drone's attitude data at this moment, the computing chip calculates the drone's attitude data for the next moment and sends this attitude data to the drone to enable the drone to track and approach the target object.

[0016] 3. The system initialization in the first stage ensures that the hardware modules and software algorithms are ready. The second stage selects and locks onto the target to be tracked. Finally, the third stage enables the UAV to autonomously track and approach the target object through automatic tracking. By executing these three stages in sequence, a complete process from system startup to target tracking is formed. This reflects the design concept of software and hardware collaboration and the integration of algorithms and control, and provides a complete technical solution for intelligent flight guidance operations of UAVs. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0018] In the attached diagram: Figure 1 This is a structural diagram of the UAV terminal flight guidance and tracking system of the present invention; Figure 2 This is a flowchart of the power-on self-test initialization process of the UAV system of the present invention; Figure 3 This is a flowchart of the target detection and target selection process of the present invention; Figure 4 This is a flowchart of the automatic tracking and approaching process of the present invention; Figure 5 This is a diagram of the UAV tracking and control algorithm based on reinforcement learning proposed in this invention. Detailed Implementation

[0019] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0020] Example 1: like Figure 1 As shown, the present invention provides a technical solution, a three-stage UAV terminal flight guidance and tracking system, including an analog camera module, a digital-to-analog conversion chip module, a computing chip module, an OSD chip module, a flight controller module, a data transmission module, an analog image transmission module, a remote controller module, and an FPV glasses module; The analog camera module is responsible for capturing real-time images in front of the drone, the digital-to-analog converter chip module is responsible for converting analog signals to digital signals, the computing chip module is responsible for running target detection algorithms, target tracking algorithms, and attitude control algorithms, and the OSD chip module is responsible for superimposing the position information of the target tracking results onto the analog video signal. The flight controller module is the core control unit of the drone. The data transmission module is responsible for establishing wireless communication. The analog image transmission module is responsible for transmitting real-time video signals collected by the analog camera. The remote controller module is used by the pilot to control the drone. The FPV glasses module is the ground station display device.

[0021] Based on the above technical solution, the analog camera module is adapted to the traditional FPV drone analog camera, and the camera's analog signal is divided into two paths. One path converts the image signal into a digital signal through an analog-to-digital converter chip and transmits it to a computing chip for image signal processing. The other path directly transmits the analog signal to an analog image transmission chip, which then transmits the real-time drone image to the FPV glasses on the ground, ensuring the real-time performance of the image in the glasses. The analog-to-digital converter chip module converts the analog video stream output by the analog camera into a digital signal that can be processed by the computing chip. The analog-to-digital converter chip module receives one signal from the analog camera, completes the digital conversion, and outputs it to the computing chip to provide the required digital image data for subsequent computer vision algorithms.

[0022] Based on the above technical solution, the computing chip module integrates target detection algorithm, target tracking algorithm and attitude data derivation algorithm; The target detection algorithm adopts the YOLO series algorithm, specifically YOLOv8-nano. The YOLO series algorithm can achieve fast and accurate target recognition on embedded platforms with limited computing resources through its efficient real-time detection capability. The target tracking algorithm adopts the KCF kernel correlation filter algorithm. The core principle of the KCF algorithm can be summarized as follows: based on correlation filtering, nonlinear feature mapping is achieved through kernel trick, training samples are constructed using cyclic matrix, filter training and target detection are accelerated by Fourier transform, and finally, the online update mechanism is used to adapt to changes in the appearance of the target. Specifically, it can be broken down into 4 key parts: correlation filtering basis, kernel trick, cyclic matrix and Fourier acceleration, and online update mechanism. Correlation filtering basics: Its essence is to construct a target template filter so that the filter produces a high response when it is correlated with the features of the target region and a low response when it is correlated with the background region. The target position is located by the peak value of the response. In the time domain, the relevant operations can be represented as: ,in, It is the filter weight vector. These are the feature vectors of the candidate regions. To avoid overfitting, a regularization term is introduced, and a ridge regression objective function is constructed: ; in, It is a regularization coefficient used to balance fitting accuracy and model complexity. These are the sample labels. Solving this objective function yields the optimal filter weights. ; in It is the sample feature matrix. It is the sample label vector. Indicates conjugate transpose; Kernel Trick: Traditional correlation filtering can only handle linearly separable features, while the features of the target and background often exhibit non-linear distributions, including feature differences caused by illumination changes and target deformation. KCF introduces the kernel trick, which avoids directly mapping features to a high-dimensional space. Instead, it indirectly achieves non-linear modeling by calculating the inner product of features in the high-dimensional space through a kernel function, significantly reducing computational complexity. Commonly used kernel functions in KCF are the Gaussian kernel and the RBF kernel, which are defined as follows: ; in There are two eigenvectors. It is a parameter that controls the width of the kernel function and is used to adjust the weights for calculating feature similarity. In addition, linear kernels and polynomial kernels can also be selected, but Gaussian kernels have the strongest nonlinear expression ability and are the default choice for KCF. After introducing the kernel trick, the filter weights It can be represented as a linear combination of high-dimensional features: ; in It is a Lagrange multiplier. It is an eigenvector In the high-dimensional space mapping, the final objective function is transformed into a ridge regression problem in the kernel space, and the filter model can be determined by solving for α. Circular Matrices and Fourier Acceleration: One of the core principles of KCF is to utilize circular matrices to construct training samples, combining this with Fourier transform to convert time-domain convolution operations into frequency-domain multiplication operations, thus reducing computational complexity from... Reduce to N is the sample dimension, which is the key to its real-time tracking. Specifically, KCF uses the initial target region as a reference and generates a large number of positive and negative samples through cyclic shifting. These samples can form a cyclic matrix. According to linear algebra theory, the cyclic matrix can be diagonalized through Fourier transform, that is, the inverse matrix and product operation of the cyclic matrix can be quickly implemented in the frequency domain. After transforming the ridge regression objective function to the frequency domain, the optimal solution can be simplified to: ; in , , They are Kernel matrix The Fourier transform result, * denotes conjugate. The regularization coefficient λ is usually taken as 1e-4-1e-2 to prevent the filter from overfitting. This transformation greatly simplifies the calculation and significantly improves the training and detection speed of the filter. Online update mechanism: To adapt to dynamic changes in the target's appearance, KCF employs an online update mechanism to dynamically adjust filter parameters, avoiding model drift. The update process uses a linear interpolation strategy to balance the weights of historical models and the current frame model, as shown in the following formula: ; ; in, This is the update rate, ranging from 0.01 to 0.05, balancing model stability and adaptability. , These are the filter coefficients and target features of the current frame. , These are the corresponding parameters from the previous frame. An excessively high update rate can easily cause model drift, while an excessively low update rate will fail to adapt to changes in the target. Adjustments need to be made based on the actual scenario. In Auto mode, the core task of the computing chip is to use the target coordinates as input, combine the UAV's own sensor data, and use multiple algorithms to collaboratively derive attitude data to achieve attitude stability control when the UAV autonomously aligns with the target, tracks the target, or plans a path according to the target coordinates.

[0023] Based on the above technical solution, the OSD chip module converts the target coordinate information output by the target tracking algorithm into display instructions that the OSD chip can recognize. After achieving time synchronization with the original video signal, visual markers, including rectangles and coordinate values, are superimposed at the specified screen position. Finally, the video with the target position marker is output. The original video data is not destroyed throughout the process. The superposition is only completed in the display stage. In essence, it is a closed-loop process of position information parsing, time synchronization, and pixel mixing. The output format of the target tracking results is as follows: After processing video frames, the target tracking algorithm outputs the target's position coordinates, which is the basis for OSD chip overlay. Common output formats include: Screen pixel coordinates: Directly outputs the pixel position of the target in the video frame, usually with the top left corner of the video frame as the origin (0,0), in the format (X1,Y1,X2,Y2), corresponding to the top left and bottom right corner coordinates of the target rectangle, X1=100,Y1=80,X2=200,Y2=180; Relative coordinates: Outputs the proportional coordinates of the target relative to the video frame, suitable for adapting videos of different resolutions; Auxiliary information: Some algorithms will simultaneously output the target ID and tracking confidence, which can be displayed together via OSD overlay; The OSD chip first needs to receive this coordinate information, then parse it through the internal control unit and convert it into display instructions that it can execute, including the mark position, mark style, and color; The core process includes four key steps: location information reception and parsing, video timing synchronization, marker generation and pixel mixing, and video output after overlay. The process of OSD chip overlaying target tracking position is deeply integrated with its own video processing flow, and the whole process is highly real-time. The specific steps are as follows. Location information reception and parsing: The target tracking algorithm sends the target's coordinate information (X1, Y1, X2, Y2) to the OSD chip's control register in real time via the I2C communication interface. The timing and control unit inside the OSD chip parses the received coordinate data and completes two key transformations: First, the coordinates output by the algorithm are converted into absolute pixel coordinates supported by the OSD chip to match the resolution of the video frame. Second, according to preset rules and user-defined marking styles, dot matrix data of the target location markers are generated, including the line dot matrix of the rectangle and the character dot matrix of the coordinate values. Video timing synchronization: To ensure that the superimposed position markers are perfectly aligned with the targets in the video, the OSD marker output must be strictly synchronized with the original video signal. This is the core to avoid marker drift and tearing. The OSD chip extracts the HSYNC and VSYNC signals from the input original video signal through an internal synchronization demultiplexer, and uses them as a timing reference. Field Synchronization (VSYNC): Synchronizes the switching of video frames to ensure that each video frame corresponds to the latest target coordinates and avoids frame misalignment; HSYNC (Horizontal Synchronization): Synchronizes the scanning of video frames line by line, ensuring that each pixel of the target marker is accurately superimposed on the corresponding line position of the video frame; At the same time, the OSD chip calculates the row and column positions of the marker in the video frame based on the parsed target coordinates, and locks the scanning timing through the timing generator to ensure that the marker is aligned with the target in the same frame, row, and pixel.

[0024] Marker generation and pixel blending: After synchronization is complete, the OSD generation core of the OSD chip generates a pixel stream of markers for the target location based on the parsed coordinates and preset styles. For simple marking: directly generate line dot matrix at the corresponding position, without complex calculations; If it is a coordinate value: the corresponding number and letter dot matrix is ​​retrieved from the character ROM inside the chip, combined into coordinate text, and then located near the target; Subsequently, pixel-level mixing is completed through the mixer of the OSD chip, and the core logic is the same as that of OSD overlaying ordinary information: When the video scans to the target location area, the mixer mixes the OSD-marked pixels with the original video pixels. For areas not involving the target location, the mixer passes directly to the original video pixels without making any modifications. Video output after overlay: After the video signal is mixed, it is amplified by the OSD chip's drive circuit, converted to digital-to-analog by DAC or directly output digitally, and sent to the display terminal. The user can then see the real-time video with target tracking markers. The whole process is synchronized with the video frame rate. Each frame of video receives the latest coordinates, regenerates the markers, and completes the overlay to ensure that the tracking markers can follow the target's movement in real time.

[0025] Based on the above technical solution, the flight controller module is the core control unit of the UAV, responsible for the attitude stabilization and flight control of the UAV, and undertakes two key functions: First, the target detection and tracking commands transmitted from the remote controller are captured through the data transmission module and transmitted to the computing chip. Second, upon receiving the Auto command, control is immediately switched to the computing chip, and the drone's own attitude data is sent to the computing chip. The computing chip calculates the drone's attitude data for the next moment based on the position of the target object in the image and the drone's attitude data at this time, and sends this attitude data to the drone to realize the drone's tracking and approach of the target object. The data transmission module is responsible for communication between the remote controller module and the flight controller module, transmitting key control information including target detection, tracking commands, and Auto mode switching commands. The analog image transmission module is responsible for transmitting the real-time video signal captured by the analog camera to the FPV glasses module on the ground, ensuring that the pilot can view the drone's perspective in real time. Working in conjunction with the OSD chip, the video signal with the target position mark superimposed on it is transmitted to the ground equipment through the analog image transmission link.

[0026] Based on the above technical solution, the remote controller module is the operating device for the pilot to control the drone. It controls the drone and selects targets by sending control commands. The pilot selects the target to be tracked in sequence through remote sensing operation, or moves and adjusts the target frame position to select the target object through joystick and dial operation. After confirming the target, the remote controller sends a confirmation signal and an Auto command to trigger the target tracking algorithm to start and the flight mode to switch, respectively. The FPV glasses module is a ground station display device responsible for receiving and displaying real-time images of the drone and target location information overlaid on the OSD. The pilot observes the drone's flight footage, target detection results, and target tracking markers through the FPV glasses, enabling real-time monitoring of the target selection and tracking process.

[0027] like Figure 2-4 As shown, a three-stage UAV terminal flight guidance and tracking method includes the following steps: S1, Unmanned Aerial Vehicle System Power-On Self-Test Initialization; S2, Target Detection and Target Selection; S3, automatic tracking and approach; S1 involves performing initialization checks on each module sequentially after the UAV system is powered on. The specific steps are as follows: S11, Flight control system initialization; S12, FPV system initialization; S13, Computing chip initialization; S14, OSD chip initialization; S2 allows the drone operator to take off and select target objects using two selectable modes: target detection manual selection and snap-in, and manual box selection and snap-in. The specific steps for manually selecting adsorption for target detection are as follows: S211, UAV takeoff and target observation; S212, Activate target detection function; S213, Predefined target location calculation; S214, test results are displayed in overlay; S215, Manually select the target; S216, Entering the tracking phase; The specific steps for manually selecting and adsorbing are as follows: S221, Unmanned Aerial Vehicle Takeoff and Target Observation; S222, activate manual selection function; S223, Adjust the target bounding box; S224, Confirm the selected target; S225, Target Verification and Tracking Initiated; S226, Entering the tracking phase; S3 involves the pilot activating the Auto command after target tracking has stabilized, enabling the drone to autonomously track and approach the target object. The specific steps are as follows: S31, activate Auto mode; S32, Control Transfer; S33, acquire multi-source data; S34, Coordinate Transformation and Direction Angle Calculation; S35, Attitude error PI correction; S36, PID attitude control; S37, control command output and attitude adjustment.

[0028] Based on the above technical solution, S11, after the UAV system is powered on, the initialization program of the flight control system is executed. After the flight control system completes the initialization, the remote controller displays that it is ready for takeoff. S12, after completing the flight control initialization, starts the FPV system initialization program. After the FPV system initialization is completed, the real-time image of the drone camera can be seen. S13, after the FPV system initialization is completed, the computing chip initialization program is started, and the computing chip initialization outputs an initialization success signal to the OSD chip; S14: After receiving the computing chip initialization success signal, start the OSD chip initialization program. After the OSD chip initialization is completed, load the computing chip initialization success signal onto the video stream and display the computing chip initialization success in the FPV glasses.

[0029] Based on the above technical solution, in S211, the pilot operates the drone to take off and observes the target object through FPV goggles; S212, the remote controller sends a target detection signal to the computing chip through the flight control jump board, and the computing chip starts the target detection algorithm; S213 calculates the position of a predefined target in the image by running a target detection algorithm through a computing chip; S214 adds the target position coordinates to the analog signal through the OSD chip, and the final detection result is displayed in the FPV glasses; S215, the pilot selects the targets to be tracked sequentially via remote sensing; S216, After the target selection is completed, proceed to step S3 to lock the selected target and automatically track it; S221, the pilot takes off the drone and observes the target object through FPV goggles; S222: The remote controller sends a manual box selection signal to the computing chip through the flight control board, and the computing chip controls the OSD to directly draw a square target box; S223: Use the remote control's dial to adjust the size of the target frame, and the remote control's joystick can move the target frame's position. S224: The pilot selects the target object by operating the joystick and dial, and the remote controller sends a confirmation signal to the computing chip. S225, after receiving the data, the computing chip starts the target detection algorithm to check whether the selected target is a predefined target. If it is, it corrects the fit to the target object and starts target tracking at the same time. If it is not, it directly starts the target tracking algorithm to track the selected target. S226, After starting target tracking, proceed to step S3 to perform automatic tracking.

[0030] Based on the above technical solution, S31, after the drone achieves stable tracking during flight, the pilot initiates the Auto command, and the remote controller sends the Auto mode to the flight controller; S32: After receiving the Auto command, the flight controller forwards it to the computing chip. S33, the computing chip acquires flight control attitude and tracking result data, and the computing chip performs attitude control algorithm calculations based on the flight control attitude and tracking result data; S34 executes a coordinate transformation algorithm through a computing chip to convert the target pixel coordinates into the target orientation angle in the body coordinate system, including pixel coordinate to camera coordinate system coordinate transformation, camera coordinate system to body coordinate system transformation, and target orientation angle calculation in the body coordinate system. The formula for transforming pixel coordinates to camera coordinates is as follows: ; ; ; The camera coordinate system is transformed to the body coordinate system using a rotation matrix R, as shown in the following formula: ; Where R is a 3×3 rotation matrix, determined by the camera mounting angle, a fixed value, and pre-stored in the calculation chip register; Target orientation angle in the airframe coordinate system: Target azimuth angle corresponds to UAV yaw angle error: ; Target pitch angle corresponds to UAV pitch angle error: ; The S35 computing chip uses the drone's camera pointing at the target as a reference, calculates the difference between the current attitude angle and the target's required attitude angle as the attitude error, eliminates the attitude error, and enables the drone's attitude to quickly converge to the target's required attitude. The formula is as follows: Pitch angle correction: ; Yaw angle correction: ; Roll angle correction: ; in, The attitude adjustment speed is determined by the scaling factor. These are the integral coefficients used to eliminate static errors. This is the integral term for the error; S36 employs a PID control law to convert the corrected attitude angle into an angular velocity command, achieving closed-loop attitude control. The core of UAV attitude control is to adjust the angular velocity so that the actual attitude angle tracks the corrected attitude angle. The specific PID formula is as follows: Pitch angular velocity: ; Roll rate: ; Yaw angular velocity: ; in, This represents the current actual attitude angle of the drone. For PID control coefficients, The output attitude angular velocity command is directly sent to the UAV motor controller to control the motor speed and adjust the attitude; S37, the computing chip sends the angular velocity command output by the PID control law to the UAV motor controller. The motor controller adjusts the speed of each motor according to the command to adjust the attitude of the UAV, so that the UAV flies towards the target direction and controls the UAV to track and approach the target object.

[0031] Example 2: S1, Model Building: Building a model based on the tracking task, such as a reinforcement learning-based model. Figure 5 As shown, the target tracking task is formulated as a Markov Decision Process (MDP) defined by tuples (S, A, P, R, γ), where S is the UAV state space, A is the UAV action space, the transition function P(st+1|st, at) simulates the environmental dynamics, and the reward function R(st, at) incentivizes actions that achieve the target while penalizing collisions and inefficient actions. The goal is to learn the optimal policy that maximizes the expected cumulative reward, which can be expressed using the Bellman optimality formula: Where γ ∈ [0, 1] is the discount factor for future rewards, this formula allows reinforcement learning to be used to train drones to achieve autonomous tracking in the environment.

[0032] The S-type drone system status here can be divided into three parts, namely the internal status of the drone; The drone's observation status of the external mission environment; the control status given by the drone operator.

[0033] First, the observation data of the UAV includes the UAV's own state, its own position data, velocity data, and attitude data, which can be represented as follows: in This indicates the location data of the drone. Drone speed data, Quaternions represent the attitude data of a drone.

[0034] The drone perceives the external environment by acquiring image data from its FPV camera, which can be represented as a three-channel tensor matrix. Here, I represents the image. Represents pixel value, These represent the number of channels in the image and the image's height and width, respectively.

[0035] The pilot's desired tracking data can be represented in two parts. The first part is the coordinate data of the target in the image, provided by the system. The second part represents the relative state of the drone with respect to the target object, indicating whether the drone is tracking at a long distance or a short distance. Here, the position and dimensions of the target object in the image are obtained, representing the pilot's desired state: here These are the 1-norm and 2-norm of the distance between the target object's position and the center of the image, respectively. This indicates the position of the target object in the image. The position of the line point in the image; This indicates the proportion of the target object's area in the image to the total image area. Represents the image area of ​​the target object. Total pixel area of ​​the image It refers to the height and width of the target object's bounding box. It refers to the overall height and width of the image.

[0036] Therefore, the final tracking data expected by the pilot is expressed as: in It is a parameter that controls the degree of tracking proximity.

[0037] Here, A represents the output action, which can be expressed as the speed of the drone's three axes and the drone's yaw angle. For output stability, the output is controlled within... Between these points, the actual speed input of the drone is: in It refers to the flight control and tracking execution speed of the drone. That is the maximum speed of the drone. It refers to the output speed of the policy network.

[0038] Similarly, the yaw angles are as follows: in It is the drone flight control tracking that executes the yaw angle. It is the maximum yaw angle of the drone. It refers to the output speed of the policy network.

[0039] Finally, the reward function is designed, mainly divided into tracking and approach rewards. Rewards are given when the object's proportion in the UAV image is closer to μ, or when the object is closer to the image center. A penalty is given for deviation, expressed as follows: Here, H and W are the height and width of the image.

[0040] Tracking mode reward: Rewards the drone for achieving the optimal tracking distance. Smoothing constraints are used to smooth the drone's flight trajectory by using the velocity difference between two consecutive moments, making the drone's flight smoother. The final reward function is expressed as: S2, Simulation Environment Setup: A simulation environment model for UAV tracking is built using IsaacSim, mainly including UAVs, FPV cameras, people, vehicles, and other tracking targets. The relative positions of the UAV and these objects are randomly initialized for simulation. At the same time, a Python script is used to read relevant UAV and environmental data and input them into the reinforcement learning model for training. To improve efficiency, N environments can be built for parallel training, ultimately achieving stable tracking of target objects by the UAV.

[0041] S3, Real-world Deployment: First, the neural network in the simulation is deployed onto the drone. A professional pilot operates the drone to track the target object. At the same time, the output of the online reinforcement learning and the output of the professional pilot are recorded. The selected data are used to perform supervised fine-tuning of the neural network. Finally, it is deployed onto the drone to achieve autonomous tracking flight.

[0042] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A three-stage unmanned aerial vehicle (UAV) terminal flight guidance and tracking system, characterized in that: It includes an analog camera module, a digital-to-analog converter chip module, a computing chip module, an OSD chip module, a flight controller module, a data transmission module, an analog image transmission module, a remote controller module, and an FPV glasses module; The analog camera module is responsible for capturing real-time images in front of the drone, the digital-to-analog converter chip module is responsible for converting analog signals to digital signals, the computing chip module is responsible for running target detection algorithms, target tracking algorithms and attitude control algorithms, and the OSD chip module is responsible for superimposing the position information of the target tracking results onto the analog video signal. The flight controller module is the core control unit of the UAV, the data transmission module is responsible for establishing wireless communication, the analog image transmission module is responsible for transmitting real-time video signals collected by the analog camera, the remote controller module is used by the pilot to control the UAV, and the FPV glasses module is the ground station display device.

2. The three-stage unmanned aerial vehicle (UAV) terminal flight guidance and tracking system according to claim 1, characterized in that: The analog camera module is compatible with traditional FPV drone analog cameras and splits the camera's analog signal into two paths. One path converts the image signal into a digital signal through an analog-to-digital converter chip and transmits it to a computing chip for image signal processing. The other path directly transmits the analog signal to an analog image transmission chip, which then transmits the real-time drone image to the FPV glasses on the ground, ensuring the real-time performance of the image in the glasses. The analog-to-digital converter chip module converts the analog video stream output by the analog camera into a digital signal that can be processed by the computing chip. The analog-to-digital converter chip module receives one signal from the analog camera, completes the digital conversion, and outputs it to the computing chip.

3. The three-stage unmanned aerial vehicle (UAV) terminal flight guidance and tracking system according to claim 1, characterized in that: The computing chip module integrates target detection algorithm, target tracking algorithm, and attitude data derivation algorithm. The target detection algorithm uses the YOLO series algorithm, the target tracking algorithm uses the KCF algorithm, and the attitude data inference algorithm is used in Auto mode to derive attitude data by combining the UAV's own sensor data and through the collaborative inference of multiple algorithms.

4. The three-stage UAV terminal flight guidance and tracking system according to claim 1, characterized in that: The OSD chip module converts the target coordinate information output by the target tracking algorithm into display instructions that the OSD chip can recognize. After achieving time synchronization with the original video signal, visual markers are superimposed at the specified screen position, and finally the video with the target position markers is output. The original video data is not destroyed throughout the process, and the superposition is only completed in the display stage. The core process includes four key steps: position information reception and parsing, video time synchronization, marker generation and pixel mixing, and superimposed video output.

5. A three-stage UAV terminal flight guidance and tracking system according to claim 1, characterized in that: The flight controller module is the core control unit of the UAV, responsible for the UAV's attitude stabilization and flight control, and undertakes two key functions: First, the target detection and tracking commands transmitted from the remote controller are captured through the data transmission module and transmitted to the computing chip. Second, upon receiving the Auto command, control is immediately switched to the computing chip, and the drone's own attitude data is sent to the computing chip. The computing chip calculates the drone's attitude data for the next moment based on the position of the target object in the image and the drone's attitude data at this time, and sends this attitude data to the drone to realize the drone's tracking and approach of the target object. The data transmission module is responsible for communication between the remote controller module and the flight controller module, transmitting key control information; The analog image transmission module is responsible for transmitting the real-time video signal collected by the analog camera to the FPV glasses module on the ground. It works in conjunction with the OSD chip. The video signal with the target position mark superimposed on it by the OSD chip is transmitted to the ground equipment through the analog image transmission link.

6. A three-stage unmanned aerial vehicle (UAV) terminal flight guidance and tracking system according to claim 5, characterized in that: The remote control module is the operating device for the pilot to control the drone. It controls the drone and selects targets by sending control commands. The pilot selects the target to be tracked in sequence through remote sensing operation, or moves and adjusts the target frame position to select the target object through joystick and dial operation. After confirming the target, the remote control sends a confirmation signal and an Auto command, which respectively trigger the target tracking algorithm to start and the flight mode to switch. The FPV glasses module is a ground station display device responsible for receiving and displaying real-time images of the UAV and target location information superimposed on the OSD. The pilot observes the UAV flight footage, target detection results, and target tracking markers through the FPV glasses, enabling real-time monitoring of the target selection and tracking process.

7. A three-stage UAV terminal flight guidance and tracking method, used in the three-stage UAV terminal flight guidance and tracking system according to any one of claims 1-5, characterized in that: Includes the following steps: S1, Unmanned Aerial Vehicle System Power-On Self-Test Initialization; S2, Target Detection and Target Selection; S3, automatic tracking and approach; The specific steps of S1 are as follows: S11, Flight control system initialization; S12, FPV system initialization; S13, Computing chip initialization; S14, OSD chip initialization; S2 includes target detection manual selection adsorption and manual box selection adsorption; The specific steps for manually selecting adsorption for target detection are as follows: S211, UAV takeoff and target observation; S212, Activate target detection function; S213, Predefined target location calculation; S214, test results are displayed in overlay; S215, Manually select the target; S216, Entering the tracking phase; The specific steps for manually selecting and adsorbing are as follows: S221, Unmanned Aerial Vehicle Takeoff and Target Observation; S222, activate manual selection function; S223, Adjust the target bounding box; S224, Confirm the selected target; S225, Target Verification and Tracking Initiated; S226, Entering the tracking phase; The specific steps of S3 are as follows: S31, activate Auto mode; S32, Control Transfer; S33, acquire multi-source data; S34, Coordinate Transformation and Direction Angle Calculation; S35, Attitude error PI correction; S36, PID attitude control; S37, control command output and attitude adjustment.

8. A three-stage UAV terminal flight guidance and tracking method according to claim 7, characterized in that: In step S11, after the UAV system is powered on, the flight control system is initialized and the remote controller displays that it is ready for takeoff. In step S12, after completing the flight control initialization, the FPV system initialization is started, and the real-time image of the UAV camera can be seen. In step S13, after the FPV system initialization is completed, the computing chip initialization is started, and the computing chip initialization outputs an initialization success signal to the OSD chip. In step S14, after receiving the computing chip initialization success signal, the OSD chip initialization is started. After the OSD chip initialization is completed, the computing chip initialization success signal is loaded onto the video stream, and the computing chip initialization success is displayed in the FPV glasses.

9. A three-stage UAV terminal flight guidance and tracking method according to claim 7, characterized in that: In S211, the pilot operates the drone to take off and observes the target object through FPV goggles; In S212, the remote controller sends a target detection signal to the computing chip through the flight control board, and the computing chip starts the target detection algorithm. In step S213, the target detection algorithm is run by the computing chip to calculate the position of the predefined target in the image; In step S214, the target position coordinates are added to the analog signal through the OSD chip, and the final detection result is displayed in the FPV glasses. In S215, the pilot selects the targets to be tracked sequentially via remote sensing. In step S216, after the target selection is completed, the process proceeds to step S3, where the selected target is locked and automatically tracked. In S221, the pilot takes off the drone and observes the target object through FPV goggles; In S222, the remote controller sends a manual box selection signal to the computing chip through the flight control board, and the computing chip controls the OSD to directly draw a square target box. In step S223, the size of the target frame is adjusted using the remote control's pulsator, and the remote control's joystick can move the position of the target frame. In S224, the pilot selects the target object by operating the joystick and the dial, and the remote controller sends a confirmation signal to the computing chip. In step S225, after receiving the data, the computing chip starts the target detection algorithm to detect whether the selected target is a predefined target. In step S226, after starting target tracking, the process proceeds to step S3 to perform automatic tracking.

10. A three-stage UAV terminal flight guidance and tracking method according to claim 9, characterized in that: In S31, after the drone achieves stable tracking during flight, the pilot initiates the Auto command, and the remote controller sends the Auto mode to the flight controller. In step S32, after receiving the Auto command, the flight controller forwards it to the computing chip. S33, the calculation chip acquires flight control attitude and tracking result data; S34 executes a coordinate transformation algorithm through a computing chip to convert the target pixel coordinates into a target orientation angle in the body coordinate system; In S35, the computing chip uses the drone camera pointing at the target as a reference to calculate the difference between the current attitude angle and the target's required attitude angle as the attitude error, eliminates the attitude error, and enables the drone's attitude to quickly converge to the target's required attitude. The S36 uses a PID control law to convert the corrected attitude angle into an angular velocity command, thereby achieving closed-loop attitude control. In S37, the computing chip sends the angular velocity command output by the PID control law to the UAV motor controller. The motor controller adjusts the speed of each motor according to the command, so that the UAV flies towards the target direction and controls the UAV to track and approach the target object.

Images