A method and system for autonomous tracking of a gimbal on a drone

Abstract

This invention provides an autonomous fire tracking method and system for UAVs with an airborne gimbal, belonging to the field of UAV control. It addresses the problems of existing UAV gimbal fire tracking methods, such as line-of-sight instability under complex disturbances, poor adaptability to dynamic fire targets, and insufficient coordination between perception and control. This invention reduces the false alarm and missed alarm rates of fire detection in complex environments through fire detection and target acquisition; it achieves intelligent understanding and tracking of the dynamic characteristics of the fire line through an intelligent tracking decision unit; it uses a weighted centroid algorithm to automatically lock the tracking point to the fire spread front; and it combines a finite state machine and Kalman prediction mechanism to ensure robust, continuous tracking, and rapid autonomous recovery capabilities for non-rigid, dynamically changing fire lines; by integrating the parameter self-tuning capability of a fuzzy adaptive PID controller with the active disturbance rejection compensation capability of a disturbance observer, it solves the problem of gimbal line-of-sight instability caused by complex disturbances in UAVs, significantly improving tracking accuracy and stability.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) control technology, and more specifically, to a method and system for autonomous fire tracking by an airborne gimbal on a UAV. Background Technology

[0002] Using drones equipped with electro-optical pods for real-time monitoring and tracking of fires has become a key technological means in modern forest fire fighting due to its high flexibility, rapid deployment, and ability to penetrate dangerous areas. The core of an efficient drone fire tracking system lies in a two-degree-of-freedom gimbal carrying visible light and infrared sensors that can continuously and stably lock the fire target in the center of its field of view, thus providing clear and continuous observational data for subsequent fire analysis, situation assessment, and command decision-making.

[0003] However, existing drone-based fire tracking technologies still face the following technical bottlenecks when dealing with real and complex fire scene environments, limiting further improvements in their effectiveness and reliability:

[0004] 1. Stability Control Issues of Gimbal Servo Controllers under Complex Disturbances. When UAVs operate over fire sites, their flight platforms are continuously subjected to the coupling effects of various time-varying, wide-frequency-domain disturbances, including rotor vibration, atmospheric turbulence (wind disturbance), and attitude changes caused by their own maneuvering. These disturbances are directly transmitted to the airborne gimbal, causing jitter or even instability in the line of sight of the sensors, resulting in blurred and jittery observed images, and severely degrading the input quality of the visual tracking algorithm. Currently, most widely used gimbal servo control methods are based on classical PID control or its simple modifications. These methods typically have fixed controller parameters and mainly rely on feedback for error correction. They lack effective active suppression and compensation capabilities for the aforementioned high-frequency, large-amplitude complex disturbances, leading to a sharp decline in the tracking accuracy and stability of the system in harsh environments.

[0005] 2. The Adaptability of Visual Tracking Algorithms to the Dynamic Characteristics of Fire Lines. Fire line targets are characterized by their non-rigidity (irregular shape, prone to branching and merging) and dynamic changes (spread speed and direction vary with environmental factors). Existing visual algorithms used for PTZ tracking, such as tracking methods based on simple image centroids or traditional correlation filters, are mostly designed for rigid targets with relatively stable appearances and simple motion models. When directly applied to fire line tracking, they often fail to accurately locate the most threatening fire front, easily resulting in tracking points lagging behind the actual fire line location or drifting to non-critical burning areas (such as already burned areas), leading to tracking failure. Furthermore, when the fire line is temporarily lost in the image due to dense smoke, sudden changes in lighting, or rapid movement of the aircraft, existing systems generally lack efficient intelligent prediction mechanisms and rapid autonomous re-acquisition capabilities, causing tracking interruptions and failing to meet the practical needs of all-weather, continuous fire monitoring.

[0006] 3. The systemic disconnect between perception, decision-making, and control. Current technical solutions typically design and optimize the upper-level visual perception and tracking decision-making modules relatively independently from the lower-level PTZ servo control modules. This separation of perception and control makes it difficult for the system to coordinate responses from a globally optimal perspective when facing complex and dynamic fire environments. The commands output by the vision module may not fully consider the dynamic characteristics and disturbance limits of the controller, and the disturbance suppression performance of the control module may not be intelligently linked with the upper-level tracking strategy. This disconnect between the various components results in significant shortcomings in the overall performance, environmental adaptability, and robustness of the entire tracking system.

[0007] In summary, there is an urgent need for a solution that can achieve deep system-level collaboration and organically integrate high-performance anti-disturbance servo control with intelligent visual tracking decision-making. This solution would overcome the core challenges of existing technologies in suppressing complex disturbances and adapting to dynamic targets, thereby significantly improving the combat-ready monitoring capabilities of UAVs in real fire environments. Summary of the Invention

[0008] The technical problem to be solved by this invention is:

[0009] To address the problems of existing UAV gimbal-based fire tracking methods, such as line-of-sight instability under complex disturbances, poor adaptability to dynamic fire targets, and insufficient coordination between perception and control.

[0010] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0011] This invention discloses an autonomous fire tracking method for an airborne gimbal on a drone, comprising the following steps:

[0012] The S100 system initializes and searches for fire lines. After powering on, the system performs hardware self-tests, sensor calibrations, and establishes communication links. The two-degree-of-freedom electro-optical pod gimbal performs a homing operation, and then the intelligent tracking decision unit controls the UAV to enter the preset search state. In this state, the two-degree-of-freedom electro-optical pod gimbal begins to move according to the planned scanning trajectory. It obtains frame images by synchronously acquiring image sequences at a fixed frame rate through the dual-optical pods, and transmits the frame images to the intelligent tracking decision unit in real time for subsequent fire detection and analysis.

[0013] S200: The frame image acquired in step S100 is processed for the first time to detect whether there is a fire in the current frame. This includes extracting multi-scale features from the visible light image and the infrared thermal imaging image in the frame image to obtain a visible light multi-scale feature set and an infrared multi-scale feature set; inputting the two modal multi-scale feature map sets into a temporally enhanced flame-specific attention fusion module to obtain enhanced multi-scale fusion features; concatenating the enhanced multi-scale fusion features by channel dimension to obtain a basic fusion feature map that fuses multi-modal information; inputting the basic fusion feature map into a superpixel edge protection module to obtain an edge-enhanced fusion feature map; and inputting the edge-enhanced fusion feature map into a deep learning model for fire detection to determine whether there is a fire.

[0014] S300: The image frames in step S200 that detected the fire are processed a second time to identify the fire front and trigger a tracking state switch. This includes segmenting the infrared image frames that detected the fire and obtaining a binary mask of the burning area; using an edge detection algorithm to extract the contour of the burning area, and then combining it with the optical flow motion analysis between consecutive frames to calculate the optical flow field between the current frame and the previous frame, obtaining the motion vector and velocity amplitude of the pixels, identifying the pixel sequence located on the contour of the burning area and whose optical flow velocity amplitude exceeds a preset threshold as the fire front, and selecting the pixel sequence with violent movement on the contour as the fire front; once the identification is successful, the finite state machine inside the intelligent tracking decision unit is immediately triggered to switch from the search state to the capture state, and the initial positioning command is calculated to drive the two-degree-of-freedom optoelectronic pod of the gimbal to quickly pull the fire target into the center of the field of view;

[0015] The S400 performs a third processing based on the dynamic motion characteristics of the fire front to obtain precise tracking points representing the fire front spread. This includes identifying the fire front region in each frame. The system constructs motion history image values ​​and fuses instantaneous light flow amplitude values ​​to calculate a dynamically weighted centroid point. The calculated centroid point is converted into control commands for gimbal pitch and yaw angles, i.e., angle control commands, through the camera imaging model.

[0016] S500: The angle control command and system feedback information calculated in step S400 are processed for the fourth time. The drive signal is calculated through the composite anti-disturbance servo control core. This includes using a fuzzy adaptive PID controller to dynamically adjust the control parameters according to the real-time tracking error, and using a disturbance observer to estimate and feed forward the total disturbance online in real time based on the nominal dynamic model and multi-sensor feedback. The output of the fuzzy adaptive PID controller is superimposed with the feed forward compensation amount generated by the disturbance observer to form the final composite control signal driving the gimbal motor.

[0017] S600 performs a fifth processing step during the tracking process, diagnosing the tracking status in real time and autonomously predicting and recovering from anomalies such as target loss. This includes constructing an intelligent recovery mechanism based on multi-dimensional confidence assessment and motion state prediction. While tracking normally, the system continuously evaluates the tracking quality in parallel. Once the overall tracking confidence falls below a safety threshold, it is determined that the target has been lost, triggering the system's finite state machine to switch to the recovery state. In the recovery state, visual measurement updates are paused, and instead, a Kalman filter is used to predict the target's possible future location based on its historical motion state. A dynamic region of interest is defined around the predicted location, guiding the two-degree-of-freedom optoelectronic pod gimbal towards this dynamic region of interest, while simultaneously restarting the target detection algorithm.

[0018] Further, in step S100,

[0019] S110. Check the status of the dual-light pod, gimbal servo controller, inertial detection unit, main controller and each communication link; perform online calibration of the visible light camera and infrared thermal imager, obtain and update the camera intrinsic parameter matrix and extrinsic parameter transformation matrix; the gimbal servo controller controls the two-degree-of-freedom optoelectronic pod gimbal to perform a return-to-zero operation, initializes the finite state machine in the intelligent tracking decision unit, and sets its initial state to the search state.

[0020] S120: The intelligent tracking decision unit sends scanning commands to the gimbal servo controller. The two-degree-of-freedom optoelectronic pod gimbal moves according to the pre-programmed scanning trajectory. The dual-light pod synchronously acquires visible light and infrared thermal imaging image sequences at a fixed frame rate to obtain frame images, and transmits the frame images to the intelligent tracking decision unit in real time through a high-speed interface.

[0021] Further, in step S200,

[0022] S210. Perform multi-scale feature extraction on the visible light image and the infrared thermal imaging image respectively. Use a lightweight convolutional neural network as the backbone feature extractor to extract features from the input visible light image. and infrared thermal imaging images Forward propagation is performed; the network design comprises multiple stages, each outputting feature maps with different spatial resolutions, thus forming a multi-scale feature set for visible light. } and infrared multi-scale feature set { }, where L is the total number of feature scales, These are the Lth visible light feature map and the infrared feature map, respectively;

[0023] S220. Input the feature maps of the two modalities into the temporally enhanced flame-specific attention fusion module to obtain the enhanced fused features. For each corresponding feature scale l, the visible light feature map... With infrared feature map The input is a flame-specific attention fusion module; the flame-specific attention fusion module generates a set of channel-related fusion weights through flame physics prior guidance and temporal dynamic correction. and ,satisfy ;

[0024] The flame physics prior is guided by calculating the flame thermal diffusion prior map using the real temperature gradient distribution of the infrared thermal imaging image; for a pixel (x, y) in the infrared thermal imaging image, the thermal diffusion prior value at that pixel is determined by the rate of temperature change in its local neighborhood. :

[0025]

[0026] in, This is an infrared thermal imaging image. Let be the spatial gradient vector of the infrared thermal image at pixel (x,y). Let be the rate of grayscale change along the horizontal direction at pixel (x,y) in the infrared thermal image. This represents the rate of grayscale change along the vertical direction at pixel (x,y) in an infrared thermal image.

[0027] The temporal dynamic correction incorporates optical flow motion information between consecutive frames to calculate the dynamic trend vector of the fire line spread direction. It obtains the motion vector and velocity amplitude of each pixel, and uses a sparse optical flow algorithm to calculate the optical flow field V(x,y) between two adjacent infrared thermal imaging frames. For pixels within the fire front region, a temporal correction factor is generated based on the consistency of their motion direction with the overall fire line spread direction. ( );

[0028] The fusion weights are dynamically generated by the channel attention mechanism within the time-enhanced flame-specific attention fusion module, combined with the heat spread prior map and the time-series correction factor.

[0029]

[0030] in, This is the feature map after multi-scale fusion at the l-th feature scale;

[0031] S230. Concatenate the multi-scale fused features to obtain a feature map that fuses multimodal information. Then, combine the fused feature maps obtained at different scales. The data is upsampled to the maximum spatial size using bilinear interpolation; subsequently, the data is stitched together along the channel dimension to form a basic fused feature map. ;

[0032] S240. The basic fused feature map is input into the superpixel edge protection module. The superpixel edge protection module first divides the basic fused feature map into multiple superpixel regions based on local illumination and texture consistency using a simple linear iterative clustering algorithm. },in, Let K be the Kth superpixel region, and K be the total number of superpixel regions. Based on the temperature gradient distribution of the combustion zone in the infrared thermal imaging image, calculate the edge confidence weight for each superpixel region. For the kth superpixel region, its edge confidence weight... Defined as the normalized result of the average value of the spatial gradient magnitude of the infrared thermal imaging image within the superpixel region, the calculation formula is:

[0033]

[0034] in, This represents the gradient magnitude of the infrared thermal image at each pixel. This represents the Sigmoid normalization function;

[0035] By weighted aggregation, an edge-enhanced fusion feature map is generated. :

[0036]

[0037] in, It belongs to the superpixel region in the basic fusion feature map;

[0038] S250. Input the edge-enhanced fused feature map into the detection head. The detection head consists of a global average pooling layer, several fully connected layers, and an output layer using the Sigmoid activation function. It outputs a confidence score indicating that a fire exists in the current frame. ,and The system is based on preset thresholds. Determine if a fire exists: If If the fire is detected, it is determined that there is a fire, and the process proceeds to step S300; otherwise, it is determined that there is no fire, and the system remains in the search state.

[0039] Further, in step S300,

[0040] S310. Segment the infrared image frame that detects the fire and obtain the binary mask image B(x,y) of the burning area;

[0041]

[0042] in, and The mean and standard deviation of gray values ​​within a local m×n neighborhood of a pixel. This is an adjustable coefficient; 0 represents the non-combustion zone, and 1 represents the combustion zone.

[0043] S320. Use an edge detection algorithm to extract the contour of the burning area, perform morphological operations on the binary mask image B, and then use a contour tracking algorithm to extract the complete and closed contour line set {C1,C2,...} of the burning area, and select the contour with the largest area as the main burning area contour C;

[0044] S330. Based on the optical flow field V(x,y) between two adjacent infrared thermal imaging images calculated in step S220, calculate the motion vector (u,v) of each pixel (x,y) in the infrared thermal imaging image and calculate the velocity amplitude. , where u and v represent the optical flow velocity components of the pixel in the horizontal and vertical directions, respectively;

[0045] The sequence of pixels located on the combustion zone contour whose optical flow velocity amplitude exceeds a preset threshold is identified as the fire front, and all pixels on the main combustion zone contour C are traversed; for the i-th point on the contour... Check its corresponding optical flow velocity amplitude. ;like Speed ​​greater than the preset threshold If a point is considered to be moving forward, it is considered to be at the forefront of the attack. Connecting all the contour points that meet the conditions in sequence forms the contour curve of the forefront. ;

[0046] S340. Once a valid frontline vanguard is identified, the finite state machine inside the intelligent tracking decision unit is triggered to switch from the "search" state to the capture state.

[0047] Calculate the initial position of the target in the image and calculate the profile curve of the fire front. The average pixel coordinates of all points on the target point are denoted as ( ). );

[0048] Generate the corresponding gimbal angle command, and convert the target point coordinates into a pitch angle in the gimbal coordinate system based on the camera intrinsic parameters and the gimbal-camera calibration relationship. and yaw angle The two-degree-of-freedom optoelectronic pod gimbal initially pulls the target into the center of the field of view, and the angle command is then executed. ) is sent to the PTZ servo controller, so that the target point ( Move the image to the area near the center of the field of view to complete the initial capture.

[0049] Further, in step S400,

[0050] S410. In tracking mode, the system updates the fire front outline region R in real time for each newly acquired infrared thermal imaging image frame.

[0051] S420, Introduce motion history images:

[0052]

[0053] Where D(x,y,t) is a binary motion detection map obtained from continuous infrared image frames by frame difference method or background subtraction method, τ is the decay time constant, and δ is the decay rate;

[0054] S430, calculate the current optical flow velocity amplitude obtained in step S330. It is fused with the motion history feature map MHI(x,y);

[0055] The motion history feature map and optical flow velocity amplitude were normalized to ensure their values ​​were within the range of [0,1].

[0056]

[0057] in, These are the normalized motion history feature map values ​​and optical flow velocity amplitudes, respectively.

[0058] Calculate a fusion weight W(x,y) for each pixel within the outline region R of the Firefront Frontline:

[0059]

[0060] Where λ is a fusion coefficient between 0 and 1;

[0061] S440. Using the dynamic feature weights W(x,y) calculated in step S430, calculate the weighted centroid of the fire front contour region R. The coordinates of this centroid are ( This refers to the precise tracking point selected for the current frame:

[0062]

[0063] S450, accurately track the points in the image pixel coordinate system ( Convert the angle control command into the coordinate system of the two-degree-of-freedom optoelectronic pod gimbal.

[0064] By using camera intrinsic parameters for back projection, the coordinates of the precise tracking point on the normalized camera plane can be obtained. ) :

[0065]

[0066] in,( ) are the coordinates of the principal point of the image, ( , These are the camera's focal lengths in the x and y directions, respectively;

[0067] The pitch angle increment required to align the two-degree-of-freedom optoelectronic pod gimbal with the precise tracking point and yaw angle increment The approximate calculation is as follows:

[0068]

[0069] The calculated raw angle increment is filtered to smooth the angle control command. and It is sent in real time to the subsequent composite anti-disturbance servo control core.

[0070] Further, in step S500,

[0071] S510, Define the angle tracking error of a single axis of the gimbal as e(t) = and its rate of change ec(t) = , serving as the two input variables of the fuzzy adaptive PID controller; where, The gimbal angle command issued by the intelligent tracking decision unit. The actual angle fed back in real time by the gimbal encoder; the error value is mapped to a preset fuzzy linguistic variable domain through a fuzzification interface;

[0072] Through fuzzy inference and defuzzification, the three parameters of the fuzzy adaptive PID controller are output in real time. Adjustment amount Parameters of the fuzzy adaptive PID controller adjusted in real time , , for:

[0073] , ,

[0074] in, , , These represent the reference proportional parameter, reference integral parameter, and reference derivative parameter of the fuzzy adaptive PID controller, respectively.

[0075] Based on this, the output of the fuzzy adaptive PID controller is calculated. :

[0076]

[0077] S520. Establish the nominal dynamic model of the gimbal servo axis:

[0078]

[0079] in, and The nominal moment of inertia and viscous damping coefficient obtained through system identification. For motor control torque, For lumped disturbances acting on the system;

[0080] In the frequency domain, the estimated value of the disturbance for:

[0081] =Q(s)

[0082] Where Q(s) is a low-pass filter, taking... = , It is a time constant;

[0083] Angular acceleration measured by an inertial measurement unit installed on the drone body After coordinate system transformation to the gimbal axis, it is introduced as an auxiliary observation signal; the enhanced total disturbance estimation for:

[0084] =

[0085] in, This is the initial lumped disturbance estimate output by the disturbance observer; The gain coefficient is calibrated based on the vibration transmission characteristics. This is the converted angular acceleration;

[0086] S530, feedback control quantity generated by fuzzy adaptive PID controller Feedforward compensation amount generated by the disturbance observer Generate the control torque applied to the gimbal servo motor. :

[0087] = .

[0088] Further, in step S600,

[0089] S610, Multi-dimensional tracking confidence fusion assessment, defining and calculating comprehensive tracking confidence in real time. :

[0090]

[0091] in, For the confidence level of the appearance sub-sub, For motion consistency sub-confidence. For regional feature sub-confidence, For controller error sub-confidence, Weights for each confidence level;

[0092] When the overall tracking confidence level N consecutive frames below a preset loss threshold When this occurs, the target is determined to be lost, triggering a state switch;

[0093] S620, based on Kalman filter, models and predicts the motion of the fire tracking point in the image plane; its state vector It typically includes position and velocity: Under normal tracking conditions, the filter performs a prediction-update loop.

[0094] Once the system enters the recovery state, the filter stops updating and only performs time updates based on the dynamic model.

[0095]

[0096]

[0097] Where F is the state transition matrix, P is the state estimation error covariance matrix, and Q is the process noise covariance matrix;

[0098] Predicted target location Together with their uncertainty, they define a dynamic region of interest in the image;

[0099] The S630, a two-degree-of-freedom optoelectronic pod gimbal retargeting and intelligent research system, receives instructions to adjust the line of sight to the center of the predicted region of interest, making the region the center of the image field of view; subsequently, the intelligent tracking decision unit re-executes the frontline detection and recognition algorithm within the region of interest with a sensitivity higher than that of conventional detection.

[0100] S640: Once a valid frontline target is rediscovered within the region of interest, the finite state machine is transitioned from the recovery state back to the acquisition state. Using the rediscovered target point as the new initial target, a gimbal angle command is generated to drive the two-degree-of-freedom electro-optical pod gimbal to re-align. After the two-degree-of-freedom electro-optical pod gimbal stably pulls the target into the center of the field of view, the system automatically switches back to the tracking state, and the Kalman filter resumes the prediction-update loop. If the target is not rediscovered within the region of interest within the preset time window, the system will determine that the target has completely left the field of view, and the finite state machine will revert to a larger-range search state to restart the global scan.

[0101] Furthermore, it also includes S700, system coordination and closed-loop operation. During the fire, the system cyclically executes steps S400-S600, including...

[0102] The intelligent tracking decision unit estimates the amplitude of the disturbance based on the real-time feedback from the main controller. Or, if the system is nearing saturation, the system adaptively smooths out the angle command output. Alternatively, the rate of change of the command can be actively limited to avoid exciting the mechanical resonance of the two-degree-of-freedom optoelectronic pod gimbal or exceeding its dynamic response capability; conversely, the composite anti-disturbance servo control center uses the target motion prediction information provided by the intelligent tracking decision unit to perform feedforward compensation in advance, thereby improving the tracking performance of fast-moving targets.

[0103] An autonomous fire tracking system for an unmanned aerial vehicle (UAV) airborne gimbal has program modules corresponding to the steps described above, and executes the steps in the aforementioned autonomous fire tracking method for an UAV airborne gimbal during runtime.

[0104] A computer-readable storage medium storing a computer program configured to implement, when invoked by a processor, the steps of a method for autonomous fire tracking of an unmanned aerial vehicle (UAV) airborne gimbal.

[0105] Compared with the prior art, the beneficial effects of the present invention are:

[0106] This invention proposes a method and system for UAV gimbal fire tracking based on adaptive anti-disturbance control. It aims to solve the problems of poor stability and weak adaptability to dynamic targets of existing UAV fire tracking systems under complex disturbances by deeply coordinating high-performance servo control and intelligent vision decision-making, thereby significantly improving the UAV's practical monitoring capabilities in real fire scene environments.

[0107] This invention achieves rapid fusion analysis of dual-light images and initial fire line acquisition through fire detection and target acquisition, reducing the false alarm and missed alarm rates in complex environments. Then, an intelligent tracking decision unit enables intelligent understanding and tracking of the dynamic characteristics of the fire line. A weighted centroid algorithm automatically locks the tracking point to the fire's spread front, and a finite state machine and Kalman prediction mechanism ensure robust, continuous tracking, and rapid autonomous recovery capabilities for non-rigid, dynamically changing fire lines. Furthermore, a composite disturbance rejection control core enables high-performance servo control of the gimbal, integrating the parameter self-tuning capability of a fuzzy adaptive PID controller with the active disturbance rejection compensation capability of a disturbance observer. This fundamentally solves the problem of gimbal line-of-sight instability caused by complex disturbances such as UAV body vibration and wind disturbance, significantly improving tracking accuracy and stability. Finally, a system collaborative management module achieves deep integration and collaborative optimization of various functional modules, constructing a complete "perception-decision-control" closed-loop system. This allows the system to adaptively adjust its operating strategy according to environmental changes and mission requirements, achieving a leap from single-function stacking to optimal system performance in UAV fire line tracking.

[0108] Through the coordinated operation of the above modules, this invention not only improves the technical performance of individual components but also optimizes the overall efficiency at the system level, providing a highly reliable and intelligent complete solution for UAV forest fire monitoring. It has significant theoretical value and promising engineering applications. Attached Figure Description

[0109] Figure 1 This is a flowchart of a method for autonomous fire tracking by an airborne gimbal on a drone, as described in an embodiment of the present invention.

[0110] Figure 2 This is a general framework diagram of an unmanned aerial vehicle (UAV) airborne gimbal autonomous tracking system according to an embodiment of the present invention;

[0111] Figure 3 This is a schematic diagram illustrating the principle of the composite disturbance rejection control core in an embodiment of the present invention;

[0112] Figure 4 This is a state transition diagram of the finite state machine (FSM) within the intelligent tracking decision unit in this embodiment of the invention;

[0113] Figure 5 This is a schematic diagram illustrating the calculation process of the weighted centroid tracking algorithm in an embodiment of the present invention;

[0114] Figure 6 This is a schematic diagram of the system on a laboratory hardware-in-the-loop simulation testing platform in an embodiment of the present invention;

[0115] Figure 7 This is a comparison diagram showing the effect of the system in a real fire source tracking experiment in an outdoor field according to an embodiment of the present invention. Detailed Implementation

[0116] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0117] The UAV of the present invention includes a UAV flight platform, a two-degree-of-freedom optoelectronic pod gimbal, a dual-light pod (visible light camera and infrared thermal imager), an inertial measurement unit (IMU), a gimbal servo motor, and a main controller integrating an intelligent tracking decision unit and a composite anti-disturbance servo control core. All components communicate with each other at high speed via a CAN FD bus.

[0118] Specific Implementation Plan 1: Combining Figures 1 to 4 As shown, this invention provides a method for autonomous fire tracking by an airborne gimbal on a drone, comprising the following steps:

[0119] S100 System initialization and fire search: After the system is powered on, it performs hardware self-test, sensor calibration, and communication link establishment; the two-degree-of-freedom electro-optical pod gimbal performs a homing operation, and then the intelligent tracking decision unit controls the UAV to enter the preset search state; in this state, the two-degree-of-freedom electro-optical pod gimbal starts according to the planned scanning trajectory, and obtains frame images by synchronously acquiring image sequences at a fixed frame rate through the dual-optical pod, and transmits the frame images to the intelligent tracking decision unit in real time for fire detection and analysis;

[0120] System initialization aims to ensure all hardware units are in normal working order and establish a stable data communication link. Specifically, it includes checking and calibrating each component, such as the dual-optical pod, gimbal servo motor, inertial measurement unit (IMU), and main controller. After initialization, the system controls the two-degree-of-freedom optoelectronic pod gimbal to perform a homing operation and then enters the search state. In this state, the two-degree-of-freedom optoelectronic pod gimbal moves according to a pre-planned trajectory, and the dual-optical pod synchronously acquires images and transmits them in real time to the intelligent tracking and decision-making unit, providing a continuous data source for subsequent real-time fire detection.

[0121] Specifically, including,

[0122] S110. Check the status of the dual-light pod, gimbal servo controller, inertial detection unit, main controller and each communication link; perform online calibration of the visible light camera and infrared thermal imager, acquire and update the camera intrinsic parameter matrix and extrinsic parameter transformation matrix; the gimbal servo controller performs a zero-return operation, initializes the finite state machine in the intelligent tracking decision unit, and sets its initial state to "search" state.

[0123] The S120 intelligent tracking decision unit sends scanning commands to the gimbal servo controller. The two-degree-of-freedom optoelectronic pod gimbal moves according to the pre-programmed scanning trajectory, for example, the pitch axis is within the range of -30° to +10° and the yaw axis is within the range of ±60°, performing raster scanning at an angular velocity of 5° / s. At the same time, the dual-optical pod synchronously acquires visible light and infrared thermal imaging image sequences at a fixed frame rate to obtain frame images, and transmits the frame images to the intelligent tracking decision unit for processing in real time through a high-speed interface.

[0124] S200, Perform the first processing on the frame image acquired in step S100 to detect whether there is a fire in the current frame, including,

[0125] The frame images include visible light images and infrared thermal imaging images. Multi-scale feature extraction is performed on both the visible light and infrared thermal imaging images to obtain visible light multi-scale feature sets and infrared multi-scale feature sets. These two sets of multi-scale feature maps are input into a temporally enhanced flame-specific attention fusion module. Using the infrared temperature gradient as a physical prior and combining optical flow motion information, the dual-light fusion weights are dynamically adjusted to focus the network on the fire front, obtaining enhanced multi-scale fusion features. The enhanced multi-scale fusion features are then concatenated along the channel dimension to obtain a basic fusion feature map that integrates multi-modal information. This basic fusion feature map is input into a superpixel edge protection module. Based on local texture consistency and temperature gradient, the feature map is divided into regions and differentially enhanced to obtain an edge-enhanced fusion feature map that retains the fine edge features of the fire front while suppressing background noise. The edge-enhanced fusion feature map is then input into a deep learning model with a lightweight convolutional neural network backbone for fire detection to determine the presence of a fire. This step aims to fuse the multi-modal information of visible light and infrared thermal imaging images to achieve a rapid and accurate initial assessment of the fire situation.

[0126] Specifically, including,

[0127] S210. Perform multi-scale feature extraction on the visible light image and the infrared thermal imaging image respectively. Use a lightweight convolutional neural network as the backbone feature extractor to extract features from the input visible light image. and infrared thermal imaging images Forward propagation is performed; the network design comprises multiple stages, each outputting feature maps with different spatial resolutions, thus forming a multi-scale feature set for visible light. } and infrared multi-scale feature set { }, where L is the total number of feature scales, These are the Lth visible light feature map and the infrared feature map, respectively;

[0128] S220. Input the feature maps of the two modalities into the temporally enhanced flame-specific attention fusion module to obtain the enhanced fused features. For each corresponding feature scale l, the visible light feature map... With infrared feature map The input is a flame-specific attention fusion module; the flame-specific attention fusion module generates a set of channel-related fusion weights through flame physics prior guidance and temporal dynamic correction. and (satisfy );

[0129] The flame physics prior is guided by calculating the flame thermal diffusion prior map using the real temperature gradient distribution of the infrared thermal imaging image; for a pixel (x, y) in the infrared thermal imaging image, the thermal diffusion prior value at that pixel is determined by the rate of temperature change in its local neighborhood. The calculation formula is:

[0130]

[0131] in, This is an infrared thermal imaging image. Let be the spatial gradient vector of the infrared thermal image at pixel (x,y). Let be the rate of grayscale change along the horizontal direction at pixel (x,y) in the infrared thermal image. This represents the rate of grayscale change along the vertical direction at pixel (x,y) in an infrared thermal image.

[0132] The temporal dynamic correction incorporates optical flow motion information between consecutive frames to calculate the dynamic trend vector of the fire spread direction. It acquires the motion vector and velocity amplitude of each pixel and uses a sparse optical flow algorithm (Lucas-Kanade pyramid algorithm) to calculate the optical flow field V(x,y) between two adjacent infrared thermal imaging frames. For pixels within the fire front region, a temporal correction factor is generated based on the consistency of their motion direction with the overall fire spread direction. ( Pixels along the direction of fire spread receive a higher correction factor, while stationary or reverse-moving regions are suppressed;

[0133] The fusion weights are dynamically generated by the channel attention mechanism within the time-enhanced flame-specific attention fusion module, combined with the thermal diffusion prior map and the time-series correction factor. The weighted fusion process is as follows:

[0134]

[0135] in, This is the feature map after multi-scale fusion at the l-th feature scale;

[0136] This step enables the system to intelligently adjust its dependence on both visible light and infrared thermal imaging modal information based on the physical characteristics and dynamic spread trend of the flame.

[0137] S230. The multi-scale fused features are stitched together to obtain a feature map that integrates multi-modal information.

[0138] The fused feature maps obtained at different scales { The data is uniformly upsampled to the maximum spatial size using bilinear interpolation; subsequently, these data are concatenated along the channel dimension to form a basic fusion feature map containing rich multi-scale contextual information. ;

[0139] S240. The basic fused feature map is input into the superpixel edge protection module. The superpixel edge protection module first divides the basic fused feature map into multiple superpixel regions based on local illumination and texture consistency using a simple linear iterative clustering algorithm. },in, Let K be the Kth superpixel region, and K be the total number of superpixel regions. Based on the temperature gradient distribution of the combustion zone in the infrared thermal imaging image, calculate the edge confidence weight for each superpixel region. For the kth superpixel region, its edge confidence weight... Defined as the normalized result of the average value of the spatial gradient magnitude of the infrared thermal imaging image within the superpixel region, the calculation formula is:

[0140]

[0141] in, This represents the gradient magnitude of the infrared thermal image at each pixel. This represents the Sigmoid normalization function; superpixel regions located at the boundary between the burning and unburning zones (i.e., the area where the fire front is located) receive higher edge confidence weights due to their larger temperature gradients, while smooth background regions are suppressed.

[0142] Through weighted aggregation, the superpixel edge protection module effectively suppresses background noise while preserving the fine edge features of the frontier, generating a fused feature map with enhanced edges. The formula for weighted aggregation is:

[0143]

[0144] in, It belongs to the superpixel region in the basic fusion feature map;

[0145] S250. Input the edge-enhanced fused feature map into the detection head. The detection head consists of a global average pooling layer, several fully connected layers, and an output layer using the Sigmoid activation function. It outputs a confidence score indicating that a fire exists in the current frame. ( The system is based on a preset threshold. Determine if a fire exists: If If the fire is detected, the system is determined to be "fire present" and proceeds to step S300; otherwise, it is determined to be "no fire present" and the system remains in the search state.

[0146] On the other hand, the fire line edge coordinate information extracted by the superpixel edge protection module is transmitted to the composite anti-disturbance control core in real time as a feedforward compensation reference signal for the disturbance observer;

[0147] S300, The image frames of the detected fire from step S200 are processed a second time to identify the fire front and trigger a tracking state switch, including,

[0148] This step is crucial for initiating active tracking. Its core lies in accurately identifying the dynamically spreading fire front boundary from the burning area. First, the burning area is segmented by dividing the infrared image frames that detect the fire and obtaining a binary mask of the burning area. An edge detection algorithm is used to extract the contour of the burning area. Then, combined with optical flow motion analysis between consecutive frames, the optical flow field between the current frame and the previous frame is calculated to obtain the motion vector and velocity amplitude of the pixels. The sequence of pixels located on the contour of the burning area and whose optical flow velocity amplitude exceeds a preset threshold is identified as the fire front. The sequence of pixels with violent movement on the contour is selected as the fire front. Once the identification is successful, the finite state machine inside the intelligent tracking decision unit is immediately triggered to switch from the "search" state to the "capture" state and calculate the initial positioning command to drive the gimbal to quickly pull the fire target into the center of the field of view, creating conditions for subsequent stable tracking.

[0149] Specifically, including,

[0150] S310. Segment the infrared image frame that detects the fire and obtain a binary mask of the burning area. To improve the sensitivity to high-temperature areas, the infrared thermal imaging image is preferred. The image is processed by using an adaptive threshold segmentation algorithm to binarize it and generate a mask image B(x,y).

[0151]

[0152] in, and A pixel value of 1 represents the burning area, which is the mean and standard deviation of the gray level in the local m×n neighborhood of the pixel, and c is an adjustable coefficient; 0 represents the non-burning area (including burned area, unburned area or background).

[0153] S320. Use an edge detection algorithm to extract the contour of the burning area, and perform morphological operations (erosion, dilation) on the binary mask image B to remove noise and small discrete areas, and smooth the boundaries; then, use a contour tracking algorithm to extract the complete and closed contour line set {C1,C2,...} of the burning area, and select the contour with the largest area as the main burning area contour C;

[0154] S330. Based on the optical flow field V(x,y) between two adjacent infrared thermal imaging images calculated in step 220, calculate the motion vector (u,v) of each pixel (x,y) in the infrared thermal imaging image, and calculate the velocity amplitude. Where u and v represent the optical flow velocity components of the pixel in the horizontal and vertical directions, respectively;

[0155] The sequence of pixels located on the combustion zone contour whose optical flow velocity amplitude exceeds a preset threshold is identified as the fire front, and all pixels on the main combustion zone contour C are traversed; for the i-th point on the contour... Check its corresponding optical flow velocity amplitude. ;like Speed ​​greater than the preset threshold If a point is considered to be moving forward, it is considered to be at the forefront of the attack. Connecting all the contour points that meet the conditions in sequence forms the contour curve of the forefront. ;

[0156] S340, once a valid frontline striker is identified (i.e. (Non-empty), which triggers the finite state machine inside the intelligent tracking decision unit to switch from the "search" state to the "capture" state;

[0157] Quickly calculate the initial position of the target in the image and calculate the profile curve of the fire front. The average pixel coordinates of all points on the target point are denoted as ( ). );

[0158] Generate the corresponding gimbal angle command based on the camera intrinsic parameters (focal length). , Main point , The calibration relationship between the gimbal and the camera is used to convert the target point coordinates into pitch angles in the gimbal coordinate system. and yaw angle The two-degree-of-freedom optoelectronic pod gimbal pulls the target into the center of the field of view in the shortest possible time, and transmits angle commands ( The data is sent to the gimbal servo controller, which uses a higher-order trajectory (such as a trapezoidal or S-curve) to drive the two-degree-of-freedom optoelectronic pod gimbal to move rapidly, making the target point ( The image is quickly moved to the area near the center of the field of view to complete the initial capture;

[0159] S400 performs a third processing based on the dynamic motion characteristics of the fire front to obtain precise tracking points characterizing the fire front spread, including:

[0160] This step is the core of maintaining high-quality tracking. Once the system enters the "tracking" state, its core task is to extract the most accurate pixel point representing the fire spread trend and threat direction from each frame of the image, rather than simply the geometric center of the combustion zone. This requires the system to intelligently understand the dynamic characteristics of the fire line. This invention, by comprehensively utilizing the target's motion history information (persistence) and current instantaneous motion characteristics (intensity), enables the generated tracking point to have "foresight" and can automatically lock onto the most active and forward-spreading fire front. Specifically, for each frame of the identified fire front region, the system constructs a motion history image (MHI) value and fuses the instantaneous light flow amplitude to calculate a dynamically weighted centroid point, so that the final tracking point automatically biases towards the most intense and freshest fire front. The calculated pixel coordinate tracking point is converted into control commands for the gimbal pitch and yaw angles through the camera imaging model, i.e., angle control commands, thereby achieving stable and robust tracking of non-rigid, time-varying fire front targets.

[0161] Specifically, including,

[0162] S410. In "tracking" mode, for each newly acquired infrared thermal imaging image frame, the system repeatedly executes the relevant steps in S310 to S330 regarding combustion zone segmentation, contour extraction, and optical flow calculation to update the fire front contour region R of the current frame in real time (based on the extracted fire front contour curve). The system further obtains a set containing the curve and its neighboring thickness pixels, which is defined as the frontier contour region R of the current frame; then, the weighted centroid is calculated based on this region R.

[0163] S420. To enhance the tracking algorithm's ability to perceive the direction of the fire's continuous spread, a motion history image is introduced. The motion history image is a grayscale image of the same size as the original image, where the brightness value of each pixel encodes the "history" and "freshness" of recent motion at that location. Its update rule is: if significant motion is detected at pixel position (x, y) in the current frame, the value of that point in the motion history image is set to the maximum value. Otherwise, its value decays over time; the mathematical expression is as follows:

[0164]

[0165] Where D(x,y,t) is a binary motion detection map obtained from continuous infrared image frames by frame difference method or background subtraction method, τ is the decay time constant (determining the history length), and δ is the decay rate;

[0166] Thus, a key motion history feature map MHI(x,y) is obtained. The higher the value, the more continuous and stable the recent motion at that location is.

[0167] S430, calculate the current optical flow velocity amplitude obtained in step S330. The motion history feature map MHI(x,y) is fused with the optical flow velocity amplitude. First, the motion history feature map and the optical flow velocity amplitude are normalized to ensure their values ​​are in the range [0,1].

[0168]

[0169] in, These are the normalized motion history feature map values ​​and optical flow velocity amplitudes, respectively.

[0170] Then, a fusion weight W(x,y) is calculated for each pixel within the frontline contour region R. This weight determines the contribution of that pixel to the final tracking point position.

[0171]

[0172] Here, λ is a fusion coefficient between 0 and 1, used to balance the importance between the persistence of historical motion (λ approaches 1) and the intensity of current instantaneous motion (λ approaches 0); by adjusting λ, the tracking point can be adapted to the fire line with different characteristics (stable spread or explosive rush).

[0173] S440. Using the dynamic feature weights W(x,y) calculated in step S430, calculate the weighted centroid of the fire front contour region R. The coordinates of this centroid are ( This refers to the precise tracking point selected for the current frame:

[0174]

[0175] This calculation method automatically biases the tracking point towards pixel areas that are both in continuous recent motion and currently in high-speed motion, thereby reliably "locking" onto the true dynamic front of the fire spread and effectively preventing the tracking point from lagging or drifting to the already burned area.

[0176] S450, accurately track the points in the image pixel coordinate system ( The coordinates of the tracking point are converted into angle control commands in the two-degree-of-freedom optoelectronic pod gimbal coordinate system. First, back projection is performed using the camera intrinsic parameters to obtain the coordinates of the precise tracking point on the normalized camera plane. ) :

[0177]

[0178] in,( ) are the coordinates of the principal point of the image, ( , These are the camera's focal lengths in the x and y directions, respectively;

[0179] Next, assuming the gimbal optical axis and the camera optical axis have been precisely aligned, the pitch angle increment required to drive the two-degree-of-freedom optoelectronic pod gimbal to align with this precise tracking point is... and yaw angle increment It can be approximated as:

[0180]

[0181] Finally, to suppress high-frequency jitter in the command caused by image noise or local flicker of the target, the calculated original angle increment is filtered (using a Kalman filter); the smoothed angle control command is then processed. and It is sent in real time to the subsequent composite anti-disturbance servo control core;

[0182] S500: The angle control command and system feedback information calculated in step S400 are processed for the fourth time. The drive signal is calculated through the composite anti-disturbance servo control core, including...

[0183] This step aims to fundamentally solve the problem of high-precision line-of-sight stabilization of the gimbal in the complex disturbance environment of a real fire scene. The vibration of the UAV body, atmospheric turbulence (wind disturbance), and attitude changes generated by its own maneuvering flight constitute a wide-frequency, time-varying composite disturbance, which is difficult for traditional fixed-parameter PID controllers to effectively suppress. Therefore, this invention designs a composite control architecture of "fuzzy adaptive PID controller + disturbance observer (DOB)". The fuzzy adaptive PID controller can dynamically adjust the control parameters according to the real-time tracking error, that is, taking the angle tracking error and its rate of change as input, and adjusting online through a fuzzy rule base. Proportional, integral, and derivative parameters are used to optimize the system's response performance under different dynamic conditions. The disturbance observer, based on the system's nominal dynamic model and feedback from multiple sensors (encoder, inertial measurement unit), estimates and feeds forward to compensate for the total disturbance (including model mismatch, nonlinear friction, and external interference) in real time online. The output of the fuzzy adaptive PID controller is superimposed with the feedforward compensation amount generated by the disturbance observer to form the final composite control signal driving the gimbal servo motor. The two work together to achieve fast and accurate tracking of commands and active and powerful suppression of disturbances while ensuring the stability of the closed-loop system.

[0184] Specifically, including,

[0185] S510. Define the angle tracking error e(t) of a single axis of the gimbal (taking the pitch axis as an example) = and its rate of change ec(t) = , serving as the two input variables of the fuzzy adaptive PID controller; where, The gimbal angle command issued by the intelligent tracking decision unit. The actual angle fed back in real time by the gimbal encoder; the precise error value is mapped to a preset fuzzy language variable domain through a fuzzification interface, such as {Negative Large (NB), Negative Medium (NM), Negative Small (NS), Zero (ZO), Positive Small (PS), Positive Medium (PM), Positive Large (PB)}.

[0186] An expert-experience-based fuzzy rule base is used for online inference. Each rule is in the form of: "IF e is A AND ec is B THEN ΔKp is C, ΔKi is D, ΔKd is E"; for example: "IF e is PB AND ec is NS THEN ΔKp is PM, ΔKi is ZO, ΔKd is NS". Through fuzzy inference and defuzzification, the three parameters of the fuzzy adaptive PID controller are output in real time. Adjustment amount Parameters of the fuzzy adaptive PID controller adjusted in real time , , for:

[0187] , ,

[0188] in, , , These represent the reference proportional parameter, reference integral parameter, and reference derivative parameter of the fuzzy adaptive PID controller, respectively.

[0189] Based on this, the output of the fuzzy adaptive PID controller is calculated. :

[0190]

[0191] S520. Establish the nominal dynamic model of the gimbal servo axis (taking the pitch axis as an example):

[0192]

[0193] in, and The nominal moment of inertia and viscous damping coefficient obtained through system identification. For motor control torque, The lumped disturbances acting on the system include model parameter uncertainties, nonlinear friction, inter-axis coupling, external wind disturbances, and torques transmitted by body vibrations.

[0194] Design a disturbance observer to estimate in real time In the frequency domain, the estimated value of the perturbation It is given by the following formula:

[0195] =Q(s)

[0196] Where Q(s) is a low-pass filter, taking... = This filter is used to limit the bandwidth of the observer, and its time constant is... A balance needs to be struck between the speed of disturbance estimation and the suppression of measurement noise;

[0197] To improve the response speed to high-frequency vibration disturbances, the angular acceleration measured by the inertial measurement unit installed on the UAV body will be... After coordinate system transformation to the gimbal axis, it is introduced as an auxiliary observation signal; the enhanced total disturbance estimation for:

[0198] =

[0199] in, The initial lumped disturbance estimate output by the disturbance observer (i.e. (temporal equivalent representation of (s)) The gain coefficient is calibrated based on the vibration transmission characteristics. This is the converted angular acceleration;

[0200] S530, the final control torque applied to the gimbal servo motor Feedback control quantity generated by fuzzy adaptive PID controller Feedforward compensation amount generated by the disturbance observer Comprehensive generation:

[0201] =

[0202] This structure constitutes a two-degree-of-freedom controller; feedback loop ( Ensure system stability and asymptotic tracking of instructions; feedforward path ( The purpose is to actively "cancel out" the estimated internal and external disturbances, thereby significantly improving the system's tracking accuracy and dynamic anti-interference capability in wide-bandwidth, high-amplitude disturbance environments;

[0203] S600 performs a fifth processing step during the tracking process, diagnosing the tracking status in real time and autonomously predicting and recovering from anomalies such as target loss, including:

[0204] To address tracking interruptions caused by targets being briefly obscured by dense smoke, rapidly moving out of the field of view, or undergoing drastic splitting / merging in fire environments, this step constructs an intelligent recovery mechanism based on multi-dimensional confidence assessment and motion state prediction. While tracking normally, the system continuously evaluates tracking quality in parallel. Once the overall tracking confidence falls below a safety threshold, the target is considered lost, triggering the system's finite state machine to switch to the "recovery" state. In the recovery state, visual measurement updates are paused, and the system instead uses a Kalman filter to predict the target's possible future location based on its historical motion state. A dynamic region of interest (ROI) is defined around the predicted location, guiding the two-degree-of-freedom electro-optical pod gimbal towards this ROI. Simultaneously, the system restarts the target detection algorithm with higher sensitivity only within a single dynamic ROI around the predicted location. This "prediction-guided + local fine-search" strategy achieves rapid re-acquisition of dynamic fire-line targets and efficient maintenance of tracking continuity.

[0205] Specifically, including,

[0206] S610, Multi-dimensional tracking confidence fusion evaluation, defines and calculates a comprehensive tracking confidence level in real time. The overall tracking confidence score is obtained by weighted fusion of sub-confidence scores from the following multiple dimensions:

[0207] Appearance confidence Calculate the similarity of the features (such as HOG, color histogram or CNN features) of the image patch centered on the current tracking point with the corresponding image patches in the previous several frames;

[0208] Motion consistency confidence The degree of agreement between the actual trajectory of the tracking point and the short-term predicted trajectory based on its historical velocity / acceleration is evaluated.

[0209] Confidence of regional features Check whether the local area where the tracking point is located still has significant fire line characteristics (high temperature, specific texture).

[0210] Controller error confidence Monitor whether the tracking error e(t) of the PTZ servo controller remains at a large value or shows a divergent trend for an extended period of time.

[0211] Final overall tracking confidence level It can be calculated as ,in Assign weights to the sub-confidence scores, and satisfy the following conditions: When the overall tracking confidence level N consecutive frames below a preset loss threshold When this occurs, it is determined that "target is lost," triggering a state switch;

[0212] S620, based on Kalman filter, models and predicts the motion of the fire tracking point in the image plane; its state vector It typically includes position and velocity: In normal "tracking" mode, the filter performs a "prediction-update" loop.

[0213] Once the system enters the "recovery" state, due to the loss of target observations, the filter stops updating and only performs time updates (prediction) based on the dynamic model.

[0214]

[0215]

[0216] Where F is the state transition matrix, P is the state estimation error covariance matrix, and Q is the process noise covariance matrix;

[0217] Predicted target location and its uncertainty (by (Description) Together, they define a dynamic "Region of Interest" in the image; this Region of Interest serves as the priority region for subsequent re-search, and its size is proportional to the prediction uncertainty;

[0218] The S630, a two-degree-of-freedom optoelectronic pod, features gimbal retargeting and intelligent research. The gimbal servo controller receives commands and prioritizes adjusting the line of sight to the center of the predicted region of interest, placing that region in the center of the image field of view. Subsequently, the intelligent tracking decision unit, within this region of interest, uses a sensitivity higher than conventional detection (e.g., employing a lower fire detection threshold). (and more refined image analysis), re-execute the frontline detection and recognition algorithm (i.e., the local version of steps S310-S330);

[0219] S640. Once a valid fire front is successfully redetected within the region of interest, the system immediately performs the following operations: the finite state machine is transitioned from the "recovery" state back to the "capture" state; the redetected target point is used as the new initial target, and a gimbal angle command is quickly generated to drive the two-degree-of-freedom electro-optical pod gimbal to re-align; after the two-degree-of-freedom electro-optical pod gimbal stably pulls the target into the center of the field of view, the system automatically switches back to the "tracking" state, and the Kalman filter resumes the complete "prediction-update" cycle, thereby seamlessly restoring continuous and stable tracking of the fire front; if the target is not recaptured within the region of interest within the preset time window, the system will determine that the target has completely left the field of view, and the finite state machine will revert to the larger "search" state to restart the global scan;

[0220] S700, system coordination and closed-loop operation: During the fire, the system cyclically executes steps S400 (intelligent tracking decision), S500 (disturbance control) and S600 (state diagnosis), forming an efficient, robust and adaptive "perception-decision-control-diagnosis" closed loop;

[0221] This step emphasizes deep collaboration and information feedforward / feedback between functional modules to achieve dynamic optimization of the system's overall performance; the intelligent tracking decision unit and the composite disturbance rejection servo control center exchange millisecond-level real-time data via a high-speed communication bus (such as CAN FD); the intelligent tracking decision unit can estimate the amplitude based on the real-time disturbance feedback from the main controller. Or, if the system is nearing saturation, the system adaptively smooths out the angle command output. (For example, by increasing the low-pass filter cutoff frequency) or actively limiting the rate of change of commands, to avoid exciting the mechanical resonance of the two-degree-of-freedom optoelectronic pod gimbal or exceeding its dynamic response capability, thereby achieving the optimal balance between safety and performance at the system level; conversely, the composite anti-disturbance servo control center can also use the target motion prediction information (from the Kalman filter) provided by the intelligent tracking decision unit to perform feedforward compensation in advance and improve the tracking performance under fast-moving targets.

[0222] Specifically, including,

[0223] In scenarios with intense fire movement and strong disturbances, the intelligent tracking decision unit can dynamically increase the historical weight coefficient λ in the weighted centroid calculation, making the tracking point more dependent on relatively stable motion trends rather than instantaneous optical flow that may be noisy, thus outputting smoother commands. At the same time, the composite anti-disturbance servo control center feeds back the currently estimated high-frequency vibration disturbance components to the intelligent tracking decision unit. Based on this, the intelligent tracking decision unit can temporarily increase the weight of "motion consistency" in the comprehensive tracking confidence assessment, because high-frequency disturbances may cause large instantaneous control errors, but this is not target loss. This step acts as the "brain," monitoring the key performance indicators of the entire closed loop (such as average tracking error, target loss rate, and controller saturation), and fine-tuning the key parameters of each module online accordingly, so that the entire system can adapt to fires of different intensities and spread modes, and always maintain excellent overall tracking performance.

[0224] Through the aforementioned tightly coupled, information-sharing, and parameter-adaptive closed-loop operation mechanism, the method described in this invention ultimately achieves fully automatic, all-weather intelligent tracking of dynamic fireline targets in forest fires with high precision, high stability, high adaptability, and high robustness, providing reliable core technical support for real-time monitoring and command decision-making of UAV forest fire fighting.

[0225] Specific Implementation Plan Two: Combining Figures 1 to 4 As shown, this invention provides an unmanned aerial vehicle (UAV) airborne gimbal autonomous tracking system, comprising:

[0226] The system initialization and search control module is configured to perform system power-on self-test, sensor online calibration and communication link establishment; control the two-degree-of-freedom optoelectronic pod gimbal to perform a return-to-zero operation and drive it to move according to a pre-programmed scanning trajectory; and simultaneously control the dual-optical pod to synchronously acquire visible light and infrared image sequences to provide a continuous data source for fire detection.

[0227] The fire detection module is configured to receive continuous frame images acquired by a dual-light pod; extract multi-scale features from visible light and infrared thermal imaging images respectively to obtain visible light multi-scale feature sets and infrared multi-scale feature sets; input the multi-scale feature maps of the two modalities into a temporally enhanced flame-specific attention fusion module to obtain enhanced multi-scale fusion features; stitch the multi-scale fusion features along the channel dimension to obtain a basic fusion feature map that fuses multi-modal information; input the basic fusion feature map into a superpixel edge protection module to obtain an edge-enhanced fusion feature map that preserves the fine edge features of the fire front while suppressing background noise; input the edge-enhanced fusion feature map into a deep learning model with a lightweight convolutional neural network as its backbone for fire detection, output a confidence score indicating the presence of a fire in the current frame, and determine whether a fire exists based on a preset threshold; simultaneously, the fire edge coordinate information extracted by the superpixel edge protection module is transmitted in real time to the composite anti-disturbance servo control center via a high-speed communication bus as a feedforward compensation reference signal for the disturbance observer;

[0228] The fire front identification and initial capture module is configured to segment infrared image frames that indicate the presence of a fire, obtain a binary mask of the burning area; extract the contour of the burning area; calculate the optical flow field between adjacent frames, obtain pixel motion vectors and velocity amplitudes; identify pixel sequences located on the contour of the burning area and whose optical flow velocity exceeds a preset threshold as fire fronts; and after identifying a valid fire front, trigger the system to switch from the "search" state to the "capture" state, quickly calculate the initial position of the target and generate a gimbal angle command, driving the gimbal to quickly pull the target into the center of the field of view.

[0229] The intelligent tracking decision unit is configured to dynamically analyze the fire front region in each frame of image when the system is in "tracking" state; calculate the fusion weight based on dynamic features by constructing a motion history image (MHI) and fusing real-time optical flow information; generate tracking points that accurately represent the fire front when the system is in "tracking" state; convert the pixel coordinates of the tracking points into high-precision gimbal pitch and yaw angle control commands; and manage a finite state machine containing "search", "capture", "tracking" and "recovery" states, and realize intelligent switching between states based on tracking confidence.

[0230] The composite anti-disturbance servo control center is configured to receive angle control commands from the intelligent tracking decision unit and drive the two-degree-of-freedom gimbal servo mechanism to perform precision motion. This module runs a composite control algorithm of "fuzzy adaptive PID controller + disturbance observer (DOB)": the fuzzy adaptive PID controller dynamically adjusts the control parameters online based on the real-time angle tracking error and its rate of change; the disturbance observer estimates and feeds forward to compensate for lumped disturbances, including model uncertainty, nonlinear friction, external wind disturbance, and body vibration, based on the gimbal's nominal dynamic model and feedback from the inertial measurement unit (IMU); through the synthesis of composite control quantities, high-precision and high-stability anti-disturbance tracking is achieved.

[0231] The tracking status diagnosis and autonomous recovery module is configured to calculate a comprehensive tracking confidence index that integrates multiple dimensions such as feature stability and motion consistency in real time during the tracking process. When the comprehensive tracking confidence index is continuously lower than the threshold, the target is determined to be lost and the "recovery" state is triggered. In this state, a motion prediction mechanism based on Kalman filter is launched to predict the future position of the target and delineate a dynamic "region of interest (ROI)". The gimbal is controlled to face the ROI and the front detection is restarted with high sensitivity only in this area to achieve rapid intelligent re-acquisition of the target and recovery of tracking continuity.

[0232] The system collaborative management module is configured to coordinate the data flow, control flow, and state transitions among the aforementioned functional modules; to achieve real-time information exchange between modules (such as decision commands, feedback status, and disturbance estimates) through a high-speed communication bus; and to adaptively adjust the key parameters of each module online based on global system performance indicators (such as average tracking error, target loss rate, and controller saturation) and environmental changes, thereby achieving deep collaboration and overall performance optimization within a closed loop of "perception-decision-control-diagnosis".

[0233] The other combinations and connections in this implementation scheme are the same as in Specific Implementation Scheme 1.

[0234] Simulation and verification experiments

[0235] To verify the effectiveness and robustness of the "method and system for autonomous fire tracking of an unmanned aerial vehicle (UAV) airborne gimbal" described in this invention, a hardware-in-the-loop (HIL) simulation test platform was built and comparative experiments were conducted.

[0236] 1. Experimental equipment and platform setup

[0237] Combination Figure 6As shown, the experimental test platform mainly includes the following equipment: a six-degree-of-freedom vibration table (used to simulate the high-frequency vibration of the rotor and atmospheric turbulence during the flight of the UAV); a two-degree-of-freedom gimbal pod fixed to the top of the vibration table (integrating visible light and infrared detectors and gimbal servo motors); a simulated fire line screen arranged directly below (used to play dynamic infrared video clips of fire line spread collected by the UAV at a real forest fire scene); and a data acquisition and monitoring computer (used to run the core algorithm, record gimbal tracking errors, and issue control commands).

[0238] 2. Validation of the algorithm's feature extraction process

[0239] When the system is in tracking mode, the weighted centroid calculation process of the algorithm of this invention is as follows: Figure 5 As shown. Figure 5 (a) in the image represents the original infrared outline, showing the irregular shape of the fire perimeter. Figure 5 (b) shows a motion history image (MHI), where brighter areas reflect pixels that have been in recent motion. Figure 5 (c) in the image is the optical flow amplitude map, and the dense black dots represent the pixels with violent instantaneous motion attributes in the current frame. Figure 5 Figure (d) shows the fusion weight distribution after integrating historical motion with instantaneous time flow, and the final calculated weighted centroid (tracking point). Experiments show that this weighted algorithm successfully eliminates interference from the already burned zone and static high-temperature targets, enabling the tracking point to be accurately locked on the dynamically spreading "fire front".

[0240] 3. Comparative Experiment and Results Analysis

[0241] The method described in this invention was compared with a traditional method (a visual tracking algorithm known in the art based on conventional image grayscale centroids and fixed-parameter PID). The UAV was subjected to a composite attitude disturbance with an amplitude of 2° and a frequency of 5Hz, and a dynamic forked fireline video was played on the target screen.

[0242] Combination Figure 7 As shown, during continuous tracking moments , Record the performance of both methods: Traditional method performance: Due to the lack of ability to distinguish the dynamic characteristics of the fire line and the inability to effectively suppress external mechanical disturbances, the "tracking point" of the traditional method is... to During this period, the camera gradually drifts towards the center of the largest "burned zone" (i.e., shifts inwards), causing the camera's line of sight to deviate from the most threatening spread front. The method of this invention demonstrates: through "composite disturbance rejection control" to strongly suppress platform vibration and "weighted centroid tracking" to accurately understand the dynamic characteristics of the fire scene, this invention... , Throughout the entire process, not only did the line of sight maintain extremely high stability, but the "tracking point" was also consistently and firmly locked onto the leading edge of the outward-spreading fire line.

[0243] Experimental data conclusions: According to the statistical data from the monitoring computer, in a 20-minute simulated tracking experiment with complex interference, the traditional method resulted in target loss four times due to changes in target shape; while the method of this invention maintained a 100% continuous tracking lock rate. Compared with the traditional algorithm, the root mean square error (RMSE) of the dynamic tracking of the gimbal line of sight in this invention is reduced by 68.5%, significantly improving the practical effectiveness of UAVs in fire monitoring under complex and disturbed environments.

[0244] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A method for autonomous fire tracking using an airborne gimbal on a drone, characterized in that, Includes the following steps: The S100 system initializes and searches for fire lines. After powering on, the system performs hardware self-tests, sensor calibrations, and establishes communication links. The two-degree-of-freedom electro-optical pod gimbal performs a homing operation, and then the intelligent tracking decision unit controls the UAV to enter the preset search state. In this state, the two-degree-of-freedom electro-optical pod gimbal begins to move according to the planned scanning trajectory. It obtains frame images by synchronously acquiring image sequences at a fixed frame rate through the dual-optical pods, and transmits the frame images to the intelligent tracking decision unit in real time for subsequent fire detection and analysis. S200. Perform the first processing on the frame image acquired in step S100 to detect whether there is a fire in the current frame. This includes extracting multi-scale features from the visible light image and infrared thermal imaging image in the frame image to obtain a set of visible light multi-scale features and a set of infrared multi-scale features. Input the multi-scale feature map sets of the two modes into the temporal enhanced flame-specific attention fusion module to obtain the enhanced multi-scale fusion features. The enhanced multi-scale fusion features are concatenated along the channel dimension to obtain a basic fusion feature map that integrates multimodal information; the basic fusion feature map is then input into the superpixel edge protection module to obtain an edge enhancement fusion feature map. The edge-enhanced fusion feature map is input into a deep learning model for fire detection to determine whether a fire exists. S300: The image frames in step S200 that detected the fire are processed a second time to identify the fire front and trigger a tracking state switch. This includes segmenting the infrared image frames that detected the fire and obtaining a binary mask of the burning area; using an edge detection algorithm to extract the contour of the burning area, and then combining it with the optical flow motion analysis between consecutive frames to calculate the optical flow field between the current frame and the previous frame, obtaining the motion vector and velocity amplitude of the pixels, identifying the pixel sequence located on the contour of the burning area and whose optical flow velocity amplitude exceeds a preset threshold as the fire front, and selecting the pixel sequence with violent movement on the contour as the fire front; once the identification is successful, the finite state machine inside the intelligent tracking decision unit is immediately triggered to switch from the search state to the capture state, and the initial positioning command is calculated to drive the two-degree-of-freedom optoelectronic pod of the gimbal to quickly pull the fire target into the center of the field of view; The S400 performs a third processing based on the dynamic motion characteristics of the fire front to obtain precise tracking points representing the fire front spread. This includes identifying the fire front region in each frame. The system constructs motion history image values ​​and fuses instantaneous light flow amplitude values ​​to calculate a dynamically weighted centroid point. The calculated centroid point is converted into control commands for gimbal pitch and yaw angles, i.e., angle control commands, through the camera imaging model. S500: The angle control command and system feedback information calculated in step S400 are processed for the fourth time. The drive signal is calculated through the composite anti-disturbance servo control core. This includes using a fuzzy adaptive PID controller to dynamically adjust the control parameters according to the real-time tracking error, and using a disturbance observer to estimate and feed forward the total disturbance online in real time based on the nominal dynamic model and multi-sensor feedback. The output of the fuzzy adaptive PID controller is superimposed with the feed forward compensation amount generated by the disturbance observer to form the final composite control signal driving the gimbal motor. S600 performs a fifth processing step during the tracking process, diagnosing the tracking status in real time and autonomously predicting and recovering from anomalies such as target loss. This includes constructing an intelligent recovery mechanism based on multi-dimensional confidence assessment and motion state prediction. While tracking normally, the system continuously evaluates the tracking quality in parallel. Once the overall tracking confidence falls below a safety threshold, it is determined that the target has been lost, triggering the system's finite state machine to switch to the recovery state. In the recovery state, visual measurement updates are paused, and instead, a Kalman filter is used to predict the target's possible future location based on its historical motion state. A dynamic region of interest is defined around the predicted location, guiding the two-degree-of-freedom optoelectronic pod gimbal towards this dynamic region of interest, while simultaneously restarting the target detection algorithm.

2. The method for autonomous fire tracking of an unmanned aerial vehicle (UAV) onboard gimbal according to claim 1, characterized in that: In step S100, S110. Check the status of the dual-light pod, gimbal servo controller, inertial detection unit, main controller and each communication link; perform online calibration of the visible light camera and infrared thermal imager, obtain and update the camera intrinsic parameter matrix and extrinsic parameter transformation matrix; the gimbal servo controller controls the two-degree-of-freedom optoelectronic pod gimbal to perform a return-to-zero operation, initializes the finite state machine in the intelligent tracking decision unit, and sets its initial state to the search state. S120: The intelligent tracking decision unit sends scanning commands to the gimbal servo controller. The two-degree-of-freedom optoelectronic pod gimbal moves according to the pre-programmed scanning trajectory. The dual-light pod synchronously acquires visible light and infrared thermal imaging image sequences at a fixed frame rate to obtain frame images, and transmits the frame images to the intelligent tracking decision unit in real time through a high-speed interface.

3. The method for autonomous fire tracking of an unmanned aerial vehicle (UAV) airborne gimbal according to claim 2, characterized in that: In step S200, S210. Perform multi-scale feature extraction on the visible light image and the infrared thermal imaging image respectively. Use a lightweight convolutional neural network as the backbone feature extractor to extract features from the input visible light image. and infrared thermal imaging images Forward propagation is performed; the network design comprises multiple stages, each outputting feature maps with different spatial resolutions, thus forming a multi-scale feature set for visible light. } and infrared multi-scale feature set { }, where L is the total number of feature scales, These are the Lth visible light feature map and the infrared feature map, respectively; S220. Input the feature maps of the two modalities into the temporally enhanced flame-specific attention fusion module to obtain the enhanced fused features. For each corresponding feature scale l, the visible light feature map... With infrared feature map Input flame-specific attention fusion module; The flame-specific attention fusion module generates a set of channel-related fusion weights through flame physics prior guidance and temporal dynamic correction. and ,satisfy ; The flame physics prior is guided by calculating the flame thermal diffusion prior map using the real temperature gradient distribution of the infrared thermal imaging image; for a pixel (x, y) in the infrared thermal imaging image, the thermal diffusion prior value at that pixel is determined by the rate of temperature change in its local neighborhood. : in, This is an infrared thermal imaging image. Let be the spatial gradient vector of the infrared thermal image at pixel (x,y). Let be the rate of grayscale change along the horizontal direction at pixel (x,y) in the infrared thermal image. This represents the rate of grayscale change along the vertical direction at pixel (x,y) in an infrared thermal image. The temporal dynamic correction incorporates optical flow motion information between consecutive frames to calculate the dynamic trend vector of the fire line spread direction. It obtains the motion vector and velocity amplitude of each pixel, and uses a sparse optical flow algorithm to calculate the optical flow field V(x,y) between two adjacent infrared thermal imaging frames. For pixels within the fire front region, a temporal correction factor is generated based on the consistency of their motion direction with the overall fire line spread direction. ( ); The fusion weights are dynamically generated by the channel attention mechanism within the time-enhanced flame-specific attention fusion module, combined with the heat spread prior map and the time-series correction factor. in, This is the feature map after multi-scale fusion at the l-th feature scale; S230. Concatenate the multi-scale fused features to obtain a feature map that fuses multimodal information. Then, combine the fused feature maps obtained at different scales. The data is upsampled to the maximum spatial size using bilinear interpolation; subsequently, the data is stitched together along the channel dimension to form a basic fused feature map. ; S240. The basic fused feature map is input into the superpixel edge protection module. The superpixel edge protection module first divides the basic fused feature map into multiple superpixel regions based on local illumination and texture consistency using a simple linear iterative clustering algorithm. },in, Let K be the Kth superpixel region, and K be the total number of superpixel regions. Based on the temperature gradient distribution of the combustion zone in the infrared thermal imaging image, calculate the edge confidence weight for each superpixel region. For the kth superpixel region, its edge confidence weight... Defined as the normalized result of the average value of the spatial gradient magnitude of the infrared thermal imaging image within the superpixel region, the calculation formula is: in, This represents the gradient magnitude of the infrared thermal image at each pixel. This represents the Sigmoid normalization function; By weighted aggregation, an edge-enhanced fusion feature map is generated. : in, It belongs to the superpixel region in the basic fusion feature map; S250. Input the edge-enhanced fused feature map into the detection head. The detection head consists of a global average pooling layer, several fully connected layers, and an output layer using the Sigmoid activation function. It outputs a confidence score indicating that a fire exists in the current frame. ,and The system is based on preset thresholds. Determine if a fire exists: If If the fire is detected, it is determined that there is a fire, and the process proceeds to step S300; otherwise, it is determined that there is no fire, and the system remains in the search state.

4. The method for autonomous fire tracking of an unmanned aerial vehicle (UAV) onboard gimbal according to claim 3, characterized in that: In step S300, S310. Segment the infrared image frame that detects the fire and obtain the binary mask image B(x,y) of the burning area; in, and The mean and standard deviation of gray values ​​within a local m×n neighborhood of a pixel. This is an adjustable coefficient; 0 represents the non-combustion zone, and 1 represents the combustion zone. S320. Use an edge detection algorithm to extract the contour of the burning area, perform morphological operations on the binary mask image B, and then use a contour tracking algorithm to extract the complete and closed contour line set {C1,C2,...} of the burning area, and select the contour with the largest area as the main burning area contour C; S330. Based on the optical flow field V(x,y) between two adjacent infrared thermal imaging images calculated in step S220, calculate the motion vector (u,v) of each pixel (x,y) in the infrared thermal imaging image and calculate the velocity amplitude. , where u and v represent the optical flow velocity components of the pixel in the horizontal and vertical directions, respectively; The sequence of pixels located on the combustion zone contour whose optical flow velocity amplitude exceeds a preset threshold is identified as the fire front, and all pixels on the main combustion zone contour C are traversed; for the i-th point on the contour... Check its corresponding optical flow velocity amplitude. ;like Speed ​​greater than the preset threshold If a point is considered to be moving forward, it is considered to be at the forefront of the attack. Connecting all the contour points that meet the conditions in sequence forms the contour curve of the forefront. ; S340. Once a valid frontline vanguard is identified, the finite state machine inside the intelligent tracking decision unit is triggered to switch from the "search" state to the capture state. Calculate the initial position of the target in the image and calculate the profile curve of the fire front. The average pixel coordinates of all points on the target point are denoted as ( ). ); Generate the corresponding gimbal angle command, and convert the target point coordinates into a pitch angle in the gimbal coordinate system based on the camera intrinsic parameters and the gimbal-camera calibration relationship. and yaw angle The two-degree-of-freedom optoelectronic pod gimbal initially pulls the target into the center of the field of view, and the angle command is then executed. ) is sent to the PTZ servo controller, so that the target point ( Move the image to the area near the center of the field of view to complete the initial capture.

5. The method for autonomous fire tracking of an unmanned aerial vehicle (UAV) airborne gimbal according to claim 4, characterized in that: In step S400, S410. In tracking mode, the system updates the fire front outline region R of each newly acquired infrared thermal imaging image in real time. S420, Introduce motion history images: Where D(x,y,t) is a binary motion detection map obtained from continuous infrared image frames by frame difference method or background subtraction method, τ is the decay time constant, and δ is the decay rate; S430, calculate the current optical flow velocity amplitude obtained in step S330. It is fused with the motion history feature map MHI(x,y); The motion history feature map and optical flow velocity amplitude were normalized to ensure their values ​​were within the range of [0,1]. in, These are the normalized motion history feature map values ​​and optical flow velocity amplitudes, respectively. Calculate a fusion weight W(x,y) for each pixel within the outline region R of the Firefront Frontline: Where λ is a fusion coefficient between 0 and 1; S440. Using the dynamic feature weights W(x,y) calculated in step S430, calculate the weighted centroid of the fire front contour region R. The coordinates of this centroid are ( This refers to the precise tracking point selected for the current frame: S450, accurately track the points in the image pixel coordinate system ( Convert the angle control command into the coordinate system of the two-degree-of-freedom optoelectronic pod gimbal. By using camera intrinsic parameters for back projection, the coordinates of the precise tracking point on the normalized camera plane can be obtained. ) : in,( ) are the coordinates of the principal point of the image, ( , These are the camera's focal lengths in the x and y directions, respectively; The pitch angle increment required to align the two-degree-of-freedom optoelectronic pod gimbal with the precise tracking point and yaw angle increment The approximate calculation is as follows: The calculated raw angle increment is filtered to smooth the angle control command. and It is sent in real time to the subsequent composite anti-disturbance servo control core.

6. The method for autonomous fire tracking of an unmanned aerial vehicle (UAV) airborne gimbal according to claim 5, characterized in that: In step S500, S510, Define the angle tracking error of a single axis of the gimbal as e(t) = and its rate of change ec(t) = , serving as the two input variables of the fuzzy adaptive PID controller; where, The gimbal angle command issued by the intelligent decision-making unit. The actual angle fed back in real time by the gimbal encoder; the error value is mapped to a preset fuzzy linguistic variable domain through a fuzzification interface; Through fuzzy inference and defuzzification, the three parameters of the fuzzy adaptive PID controller are output in real time. Adjustment amount Parameters of the fuzzy adaptive PID controller adjusted in real time , , for: ， ， in, , , These represent the reference proportional parameter, reference integral parameter, and reference derivative parameter of the fuzzy adaptive PID controller, respectively. Based on this, the output of the fuzzy adaptive PID controller is calculated. : S520. Establish the nominal dynamic model of the gimbal servo axis: in, and The nominal moment of inertia and viscous damping coefficient obtained through system identification. For motor control torque, For lumped disturbances acting on the system; In the frequency domain, the estimated value of the disturbance for: =Q(s) Where Q(s) is a low-pass filter, taking... = , It is a time constant; Angular acceleration measured by an inertial measurement unit installed on the drone body After coordinate system transformation to the gimbal axis, it is introduced as an auxiliary observation signal; the enhanced total disturbance estimation for: = in, This is the initial lumped disturbance estimate output by the disturbance observer; The gain coefficient is calibrated based on the vibration transmission characteristics. This is the converted angular acceleration; S530, feedback control quantity generated by fuzzy adaptive PID controller Feedforward compensation amount generated by the disturbance observer Generate the control torque applied to the gimbal servo motor. : = 。 7. The method for autonomous fire tracking of an unmanned aerial vehicle (UAV) onboard gimbal according to claim 6, characterized in that: In step S600, S610, Multi-dimensional tracking confidence fusion assessment, defining and calculating comprehensive tracking confidence in real time. : in, For the confidence level of the appearance sub-sub, For motion consistency sub-confidence. For regional feature sub-confidence, For controller error sub-confidence, Weights for each confidence level; When the overall tracking confidence level N consecutive frames below a preset loss threshold When this occurs, the target is determined to be lost, triggering a state switch; S620, based on Kalman filter, models and predicts the motion of the fire tracking point in the image plane; its state vector It typically includes position and velocity: Under normal tracking conditions, the filter performs a prediction-update loop. Once the system enters the recovery state, the filter stops updating and only performs time updates based on the dynamic model. Where F is the state transition matrix, P is the state estimation error covariance matrix, and Q is the process noise covariance matrix; Predicted target location Together with their uncertainty, they define a dynamic region of interest in the image; The S630, a two-degree-of-freedom optoelectronic pod gimbal retargeting and intelligent research system, receives instructions to adjust the line of sight to the center of the predicted region of interest, making the region the center of the image field of view; subsequently, the intelligent tracking decision unit re-executes the frontline detection and recognition algorithm within the region of interest with a sensitivity higher than that of conventional detection. S640: Once a valid frontline target is rediscovered within the region of interest, the finite state machine is transitioned from the recovery state back to the acquisition state. Using the rediscovered target point as the new initial target, a gimbal angle command is generated to drive the two-degree-of-freedom electro-optical pod gimbal to re-align. After the two-degree-of-freedom electro-optical pod gimbal stably pulls the target into the center of the field of view, the system automatically switches back to the tracking state, and the Kalman filter resumes the prediction-update loop. If the target is not rediscovered within the region of interest within the preset time window, the system will determine that the target has completely left the field of view, and the finite state machine will revert to a larger-range search state to restart the global scan.

8. The method for autonomous fire tracking of an unmanned aerial vehicle (UAV) onboard gimbal according to claim 7, characterized in that: It also includes S700, system coordination and closed-loop operation. During the fire, the system cyclically executes steps S400-S600, including... The intelligent tracking decision unit estimates the amplitude of the disturbance based on the real-time feedback from the main controller. Or, if the system is nearing saturation, the system adaptively smooths out the angle command output. Alternatively, the rate of change of the command can be actively limited to avoid exciting the mechanical resonance of the two-degree-of-freedom optoelectronic pod gimbal or exceeding its dynamic response capability; conversely, the composite anti-disturbance servo control center uses the target motion prediction information provided by the intelligent tracking decision unit to perform feedforward compensation in advance, thereby improving the tracking performance of fast-moving targets.

9. A drone-borne gimbal-based autonomous tracking system, characterized in that: The system has a program module corresponding to the steps of any one of the claims 1-8 above, and executes the steps in the above-described method for autonomous fire tracking of an unmanned aerial vehicle (UAV) airborne gimbal.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program configured to, when invoked by a processor, implement the steps of the UAV airborne gimbal fireline autonomous tracking method according to any one of claims 1-8.

Images