An unmanned aerial vehicle optoelectronic pod tracking control system and method

Abstract

To address the issues of target tracking drift, servo control jitter, and unsmooth mode switching in existing UAV electro-optical pods under complex environments, this invention provides a UAV electro-optical pod tracking control system and method, relating to the field of UAV intelligent control technology. The method includes: acquiring image information and identifying target category and mission scenario; synchronously outputting tracking and detection results through a tracker and parallel detector to generate tracking evaluation information and determine the tracking status; executing correction, mode switching, and target re-acquisition control strategies based on the tracking status; dynamically adjusting control parameters based on position offset, and performing adaptive drive control in conjunction with actual gimbal attitude and zoom parameters. This invention improves the stability of target tracking, the smoothness of servo control, and the system's environmental adaptability in complex environments through parallel detection and tracking coordination, smooth mode switching, and closed-loop adaptive control.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology for unmanned aerial vehicles (UAVs), specifically to a tracking and control system and method for an UAV photoelectric pod. Background Technology

[0002] Unmanned aerial vehicle (UAV) optoelectronic pods are widely used in security patrol, target surveillance, emergency rescue and other fields. Their core function is to automatically identify, lock onto and continuously track specific targets on the ground or buildings during flight, so that the target remains stable in the center of the pod's field of view.

[0003] Most of the publicly available UAV optoelectronic pod control methods adopt a serial architecture of "detection-tracking-control". This serial architecture can cause the tracking algorithm output results to jump or drift in complex scenarios (such as forests and buildings) and environmental interference such as temporary target occlusion and changes in lighting, which in turn can cause severe oscillations in servo control.

[0004] Secondly, existing control methods for UAV electro-optical pods typically calculate the control values ​​for the gimbal's azimuth and pitch axes based on the pixel deviation between the target center and the image center. This control method can control the gimbal to follow the target when the target's position deviation is relatively stable, but its control parameters usually depend on the current position deviation. This causes the servo control response to depend entirely on the magnitude of the target's position error in the image, resulting in gimbal jitter due to changes in the magnitude of the position error.

[0005] In addition, existing optoelectronic pod tracking systems typically change control parameters or commands directly when switching between working modes such as precise tracking, target correction, and target search, resulting in discontinuity in gimbal movement at the moment of mode switching. Summary of the Invention

[0006] Therefore, this invention provides a UAV optoelectronic pod tracking control system and method with parallel detection-tracking collaboration, closed-loop adaptive optimization mechanism and smooth switching working mode, aiming to solve the problems of tracking drift and servo oscillation in the serial architecture under complex environment, lack of comprehensive judgment of tracking status in servo control system, and distortion and jitter of gimbal control parameters when switching modes in the prior art.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A UAV photoelectric pod tracking and control system includes:

[0009] The external interface layer communicates with the ground control station and includes an image acquisition module and a communication module. The image acquisition module is used to acquire image information within the field of view of the electro-optical pod and output the image information to the algorithm processing layer. The communication module is used to acquire control commands, including target selection commands, and output the target selection commands to the algorithm processing layer.

[0010] The algorithm processing layer, communicating with the external interface layer, includes an image recognition module, a tracker module, a parallel detector module, a target correction module, and an anti-interference module. The image recognition module identifies the target category and task scenario based on image information, determines the tracking target based on the target category and target selection instructions, and outputs the target category and task scenario to the tracker module and parallel detector module, respectively. The task scenario includes at least dense building scenes and open scenes. The tracker module continuously tracks the target by tracking bounding boxes and outputs the tracking results to the target correction module and anti-interference module. The tracking results include at least the coordinates of the target's center position and the coordinates of the tracking bounding box. The parallel detector module processes image information in parallel with the tracker module, detecting the target by detecting bounding boxes. The system tracks the target and outputs the detection results to the target correction module and the anti-interference module. The detection results include at least the coordinates of the center position of the detected target and the coordinates of the detected bounding box. The target correction module calculates the tracking detection position offset based on the coordinates of the center position of the detected target and the center position of the tracked target, generates a compensation control quantity based on the tracking detection position offset, and outputs the compensation control quantity to the main control decision layer. The anti-interference module calculates the bounding box overlap rate based on the coordinates of the detected bounding box and the tracked bounding box, and identifies abnormal tracking bounding boxes based on the bounding box overlap rate. When the bounding box overlap rate is lower than a preset overlap rate threshold, the target confidence meets a preset confidence threshold, and the box rationality index meets a preset rationality threshold, the tracking bounding box is identified as an abnormal tracking bounding box and filtered out.

[0011] The main control decision layer, communicating with the algorithm processing layer, includes a mode management module, a smooth switching module, and a judgment and decision module. The judgment and decision module is configured with a tracking state mapping relationship, used to generate tracking evaluation information based on detection and tracking results, determine the tracking state, and switch the corresponding control strategy according to different tracking states. The control strategies include angle-maintaining control, deviation correction control, and re-acquisition control. The tracking evaluation information includes at least target confidence, bounding box rationality index, and continuous loss time index. The bounding box rationality index characterizes whether the tracking bounding box and the detection bounding box exceed the image boundary and whether the aspect ratio exceeds a preset normal range. The tracking state includes at least a stable tracking state, a pre-loss state, and a target loss state; the target confidence level is used to characterize the reliability of the current target detection result and the target tracking result; the bounding box overlap rate refers to the ratio of the intersection of the tracking bounding box area and the detection bounding box area to the union of the tracking bounding box area and the detection bounding box area; the mode management module is configured with a mode mapping relationship to determine the working mode according to the tracking state and task scenario; the working mode includes at least a search mode, a tracking-parallel detection mode, a tracking enhancement mode, and an anti-disturbance tracking mode; the smooth switching module is used to implement a gradual control transition between the control variables corresponding to the working mode before the switch and the control variables corresponding to the working mode after the switch;

[0012] The servo drive layer, communicating with the main control decision layer, includes a speed adaptive control module, an angle adaptive control module, and a zoom adaptive control module. The speed adaptive control module dynamically adjusts the motor speed control parameters based on the detected tracking position offset and tracking evaluation information; these parameters include proportional coefficients, integral parameters, and derivative parameters. The angle adaptive control module dynamically adjusts the gimbal angle control parameters based on the detected tracking position offset, achieving adaptive control for rapid large-angle rotation and fine small-angle adjustment; these parameters include gimbal angular velocity, angular acceleration, and angular velocity limiting. The zoom adaptive control module dynamically adjusts the zoom control parameters, achieving adaptive matching between the control parameters and the focal length parameters.

[0013] The adaptive optimization layer communicates with the algorithm processing layer, the master control decision layer, and the servo drive layer, respectively, and is used to construct a closed-loop adaptive optimization mechanism between the algorithm processing layer, the master control decision layer, and the servo drive layer. The adaptive optimization layer is used to update the control parameters in the speed adaptive control module, angle adaptive control module, and zoom adaptive control module in the servo drive layer in the next control cycle based on the detection results and tracking results output by the algorithm processing layer and the tracking status feedback from the master control decision layer. The control parameters include motor speed control parameters, gimbal angle control parameters, and zoom control parameters.

[0014] Furthermore, the judgment and decision module is configured with a tracking state mapping relationship, specifically including:

[0015] The tracking state is a stable tracking state when the target confidence level is not lower than the preset confidence threshold and the frame rationality index is not lower than the preset rationality threshold.

[0016] The tracking status corresponding to the target confidence level being lower than the preset confidence threshold or the frame rationality index being lower than the preset rationality threshold is a pre-loss status. The pre-loss status corresponds to the initialization and accumulation of the continuous loss time index.

[0017] In the pre-loss state, the tracking state corresponding to the continuous loss time index exceeding the preset tolerance threshold is the target loss state; the box rationality index is used to characterize whether the detection bounding box and the tracking bounding box exceed the image boundary, and whether the aspect ratio of the bounding box is within the preset normal range.

[0018] Furthermore, the pattern management module is configured with pattern mapping relationships, specifically including:

[0019] The anti-disturbance tracking mode is used when the tracking is stable and the current task scenario is a dense building scene.

[0020] The tracking enhancement mode is corresponding to a stable tracking state and a current task scenario that is an open scene.

[0021] For stable tracking and other task scenarios other than dense building scenes and open scenes, the corresponding tracking-parallel detection mode is used.

[0022] The pre-loss state corresponds to the target correction mode;

[0023] The search mode corresponds to the target loss status.

[0024] Furthermore, the target correction module includes:

[0025] The position offset calculation unit is configured to output horizontal tracking detection position offset and vertical tracking detection position offset based on the detection results and tracking results; the detection results include at least the center position coordinates of the detected target and the position coordinates of the detected bounding box; the tracking results include at least the center position coordinates of the tracked target and the position coordinates of the tracked bounding box.

[0026] The compensation amount generation unit is configured to output azimuth and pitch compensation amounts based on the horizontal tracking detected position offset and the vertical tracking detected position offset.

[0027] Furthermore, the master control decision layer executes corresponding control strategies based on different tracking states, including:

[0028] When the tracking state is stable, the corresponding angle control strategy is maintained, and the speed adaptive control module performs fine-tuning of the motor speed control parameters; the motor speed control parameters include proportional coefficient, integral limit, integral parameters and derivative parameters.

[0029] When the tracking status is in a pre-loss state, the corresponding correction control strategy is implemented, and the angle adaptive control module dynamically adjusts the gimbal angle control parameters; the gimbal angle control parameters include gimbal angular velocity, angular acceleration, and angular velocity limit.

[0030] When the tracking status is that the target is lost, a corresponding re-acquisition control strategy is adopted. The zoom adaptive control module dynamically adjusts the zoom control parameters to expand the gimbal scanning range. The zoom control parameters include the lens focal length.

[0031] A method for tracking and controlling an unmanned aerial vehicle (UAV) optoelectronic pod, implemented based on the aforementioned UAV optoelectronic pod tracking and control system, includes the following steps:

[0032] S10: Receive ground control commands and acquire image information, identify target category and current mission scenario, and determine the tracking target based on the ground control commands and target category; perform parallel processing of the image information using a tracker and a parallel detector, wherein the tracker continuously tracks the tracking target using a tracking bounding box and outputs tracking results, and the parallel detector detects the tracking target using a detection bounding box and outputs detection results; the tracking results include at least the center coordinates of the tracking target and the coordinates of the tracking bounding box, and the detection results include at least the center coordinates of the detected target and the coordinates of the detected bounding box.

[0033] S20: Generate tracking evaluation information based on the detection results and tracking results, calculate the overlap rate between the detected bounding box and the tracking bounding box, and filter out abnormal tracking bounding boxes based on the overlap rate; obtain the tracking status based on the tracking evaluation information; determine the corresponding working mode based on the tracking status and task scenario, and switch from the current working mode to the next working mode through a progressive control transition method.

[0034] S30, execute the corresponding control strategy according to the tracking status; and update the motor speed control parameters, gimbal angle control parameters, and zoom control parameters for the next control cycle according to the detection results, tracking results, tracking status, and tracking evaluation information; the motor speed control parameters include proportional coefficient, integral limit, integral parameter, and derivative parameter; the gimbal angle control parameters include at least gimbal angular velocity, angular acceleration, and angular velocity limit; the zoom control parameters include at least lens focal length.

[0035] Further, in step S20, the process of determining the working mode includes:

[0036] When the tracking state is stable and the current task scenario is a dense building scenario, the system switches to anti-disturbance tracking mode;

[0037] When the tracking status is stable and the current task scenario is an open scene, the system switches to tracking enhancement mode;

[0038] When the tracking state is stable and the current task scenario is a task scenario other than dense building scenario and open space scenario, the system switches to tracking-parallel detection mode;

[0039] When the tracking status is in the pre-loss state, the system switches to target correction mode;

[0040] When the tracking status indicates that the target is lost, the system switches to search mode.

[0041] Furthermore, the anti-interference tracking mode specifically includes:

[0042] The target detection area is determined based on the coordinates of the tracking bounding box position.

[0043] Increase the target detection frequency and anomaly tracking bounding box detection frequency in the target detection region;

[0044] When the target confidence level is lower than the preset confidence threshold or the frame rationality index is lower than the preset rationality threshold, the anti-disturbance tracking mode is exited, and the target correction mode is switched to gradually through control.

[0045] Furthermore, step S30, which executes a corresponding control strategy based on the tracking status, includes:

[0046] When the tracking state is stable, the angle holding control strategy is switched and the motor speed control parameters are finely adjusted; the motor speed control parameters include proportional coefficient, integral limit, integral parameter and derivative parameter.

[0047] When the tracking status is in a pre-loss state, the correction control strategy is switched and the gimbal angle control parameters are dynamically adjusted; the gimbal angle control parameters include gimbal angular velocity, angular acceleration and angular velocity limit.

[0048] When the tracking status is that the target is lost, the reacquisition control strategy is switched and the zoom control parameters are dynamically adjusted to expand the gimbal scanning range. The zoom control parameters include the lens focal length.

[0049] Furthermore, the angle-maintaining control strategy calculates the target center position offset based on the target center position coordinates and the image center position coordinates, and generates an angular velocity control quantity based on the target center position offset; when the target center position offset is within the preset tracking dead zone range, the current attitude of the gimbal is maintained; when the target center position offset exceeds the preset tracking dead zone range, the proportional coefficient, integral limit, integral parameter and differential parameter are finely adjusted.

[0050] The correction control strategy calculates the horizontal and vertical position offsets based on the coordinates of the center position of the detected target and the center position of the tracked target. Combined with the current attitude of the gimbal, the horizontal and vertical position offsets are converted into azimuth and pitch compensations, and the gimbal angular velocity, angular acceleration and angular velocity limit are dynamically adjusted.

[0051] The reacquisition control strategy determines the search center and initial search direction based on the target's movement direction before loss and the gimbal's current attitude, reduces the lens focal length to increase the optical field of view, and controls the gimbal to expand the scanning range according to the preset scanning trajectory with the search center as the reference.

[0052] The present invention has the following advantages over the prior art:

[0053] 1. This invention utilizes a collaborative working mechanism between a parallel detector and a tracker. The judgment and decision module generates tracking evaluation information based on the detection results of the parallel detector and the tracking results of the tracker. The target correction module then makes a comprehensive judgment based on the offset and overlap rate between the detection bounding box and the tracking bounding box, combined with the target confidence and tracking status. This solves the problem of tracking results fluctuating and drifting under environmental interference such as complex backgrounds, temporary target occlusion, and changes in lighting in existing serial architectures, which in turn causes severe oscillations in servo control. This invention improves the stability of target tracking and the smoothness of servo control in complex environments.

[0054] 2. This invention employs a closed-loop adaptive optimization mechanism, using three control strategies—angle maintenance, target correction, and target reacquisition—for the three tracking states of stable tracking, pre-loss, and target loss, respectively. The adaptive optimization layer receives the tracking state and confidence level output from the algorithm layer, as well as the gimbal attitude and lens zoom information fed back from the servo drive layer, to form a closed-loop adaptive adjustment. This solves the problem in existing technologies where servo control relies solely on target offset and lacks the ability to comprehensively discriminate position offset, enabling servo control parameters to adaptively match in real time according to changes in the tracking state.

[0055] 3. This invention exits the current working mode and enters the next working mode through a progressive control transition method, and dynamically determines the mode switching time based on the current tracking status and target confidence level. This solves the problem that the existing system causes the gimbal movement to be inconsistent and the target may be lost when switching between wide-area detection mode and precise tracking mode due to unchanged control parameters. It achieves smooth switching of working modes against disturbances. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a block diagram of a UAV optoelectronic pod tracking and control system in one embodiment;

[0058] Figure 2 This is a flowchart of a UAV optoelectronic pod tracking and control method in one embodiment;

[0059] Figure 3 This is a logic block diagram of the working mode switching in the embodiment.

[0060] Figure label:

[0061] 1-External Interface Layer; 11-Image Acquisition Module; 12-Communication Module; 2-Algorithm Processing Layer; 21-Image Recognition Module; 22-Tracker Module; 23-Parallel Detector Module; 24-Target Correction Module; 25-Anti-interference Module; 3-Main Control Decision Layer; 31-Mode Management Module; 32-Smooth Switching Module; 33-Judgment and Decision Module; 4-Servo Drive Layer; 41-Speed ​​Adaptive Control Module; 42-Angle Adaptive Control Module; 43-Zoom Adaptive Control Module; 5-Adaptive Optimization Layer. Detailed Implementation

[0062] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0063] The following is in conjunction with the appendix Figure 1 To be continued Figure 3 The embodiments of the present invention will be described in detail below. It should be understood that the following embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0064] Example 1

[0065] Please see the appendix Figure 1 This embodiment provides a UAV electro-optical pod tracking and control system, which can be applied to target detection, target tracking, and gimbal servo control scenarios of UAV electro-optical pods. The UAV can be a multi-rotor UAV, a fixed-wing UAV, a compound-wing UAV, or other flight platforms equipped with electro-optical pods. The tracked target can be a vehicle, ship, person, animal, building, facility equipment, or other moving or stationary target that can be identified through photoelectric image recognition.

[0066] In scenarios involving dense buildings, open long-distance surveillance, and short-term target occlusion, inconsistencies may arise between detection and tracking results, and the tracking bounding box may also exhibit jitter or drift. Furthermore, sudden changes in target position error can easily cause abrupt changes in the gimbal's angular velocity and acceleration, leading to servo jitter when switching between different operating modes. To address these issues, this embodiment employs parallel processing of detection and tracking, tracking status assessment, differentiated servo control strategies, and smooth switching control between operating modes to achieve continuous and stable target tracking.

[0067] The UAV electro-optical pod tracking and control system comprises an external interface layer 1, an algorithm processing layer 2, a master control decision layer 3, a servo drive layer 4, and an adaptive optimization layer 5. The external interface layer 1 is used to acquire image information and complete data interaction between the system and external devices; the algorithm processing layer 2 is used to identify the task scenario and target category, and to perform parallel detection, continuous tracking, target correction, and abnormal result processing on the target; the master control decision layer 3 is used to determine the tracking state and corresponding working mode based on the detection and tracking results; the servo drive layer 4 is used to dynamically adjust the corresponding control parameters according to the control strategy; and the adaptive optimization layer 5 is used to update the control parameters for the next control cycle based on the algorithm processing and decision results, thus forming a closed-loop control architecture between image processing, state decision-making, and gimbal control.

[0068] The aforementioned hierarchical closed-loop control architecture links image processing results, tracking status judgment results, and servo execution results according to the control cycle: External interface layer 1 provides image information and target selection information to algorithm processing layer 2; algorithm processing layer 2 provides detection results, tracking results, tracking detection position offset, and bounding box overlap rate to master control decision layer 3; master control decision layer 3 determines the tracking status, working mode, and control strategy based on the above information; servo drive layer 4 drives the gimbal according to the corresponding control strategy; adaptive optimization layer 5 updates the algorithm parameters, mode parameters, and servo control parameters for the next control cycle based on the actual execution results of the gimbal. This avoids parameter mismatches caused by the independent operation of image algorithms, status judgment, and servo control, enabling target position changes to be progressively converted into status judgment results and gimbal control quantities, and allowing control execution results to participate in subsequent parameter updates.

[0069] Specifically, the external interface layer 1 includes an image acquisition module 11 and a communication module 12.

[0070] The image acquisition module 11 is used to acquire image information within the field of view of the optoelectronic pod and output the image information to the image recognition module 21. The image acquisition module 11 may include one or more of a visible light camera, an infrared thermal imager, a zoom lens assembly, and an image acquisition board. The visible light camera can be used for daytime target imaging under good lighting conditions, the infrared thermal imager can be used for nighttime, low-light, or scenarios where there is a temperature difference between the target and the background, and the zoom lens assembly can adjust the optical field of view and target imaging size by changing the lens focal length.

[0071] Image information and ground control commands are uniformly accessed through external interface layer 1, enabling target selection commands to establish a correspondence with candidate targets in the corresponding image frames, reducing the risk of misselection of targets due to the asynchrony between target selection information and image processing results.

[0072] The communication module 12 is used to acquire ground control commands sent by the ground control station and output the ground control commands to the algorithm processing layer 2 and the master control decision layer 3. The ground control commands include at least target selection commands, and may also include working mode switching commands, angle maintenance commands, and tracking exit commands.

[0073] The communication module 12 is also used to feed back the tracking status, gimbal attitude, and current operating mode to the ground control station. The tracking status may be, for example, a stable tracking status, a pre-loss status, or a target loss status; the gimbal attitude may include, for example, the gimbal's current azimuth angle, pitch angle, azimuth axis angular velocity, and pitch axis angular velocity; the current operating mode may be, for example, a tracking-parallel detection mode, a tracking enhancement mode, a target correction mode, a search mode, or an anti-interference tracking mode. The communication module 12 may be implemented using a serial port, Ethernet, CAN bus, wireless data transmission link, or a combination thereof.

[0074] Through the external interface layer 1, ground control commands, real-time image information, and system feedback information can be uniformly connected to the tracking and control system, providing a data foundation for subsequent target determination, status judgment, and servo control.

[0075] Algorithm processing layer 2 is used to perform target category recognition, task scene recognition, target detection, target tracking, target correction, and anomaly result processing to address the problems of missed detections, jumps, or drifts that easily occur when relying solely on detection or tracking results. Algorithm processing layer 2 can be implemented using a GPU computing platform, embedded AI processor, industrial computer, ARM processor, or other computing platforms with image processing capabilities.

[0076] The algorithm processing layer 2 includes an image recognition module 21, a tracker module 22, a parallel detector module 23, a target correction module 24, and an anti-interference module 25.

[0077] The image recognition module 21 is used to identify target categories and task scenarios in image information, obtaining target category information and task scenario information. The target category information is output to the tracker module 22 and the parallel detector module 23, while the task scenario information is output to the main control decision layer 3. The target categories may include vehicles, personnel, ships, animals, building windows, or facilities and equipment. The task scenarios include at least dense building scenes and open scenes, and may also include water scenes and other complex scenes. The system determines the tracking target based on the target selection command and target category information. For example, when the image recognition module 21 identifies multiple vehicle targets, it can select one vehicle as the tracking target based on the target selection command sent by the ground control station.

[0078] The tracker module 22 can be used to establish a target tracking model based on the determined tracking target, continuously track the tracking target through the tracking bounding box, and output the tracking results to the target correction module 24, the anti-interference module 25, and the judgment and decision module 33.

[0079] The tracking results include at least the coordinates of the target center position and the coordinates of the tracking bounding box position, and may also include the target movement direction, target movement speed, and tracking response value. The tracker module 22 may employ a correlation filter tracking algorithm, a twin network tracking algorithm, or other tracking algorithms capable of outputting the target position and tracking bounding box.

[0080] The parallel detector module 23 is used to detect the determined tracking target by detecting the bounding box, and outputs the detection results to the target correction module 24, the anti-interference module 25, and the judgment and decision module 33. The detection results include at least the coordinates of the detected target center position and the coordinates of the detected bounding box position, and may also include the target category, detection confidence, and target category probability. The parallel detector module 23 may employ a neural network-based target detection algorithm or other detection algorithms capable of outputting target category, target position, and detection confidence.

[0081] The tracker module 22 and the parallel detector module 23 operate in parallel. Parallel operation means that within the same image processing cycle, the tracker module 22 and the parallel detector module 23 process the image information separately, without requiring them to physically start or finish their calculations simultaneously. Through parallel operation, the continuity of the tracker module 22 and the target re-identification capability of the parallel detector module 23 can be used to mutually verify the target position.

[0082] Therefore, parallel detection and continuous tracking are not simply a repetition of the same function, but rather provide temporal continuity information and target re-identification information, respectively. When the output positions of the two are basically consistent, the current tracking model can be maintained; when there is only a small positional difference between the two, the gimbal control quantity or tracking bounding box can be corrected by the target correction module 24; when there is a large positional difference between the two, the anti-interference module 25 can further determine whether the tracking result has drifted or whether the detection result is abnormal. Thus, the gimbal jitter caused by frame-by-frame fluctuations of the detection box when only a detector is used can be reduced, and the cumulative drift caused by changes in target appearance, occlusion, or complex backgrounds when only a tracker is used can also be reduced.

[0083] The target correction module 24 is used to calculate the tracking detection position offset based on the center position coordinates of the detected target and the center position coordinates of the tracked target, generate a compensation control quantity based on the tracking detection position offset, and output the compensation control quantity to the main control decision layer 3.

[0084] The target correction module 24 can combine the image resolution and the current optical field of view to convert the horizontal tracking detection position offset and the vertical tracking detection position offset into azimuth compensation and pitch compensation, respectively. When the detected bounding box and the tracking bounding box meet preset correction conditions, and the tracking detection position offset exceeds a preset correction threshold, the target correction module 24 can also correct the position and size of the tracking bounding box based on the position coordinates of the detected bounding box. The preset correction conditions may include a bounding box overlap rate not lower than a preset overlap rate threshold, and the detection result meeting the corresponding confidence and box rationality conditions.

[0085] The anti-interference module 25 is used to calculate the bounding box overlap rate based on the detection bounding box position coordinates and the tracking bounding box position coordinates, and to determine whether the detection result and the tracking result are consistent based on the bounding box overlap rate, detection confidence and box rationality index.

[0086] When the bounding box overlap rate is lower than the preset overlap rate threshold, and the detection confidence is not lower than the preset detection confidence threshold and the detected bounding box meets the preset box rationality conditions, the anti-interference module 25 determines that the tracking result has a risk of drift, filters out abnormal tracking bounding boxes, and outputs the tracker initialization command and the corresponding detection bounding box position coordinates to the tracker module 22, so that the tracker module 22 can re-establish the target tracking model.

[0087] When the bounding box overlap rate is lower than the preset overlap rate threshold, but the detection confidence is lower than the preset detection confidence threshold or the detected bounding box does not meet the preset box rationality condition, the anti-interference module 25 determines the current detection result as an unreliable detection result and filters out the unreliable detection result to avoid erroneous detection results from changing the current tracking target.

[0088] When the bounding box overlap rate is not lower than the preset overlap rate threshold, the anti-interference module 25 maintains the current tracking result and allows the target correction module 24 to generate compensation control quantity when the tracking detection position offset exceeds the preset correction threshold.

[0089] Through the cooperation of the tracker module 22, parallel detector module 23, target correction module 24 and anti-interference module 25, slight position deviation, tracking bounding box drift and abnormal detection results can be distinguished, and target correction, tracker re-initialization or abnormal detection result filtering are adopted respectively, thereby reducing the risk of mistracking caused by complex background and target occlusion.

[0090] The master control decision layer 3 is used to complete tracking status judgment, working mode management, and smooth switching of working modes, in order to solve the problem that the system has difficulty in comprehensively determining the control state based on detection results, tracking results, and task scenarios. The master control decision layer 3 includes a mode management module 31, a smooth switching module 32, and a judgment and decision module 33.

[0091] The judgment and decision module 33 is configured with a tracking state mapping relationship to determine the tracking state and output the tracking state to the mode management module 31, the servo drive layer 4, and the adaptive optimization layer 5. It is also used to generate tracking evaluation information based on the tracking and detection results. This evaluation information includes at least target confidence, bounding box rationality index, and continuous loss time index. Specifically, the target confidence is used to characterize the reliability of the current detection and tracking results; the bounding box rationality index characterizes whether the detection and tracking bounding boxes exceed the image boundaries and whether the aspect ratio of the bounding boxes is within a preset normal range; and the continuous loss time index records the duration for which valid target information is not continuously obtained.

[0092] When the target confidence level is not lower than the preset confidence threshold and the frame rationality index is not lower than the preset rationality threshold, the judgment and decision module 33 determines that the system is in a stable tracking state.

[0093] When the target confidence level is lower than the preset confidence threshold or the box rationality index is lower than the preset rationality threshold, the judgment and decision module 33 determines that the system has entered the pre-loss state, initializes the continuous loss time index, and accumulates the continuous loss time index during the period when the system is in the pre-loss state.

[0094] When the system is in a pre-loss state and the continuous loss time index exceeds the preset tolerance threshold, the judgment and decision module 33 determines that the system has entered the target loss state.

[0095] By setting stable tracking, pre-loss, and target loss states, the system can distinguish between a single frame's confidence drop and continuous target loss. When the target confidence or bounding box validity index temporarily falls below a corresponding threshold, the system first enters the pre-loss state and accumulates the continuous loss time, rather than immediately entering search mode. This allows time for correction and recovery in the event of short-term occlusion, motion blur, or fluctuations in detection confidence. The system only enters the target loss state when the duration of the pre-loss state exceeds a preset tolerance threshold. This time condition suppresses frequent back-and-forth switching between stable tracking and search modes, reducing sudden changes in gimbal movement direction, frequent lens zooming, and repeated restarts of the search trajectory caused by repeated mode switching.

[0096] The mode management module 31 is configured with a mode mapping relationship, which is used to determine the working mode according to the tracking status and task scenario, and output the mode switching instruction to the smooth switching module 32. The working modes include at least the tracking-parallel detection mode, tracking enhancement mode, target correction mode, search mode, and anti-disturbance tracking mode.

[0097] When the system is in a stable tracking state and the current task scenario is a dense building scenario, the mode management module 31 determines that the system enters the anti-disturbance tracking mode.

[0098] When the system is in a stable tracking state and the current task scenario is an open scene, the mode management module 31 determines that the system enters the tracking enhancement mode.

[0099] When the system is in a stable tracking state and the current task scenario is a task scenario other than dense building scenario and open space scenario, the mode management module 31 determines that the system enters the tracking-parallel detection mode.

[0100] When the system is in a pre-loss state, the mode management module 31 determines that the system enters the target correction mode;

[0101] When the system is in a target loss state, the mode management module 31 determines that the system enters the search mode.

[0102] The anti-disturbance tracking mode is used to determine the local target detection area based on the tracking bounding box, increase the target detection frequency and abnormal tracking bounding box detection frequency within the local target detection area, and reduce the maximum output angular velocity, angular acceleration limit and control gain of the gimbal to improve the target locking accuracy in the background of dense buildings.

[0103] The preset tolerance threshold for continuous loss time corresponding to the tracking enhancement mode is greater than the preset tolerance threshold for continuous loss time corresponding to the tracking-parallel detection mode. In open scenes, distant targets may experience a short-term decrease in confidence due to their small imaging size; in tracking enhancement mode, the gimbal movement direction before the target loss can be maintained within the preset tolerance threshold for continuous loss time to prevent the system from entering search mode prematurely.

[0104] The smooth switching module 32 is used to implement a gradual control transition during the exit from the current working mode and the entry into the next working mode, and outputs the smoothed control commands to the servo drive layer 4.

[0105] The progressive control transition includes one or more of the following: gradually decreasing the control output corresponding to the current operating mode, gradually clearing the integral term in the control parameters, sending a stop command, and gradually increasing the control output corresponding to the next operating mode. Gradually clearing the integral term can reduce the integral saturation caused by the carried-over of the accumulated integral from the previous operating mode into the next operating mode; sending a stop command between two operating modes with opposite directions or significantly different control gains can provide a short-term stabilization process for the gimbal motor; gradually increasing the control output corresponding to the next operating mode can reduce the impact during motor startup. These processes collectively reduce angular velocity steps, peak angular acceleration, control overshoot, and reciprocating oscillations during mode switching, ensuring good continuity of the target image during the switching process.

[0106] Let the control variable at the start of the switch be u. a The control variable corresponding to the switched working mode is u. b Then the control quantity during the switching process can be expressed as:

[0107] u(k)=(1-λ k )u a +λ k u b ,

[0108] in, , which is a transition coefficient that gradually increases with the switching cycle. By limiting the changes in control quantity between adjacent control cycles, the sudden changes in gimbal angular velocity and servo jitter caused by the switching of working modes can be reduced.

[0109] The servo drive layer 4 is used to drive the gimbal according to the tracking status and smoothed control commands, in order to solve the problem that using the same control parameters under different tracking states can easily cause insufficient response or control jitter. The servo drive layer 4 includes a speed adaptive control module 41, an angle adaptive control module 42, and a zoom adaptive control module 43.

[0110] Let the coordinates of the image center be (x0, y0), and the coordinates of the target center be (x0, y0).t ,y t If the horizontal and vertical components of the target center position offset are:

[0111] e x (t)=x0-x t ,

[0112] e y (t)=y0-y t ,

[0113] Among them, e x (t) represents the horizontal position deviation, e y (t) represents the vertical position deviation; the target center position offset is used to characterize the degree of deviation of the tracked target from the image center, which is different from the tracking detection position offset between the detection target center and the tracked target center.

[0114] The speed adaptive control module 41 is used to select the target center position offset or the tracking detection position offset as the control error according to the tracking status, and dynamically adjust the proportional coefficient, integral coefficient, integral limit and maximum output angular velocity in combination with the target confidence.

[0115] When the system is in a stable tracking state, the speed adaptive control module 41 executes the holding angle control strategy according to the target center position offset, and uses a small proportional coefficient, a limited integral term and a low maximum output angular velocity to finely adjust the gimbal attitude.

[0116] When the system is in a pre-loss state, the speed adaptive control module 41 executes a correction control strategy based on the tracking and detection position offset to improve the control response speed and adjust the integral limit and maximum output angular velocity.

[0117] When the system is in a target loss state, the speed adaptive control module 41 determines the recapture control parameters based on the target's motion state before loss and increases the scanning speed corresponding to the preset scanning trajectory.

[0118] The angle adaptive control module 42 is used to select the target center position offset or the tracking detection position offset as the angle control error according to the tracking status, and dynamically adjust the azimuth axis angular velocity, pitch axis angular velocity, angular acceleration and angular velocity limit in combination with the current attitude of the gimbal, so as to realize rapid rotation under large angle deviation and fine adjustment under small angle deviation.

[0119] The zoom adaptive control module 43 is used to obtain the current focal length, zoom magnification and field of view parameters of the lens, correct the conversion relationship between pixel offset and angle compensation based on the above optical parameters, and reduce the focal length of the lens when the system executes the recapture control strategy to increase the optical field of view.

[0120] When the system is in a target loss state, the angle adaptive control module 42 determines the search center and initial search direction based on the target's movement direction before loss, the coordinates of the last effective target center, and the current attitude of the gimbal. It then controls the gimbal to expand the scanning range according to a preset scanning trajectory, using the search center as a reference. The preset scanning trajectory can be a reciprocating scanning trajectory, a spiral scanning trajectory, a fan-shaped scanning trajectory, or a grid-like scanning trajectory.

[0121] Servo drive layer 4 feeds back the gimbal's current attitude, azimuth axis angular velocity, pitch axis angular velocity, angular acceleration, lens focal length, zoom magnification, and current servo control parameters to adaptive optimization layer 5 for evaluating the execution results of the current control parameters.

[0122] The adaptive optimization layer 5 is used to determine the algorithm optimization parameters, mode optimization parameters, and servo control optimization parameters for the next control cycle based on the target confidence level, tracking status, task scenario information, and feedback information from the servo drive layer 4. The algorithm optimization parameters include at least the target detection area and target detection frequency; the mode optimization parameters include at least the operating mode and a preset tolerance threshold for continuous loss time; and the servo control optimization parameters include at least the scaling factor, integral limiting, maximum output angular velocity, angular acceleration limiting, and lens focal length.

[0123] The adaptive optimization layer 5 outputs the algorithm optimization parameters to the algorithm processing layer 2, the mode optimization parameters to the master control decision layer 3, and the servo control optimization parameters to the servo drive layer 4. The adaptive optimization layer 5 also updates the algorithm optimization parameters, mode optimization parameters, and servo control optimization parameters for the next control cycle based on the gimbal attitude and lens optical parameters fed back by the servo drive layer 4 after executing the servo control optimization parameters.

[0124] It should be noted that the adaptive optimization layer 5 does not simply pre-select a set of fixed parameters based on the algorithm output, but further utilizes the actual gimbal attitude, angular velocity, angular acceleration, and lens optical parameters fed back by the servo drive layer 4 to evaluate the execution result of the current parameters. When the actual gimbal response is insufficient, the scaling factor, maximum output angular velocity, or angular acceleration limit can be appropriately adjusted in the next control cycle; when the gimbal response is too fast and causes the target position to repeatedly cross the image center, the corresponding control gain can be reduced or the output limit tightened.

[0125] This embodiment provides target motion continuity information and target re-identification information through the tracker module 22 and the parallel detector module 23, respectively. The target correction module 24 handles slight positional differences between the detection and tracking results, and the anti-interference module 25 distinguishes between tracking bounding box drift and unreliable detection results, thus forming a hierarchical processing mechanism for normal tracking, position mismatch, tracking drift, and detection anomalies. The master control decision layer 3 selects the appropriate working mode based on the current target reliability and background environment through a joint mapping of the tracking state and the task scenario. The servo drive layer 4 handles target centering, tracking correction, and target search problems through angle-maintaining control, correction control, and re-acquisition control strategies, respectively. The smooth switching module 32 implements a gradual transition of the control quantities corresponding to different working modes. The adaptive optimization layer 5 then updates the algorithm parameters, mode parameters, and servo control parameters for the next control cycle based on servo execution feedback. The aforementioned modules work together to enable the system to simultaneously suppress cumulative drift of the tracking bounding box, target switching errors caused by erroneous detection results, frequent searches caused by single-frame confidence fluctuations, servo jitter caused by control quantity steps, and compensation errors caused by changes in the pixel-angle conversion relationship before and after zooming. This improves the continuity of target tracking in complex environments, the stability of gimbal control, target re-acquisition capability, and mission scenario adaptability. Therefore, the closed-loop control process of the UAV electro-optical pod tracking control system—"image acquisition—parallel detection and tracking—state judgment—mode and strategy switching—servo drive—parameter feedback update"—can reduce the risks of target tracking bounding box drift, detection result fluctuations, gimbal servo jitter, and discontinuous switching of operating modes in complex environments, improving the stability, continuity, and environmental adaptability of UAV electro-optical pod target tracking.

[0126] Example 2

[0127] Please see the appendix Figure 2 To be continued Figure 3 This embodiment provides a UAV electro-optical pod tracking and control method to solve the problems of target tracking drift, gimbal control jitter, and sudden changes in control quantities when switching between different operating modes caused by inconsistencies between detection and tracking results in complex mission scenarios. This method can be executed by the UAV electro-optical pod tracking and control system described in Embodiment 1.

[0128] The method performs target detection and target tracking in parallel on the target image to obtain detection and tracking results; generates tracking evaluation information based on the consistency between the detection and tracking results to determine the tracking state; determines the working mode based on the tracking state and task scenario, and uses progressive control to transition between working modes; executes angle-maintaining control strategy, correction control strategy, or re-acquisition control strategy based on the tracking state; and finally updates the control parameters for the next control cycle based on feedback information such as gimbal attitude and lens optical parameters, thereby forming a closed-loop tracking control process.

[0129] The UAV electro-optical pod tracking and control method includes steps S10 to S30.

[0130] Step S10: Receive ground control commands and acquire image information within the field of view of the electro-optical pod. The ground control commands include at least target selection commands, and may also include operating mode switching commands, angle maintenance commands, and tracking exit commands.

[0131] The image information is subjected to target category recognition and task scene recognition to obtain the target category and the current task scene. The target category may include categories such as vehicles, people, ships, animals, building windows, and facilities and equipment. The task scene includes at least dense building scenes and open scenes, and may also include forest scenes, water scenes, and other complex scenes.

[0132] The tracking target is determined based on the target selection instruction and the target category. For example, when there are multiple vehicle targets in the image information, one vehicle target is selected as the tracking target based on the target selection instruction.

[0133] Target tracking and target detection are performed in parallel on the image information. In target tracking, the target is continuously tracked using a tracking bounding box to obtain a tracking result; in target detection, the target is detected using a detection bounding box to obtain a detection result. Parallel execution means that target tracking and target detection are performed separately within the same image processing cycle; the two processes do not need to start or end physically simultaneously. The tracking result includes at least the coordinates of the target's center position and the coordinates of the tracking bounding box, and may also include the target's motion direction, motion speed, and tracking response value. The detection result includes at least the coordinates of the detected target's center position, the coordinates of the detected bounding box, and a detection confidence score, and may also include the target category and the target category probability.

[0134] Let the coordinates of the center position of the detected target be (x d ,y d The coordinates of the center position of the tracked target are (x... t ,y tIf the horizontal tracking detection position offset and the vertical tracking detection position offset between the detection result and the tracking result are respectively:

[0135] Δx=x d -x t ,

[0136] Δy=y d -y t ,

[0137] Where Δx represents the tracking and detection position offset in the horizontal direction, and Δy represents the tracking and detection position offset in the vertical direction.

[0138] Step S20: Calculate the area of ​​the detected bounding box and the area of ​​the tracked bounding box based on the position coordinates of the detected bounding box and the position coordinates of the tracked bounding box, and calculate the bounding box overlap rate between the detected bounding box and the tracked bounding box based on the bounding box area.

[0139] Let the area of ​​the detection bounding box be S. d The area of ​​the tracking bounding box is S t Then the bounding box overlap rate is:

[0140] ,

[0141] Among them, R IoU Indicates the bounding box overlap rate. This represents the intersection area of ​​the detected bounding box and the tracked bounding box. This represents the area of ​​the union of the detected bounding box and the tracked bounding box.

[0142] Consistency verification is performed between the detection results and the tracking results based on the bounding box overlap rate, detection confidence, and bounding box rationality.

[0143] When the bounding box overlap rate is lower than the preset overlap rate threshold, and the detection confidence is not lower than the preset detection confidence threshold, and the detected bounding box meets the preset box rationality condition, it is determined that the current tracking bounding box has a drift risk, the tracking bounding box is filtered out, and the tracking bounding box is re-determined based on the position and size of the detected bounding box.

[0144] When the bounding box overlap rate is lower than the preset overlap rate threshold, but the detection confidence is lower than the preset detection confidence threshold, or the detected bounding box does not meet the preset box rationality condition, the current detection result is determined to be unreliable and the detection result is filtered out.

[0145] When the bounding box overlap rate is not lower than the preset overlap rate threshold and the tracking detection position offset exceeds the preset correction threshold, the tracking bounding box is corrected according to the position and size of the detected bounding box.

[0146] Tracking evaluation information is generated based on the detection and tracking results that have undergone consistency verification. This tracking evaluation information includes at least the target confidence level, the reasonableness index of the bounding box, and the continuous loss time index.

[0147] The target confidence score is used to characterize the reliability of the current detection and tracking results; the bounding box rationality index is used to characterize whether the detection bounding box and the tracking bounding box exceed the image boundary, and whether the aspect ratio of the bounding box is within a preset normal range; the continuous loss time index is used to record the duration for which no valid target information is obtained continuously.

[0148] When the target confidence level is not lower than the preset confidence threshold and the box rationality index is not lower than the preset rationality threshold, the system is determined to be in a stable tracking state, and the continuous loss time index is cleared to zero.

[0149] When the target confidence level is lower than the preset confidence threshold, or the box rationality index is lower than the preset rationality threshold, the system is determined to enter the pre-loss state, the continuous loss time index is initialized, and the continuous loss time index is accumulated during the period when the system is in the pre-loss state.

[0150] When the system is in a pre-loss state and the continuous loss time index exceeds the preset tolerance threshold, the system is determined to enter the target loss state.

[0151] When the system regains valid target information before the continuous loss time index exceeds the preset tolerance threshold, the system will be restored to a stable tracking state and the continuous loss time index will be cleared to zero.

[0152] The corresponding working mode is determined based on the tracking status and the current task scenario.

[0153] When the system is in a stable tracking state and the current task scenario is a dense building scenario, determine the anti-disturbance tracking mode;

[0154] When the system is in a stable tracking state and the current task scenario is an open scene, determine the tracking enhancement mode;

[0155] When the system is in a stable tracking state and the current task scenario is a task scenario other than dense building scenarios and open space scenarios, determine the tracking-parallel detection mode;

[0156] When the system is in a pre-loss state, determine the target correction mode;

[0157] When the system is in a state where the target is lost, determine the search mode.

[0158] In anti-disturbance tracking mode, the local target detection area is determined based on the tracking bounding box, the target detection frequency within the local target detection area and the consistency verification frequency between the detection result and the tracking result are increased, and the maximum output angular velocity, angular acceleration limit and control gain of the gimbal are reduced.

[0159] In the tracking enhancement mode, a preset tolerance threshold for continuous loss time is adopted that is greater than that of the tracking-parallel detection mode, and the gimbal movement direction before the target loss is maintained within the preset tolerance threshold for continuous loss time, so as to avoid premature entry into the search mode due to the small imaging size of distant targets or the short-term drop in target confidence.

[0160] When switching from the current operating mode to the next operating mode, a gradual control transition is performed on the control variables before and after the switch. The gradual control transition includes one or more of the following: gradually decreasing the control output corresponding to the current operating mode, gradually clearing the integral term, sending a stop command, and gradually increasing the control output corresponding to the next operating mode.

[0161] Let the control variable at the start of the switch be u. a The control variable corresponding to the switched working mode is u. b The control quantity during the switching process is:

[0162] u(k)=(1-λ k )u a +λ k u b ,

[0163] in, , which is a transition coefficient that gradually increases with the switching cycle. By gradually changing the transition coefficient, the control quantity is changed from u... a Smooth change to u b This reduces sudden changes in gimbal angular velocity caused by switching working modes.

[0164] Step S30: Based on the tracking state obtained in step S20, determine and execute the corresponding control strategy. Specifically, the stable tracking state corresponds to the angle-maintaining control strategy, the pre-loss state corresponds to the deviation correction control strategy, and the target loss state corresponds to the reacquisition control strategy.

[0165] When the system is in a stable tracking state, an angle-maintaining control strategy is adopted to calculate the target center position offset between the target center position coordinates and the image center position coordinates.

[0166] Let the coordinates of the image center be (x0, y0), and the coordinates of the target center be (x0, y0). t ,y t If the target center position offsets in the horizontal and vertical directions are respectively:

[0167] e x (t)=x0-x t ,

[0168] e y (t)=y0-y t ,

[0169] Among them, e x (t) represents the horizontal position deviation, e y (t) represents the vertical position deviation; when the target center position offset in both the horizontal and vertical directions is within the corresponding preset tracking dead zone, the current attitude of the gimbal is maintained.

[0170] When the target center position offset in any direction exceeds the corresponding preset tracking dead zone range, proportional-integral control is used to generate azimuth and pitch axis velocity control quantities. The azimuth and pitch axis velocity control quantities are as follows:

[0171] ,

[0172] ,

[0173] Among them, K px and K py The scaling factors for the azimuth and pitch axes are K, respectively. ix and K iy The integral coefficients for the azimuth and pitch axes are respectively, I. x,max and I y,max These are the integral limits for the azimuth and pitch axes, respectively. and These are the maximum output angular velocities for the azimuth and pitch axes, respectively, and sat() is the amplitude limiting function.

[0174] When the system is in a pre-loss state, a correction control strategy is adopted to generate azimuth and pitch compensation based on the horizontal tracking and vertical tracking position offset.

[0175] Based on the image resolution and the current optical field of view, the azimuth compensation and elevation compensation are as follows:

[0176] ,

[0177] ,

[0178] Where W and H are the number of pixels horizontally and vertically in the image, respectively. and These are the current horizontal and vertical field of view, respectively. The gimbal's target attitude is determined based on the azimuth and pitch compensation amounts and the gimbal's current attitude. The gimbal's angular velocity, angular acceleration, and angular velocity limit are dynamically adjusted according to the magnitude of the tracking and detection position offset. When the tracking and detection position offset is large, the gimbal's response speed is increased; when the tracking and detection position offset gradually decreases, the gimbal's angular velocity and angular acceleration are reduced, allowing the gimbal to gradually approach the corrected target position and avoiding jitter caused by applying a large compensation control amount all at once.

[0179] When the system is in a target loss state, a reacquisition control strategy is adopted. The search center and initial search direction are determined based on the target's direction of motion before loss, the coordinates of the last valid target's center, and the current attitude of the gimbal. The lens focal length is reduced to increase the optical field of view, and the gimbal is controlled to expand the scanning range according to a preset scanning trajectory, using the search center as a reference. The preset scanning trajectory can be a reciprocating scanning trajectory, a spiral scanning trajectory, a fan-shaped scanning trajectory, or a grid-shaped scanning trajectory.

[0180] When a candidate target that matches the target category, appearance features, or motion features of the lost target is detected, and the detection confidence and bounding box rationality indicators of the candidate target meet the corresponding preset conditions, the candidate target is determined as the target to be recaptured, and the tracking bounding box is re-established based on the detection bounding box corresponding to the candidate target.

[0181] After reacquiring the target, the reacquisition control strategy is switched to the angle-holding control strategy, and the scaling factor, integral limit, maximum output angular velocity and lens focal length corresponding to the stable tracking state are gradually restored.

[0182] After executing the angle-maintaining control strategy, the yaw correction control strategy, or the re-acquisition control strategy, the current attitude of the gimbal, the azimuth axis angular velocity, the pitch axis angular velocity, the angular acceleration, the lens focal length, the zoom magnification, and the current control parameters are obtained.

[0183] Based on the detection results, tracking results, tracking status, tracking evaluation information, and the aforementioned feedback information, the target detection parameters, working mode adjustment parameters, and servo control parameters for the next control cycle are updated.

[0184] The target detection parameters include at least the target detection area and the target detection frequency; the working mode adjustment parameters include at least the working mode and a preset tolerance threshold for continuous loss time; the servo control parameters include at least the proportional coefficient, integral coefficient, integral limit, maximum output angular velocity, angular acceleration limit, and lens focal length. For example, when the system is in a stable tracking state and the target center position offset is small, the proportional coefficient and maximum output angular velocity are reduced; when the system enters a pre-loss state and the tracking detection position offset increases, the target detection frequency and gimbal response speed are increased; when the system enters a target loss state, the lens focal length is reduced and the scanning range corresponding to the preset scanning trajectory is expanded.

[0185] The system executes steps S10 to S30 in a loop according to the image processing cycle, continuously updating the detection results, tracking results, tracking status, working mode, and control parameters.

[0186] This embodiment performs target detection and target tracking in parallel, verifies the consistency between the detection and tracking results, and determines the tracking state and working mode based on target confidence, bounding box rationality index, continuous loss time index, bounding box overlap rate, and task scenario. For stable tracking state, pre-loss state, and target loss state, it executes angle-maintaining control strategy, deviation correction control strategy, and re-acquisition control strategy respectively, and updates the processing parameters and control parameters for the next control cycle based on the feedback information after control execution.

[0187] Example 3

[0188] This embodiment uses the example of a drone continuously tracking ground vehicles moving on an open road to illustrate the specific working process of the present invention. All related numerical values ​​are examples and do not constitute limitations.

[0189] The drone, equipped with an electro-optical pod, performs road inspection tasks at a cruising altitude of approximately 80 meters above the ground. The electro-optical pod uses a visible light camera to capture road images with a resolution of 1920×1080, a frame rate of 30 frames per second, and a current horizontal field of view of 60° and a vertical field of view of 34°.

[0190] In this embodiment, the target confidence threshold is preset to 0.75, the bounding box rationality index threshold is 0.80, the bounding box overlap rate threshold is 0.50, and the tracking dead zone corresponding to the target center position offset is no more than 8 pixels in both the horizontal and vertical directions. The preset tolerance threshold for continuous loss time in tracking-parallel detection mode is 0.6s, and the preset tolerance threshold for continuous loss time in tracking enhancement mode is 1.2s.

[0191] It should be noted that the above-mentioned cruising altitude, image resolution, image frame rate, field of view, and various thresholds are all example parameters used in this embodiment, and can be adjusted according to the electro-optical pod model, target type, target movement speed, and actual mission environment.

[0192] The image acquisition module 11 continuously acquires images of the ground road and outputs these images to the image recognition module 21. The image recognition module 21 identifies the current task scene as an open road scene and identifies three vehicles in the road image: a white sedan, a black SUV, and a truck.

[0193] The ground control station sends a target selection command via communication module 12, identifying the white sedan as the tracking target. The white sedan moves along the road from the left to the right of the image at a speed of approximately 12 m / s.

[0194] Tracker module 22 establishes a target tracking model based on the initial position of the white sedan and continuously outputs the center coordinates and bounding box coordinates of the tracked target. Parallel detector module 23 synchronously detects the white sedan and outputs the center coordinates, bounding box coordinates, and detection confidence level of the detected target.

[0195] During a normal tracking cycle, the tracking bounding box output by tracker module 22 is B. t =(950,475,160,90); where the first two parameters represent the x and y coordinates of the top-left corner of the tracking bounding box, and the last two parameters represent the width and height of the tracking bounding box. Therefore, the coordinates of the center position of the tracked target are: (x... t ,y t = (1030, 520); Within the same image processing cycle, the detection bounding box output by the parallel detector module 23 is B. d =(952,473,164,92), the corresponding coordinates of the center position of the detected target are approximately (x... d ,y d = (1034, 519), the detection confidence is 0.94, both the detection bounding box and the tracking bounding box are located within the image boundary, and the aspect ratio is within the preset normal range corresponding to the vehicle bounding box.

[0196] Based on the detected bounding boxes and the tracked bounding boxes, the bounding box overlap rate is calculated to be approximately: R IoU =0.88; The bounding box overlap rate is higher than the preset overlap rate threshold of 0.50, the target confidence level is higher than the preset confidence threshold of 0.75, and the box rationality index is higher than the preset rationality threshold of 0.80. Therefore, the judgment and decision module 33 determines that the system is in a stable tracking state.

[0197] Since the current task scenario is an open road scene, the mode management module 31 determines that the system enters the tracking enhancement mode and extends the preset tolerance threshold for continuous loss time from 0.6s in the tracking-parallel detection mode to 1.2s. The image center position coordinates are: (x0, y0) = (960, 540). Based on the image center position coordinates and the tracking target center position coordinates, the target center position offsets in the horizontal and vertical directions are respectively: e x =x0-x t =960-1030=-70; e y =y0-y t =540-520=20; The target center position offset in both the horizontal and vertical directions exceeds the corresponding preset tracking dead zone, so the azimuth axis velocity control and pitch axis velocity control are generated for fine-tuning.

[0198] During this stable tracking phase, the speed adaptive control module 41 uses a proportional coefficient of 0.30, sets the maximum output angular velocity to 6° / s, and sets the angular acceleration limit to 15° / s², to control the gimbal to precisely follow the vehicle target. As the gimbal rotates, the vehicle target gradually moves to the vicinity of the image center. When the target center position offset in both the horizontal and vertical directions does not exceed 8 pixels, the current attitude of the gimbal is maintained to avoid frequent reciprocating adjustments near the image center.

[0199] The vehicle continued forward and briefly entered the shaded area formed by trees along the roadside. Due to the partial obstruction from the shadows and branches, the detection confidence level decreased from 0.94 to 0.78, and the tracking response value also decreased.

[0200] Within one image processing cycle at this time, the tracking bounding box is: B t =(1040,485,160,90); the bounding box to be detected is B d =(1009,479,166,94); the corresponding coordinates of the center position of the tracked target and the center position of the detected target are approximately (x...). t ,y t )=(1120,530),(x d ,y d )=(1092,526);

[0201] Therefore, the horizontal tracking detection position offset and the vertical tracking detection position offset are obtained as follows:

[0202] Δx=x d -x t =1092-1120=-28, Δy=y d -y t =526-530=-4;

[0203] At this point, the overlap rate between the detected bounding box and the tracking bounding box is approximately 0.66, which is still higher than the preset overlap rate threshold of 0.50. However, the target confidence rate, which is obtained by combining the detection confidence, tracking response value, and bounding box consistency, drops to 0.69, which is lower than the preset confidence threshold of 0.75. Therefore, the judgment and decision module 33 determines that the system has entered the pre-loss state and starts accumulating the continuous loss time index from zero.

[0204] The mode management module 31 switches the system from tracking enhancement mode to target correction mode. The smooth switching module 32 gradually reduces the control output corresponding to the tracking enhancement mode and gradually increases the control output corresponding to the target correction mode to avoid sudden changes in the gimbal angular velocity during the switching.

[0205] Based on the image resolution and the current optical field of view, the horizontal tracking detection position offset and the vertical tracking detection position offset are converted into azimuth and pitch compensation values:

[0206] ,

[0207] ,

[0208] Where W=1920, H=1080, =60 , =30 .

[0209] The target correction module 24 generates compensation control quantities based on the azimuth and pitch compensation quantities. The speed adaptive control module 41 increases the proportional coefficient from 0.30 to 0.45 and the maximum output angular velocity from 6° / s to 10° / s. The angle adaptive control module 42 adjusts the azimuth axis angular velocity, pitch axis angular velocity, and angular acceleration limit based on the compensation control quantities and the current attitude of the gimbal, so that the gimbal maintains the approximate direction of movement of the vehicle target.

[0210] The vehicle was obscured by shadows and tree branches for approximately 0.8 seconds. Since the preset tolerance threshold for continuous loss time in tracking enhancement mode is 1.2 seconds, and the continuous loss time index has not yet exceeded the preset tolerance threshold, the system did not enter the target loss state, nor did it switch to search mode.

[0211] During the image processing cycle before the vehicle leaves the shade, the tracking bounding box shifts significantly towards the road background due to localized strong light and similar road textures. At this point, the tracking bounding box and the detection bounding box are respectively, B t =(1110,505,160,90), B d=(1017,480,166,94); the calculated bounding box overlap rate is approximately: R IoU =0.20. The bounding box overlap rate is lower than the preset overlap rate threshold of 0.50. At the same time, the detection confidence is 0.84, the detected bounding box is located within the image boundary, and the aspect ratio of the detected bounding box meets the preset normal range corresponding to the vehicle target. Therefore, the anti-interference module 25 determines that the tracking bounding box has a drift risk, filters out the abnormal tracking bounding box, and re-establishes the target tracking model based on the position and size of the detected bounding box.

[0212] After the vehicle leaves the shade, the parallel detector module 23 re-outputs a high-confidence detection result. At this time, the detection confidence level recovers to 0.93, the overlap rate between the newly established tracking bounding box and the detection bounding box increases to 0.85, and the box rationality index recovers to above the preset rationality threshold. The judgment and decision module 33 determines that the system has re-entered a stable tracking state and resets the continuous loss time index to zero.

[0213] The system recovers from target correction mode to tracking enhancement mode, and switches from correction control strategy to angle-holding control strategy. In this embodiment, the switching process is set to 10 control cycles, and the transition coefficient of the k-th control cycle is: λ k =k / 10, k=1,2,...,10; the control quantity during the switching process is u(k)=(1-λ) / 10. k )u a +λ k u b , where u a u is the control variable corresponding to the target correction mode. b To track the control quantity corresponding to the enhanced mode. With λ k The control quantity gradually increases from 0.1 to 1, transitioning from a correction control quantity to a stable tracking control quantity. During this switching process, the proportional coefficient gradually recovers from 0.45 to 0.30, the maximum output angular velocity gradually recovers from 10° / s to 6° / s, and the integral term accumulated during the target correction stage is gradually cleared, thereby reducing gimbal jitter caused by sudden changes in control parameters.

[0214] Through the above specific control process, when the vehicle is briefly shaded or obscured by trees, the system uses the extended continuous loss time to preset a tolerance threshold to maintain the vehicle's movement trend, and corrects the gimbal attitude based on the tracking detection position offset between the detection result and the tracking result; when the tracking bounding box drifts significantly, abnormal tracking bounding boxes are filtered out based on the bounding box overlap rate, detection confidence, and box rationality index, and the target tracking model is re-established; when the vehicle reappears clearly, stable tracking parameters are gradually restored through control transition.

[0215] This embodiment can maintain the continuity of target tracking under conditions of short-term vehicle occlusion, changes in lighting, and road background interference, reduce tracking bounding box drift and gimbal servo jitter, and avoid target loss due to premature entry into search mode.

[0216] It should be noted that the cruising altitude, image resolution, field of view, target speed, bounding box coordinates, control parameters, and judgment thresholds listed in this embodiment are only used to illustrate the specific implementation process of the present invention and do not constitute a limitation on the scope of protection of the present invention.

Claims

1. A tracking and control system for an unmanned aerial vehicle (UAV) optoelectronic pod, characterized in that, include: The external interface layer communicates with the ground control station and includes an image acquisition module and a communication module; the image acquisition module is used to acquire image information; and the communication module is used to obtain control commands. The algorithm processing layer, which communicates with the external interface layer, includes an image recognition module, a tracker module, a parallel detector module, a target correction module, and an anti-interference module. The image recognition module determines the tracking target, identifies the target category and task scenario, and outputs the results to the tracker and parallel detector modules. The tracker module tracks the target by tracking bounding boxes and outputs the tracking results to the target correction and anti-interference modules. The parallel detector module detects the tracking target by detecting bounding boxes and outputs the detection results to the target correction and anti-interference modules. The target correction module calculates the tracking detection position offset and generates compensation control variables. The anti-interference module calculates the bounding box overlap rate and filters out abnormal tracking bounding boxes. The main control decision layer communicates with the algorithm processing layer and includes a mode management module, a smooth switching module, and a judgment and decision module. The judgment and decision module is used to generate tracking evaluation information, determine the tracking status, and select the corresponding control strategy. The mode management module is used to determine the corresponding working mode. The smooth switching module is used to achieve smooth switching of the working mode through gradual control transition. The servo drive layer communicates with the main control decision layer and includes a speed adaptive control module, an angle adaptive control module, and a zoom adaptive control module, which are used to dynamically adjust the motor speed control parameters, gimbal angle control parameters, and zoom control parameters, respectively. The adaptive optimization layer communicates with the algorithm processing layer, the master control decision layer, and the servo drive layer, respectively, and is used to update the control parameters for the next control cycle.

2. The UAV photoelectric pod tracking and control system according to claim 1, characterized in that, The judgment and decision module is configured with a tracking state mapping relationship, specifically including: The tracking state is a stable tracking state when the target confidence level is not lower than the preset confidence threshold and the frame rationality index is not lower than the preset rationality threshold. The tracking status corresponding to the target confidence level being lower than the preset confidence threshold or the frame rationality index being lower than the preset rationality threshold is a pre-loss status. The pre-loss status corresponds to the initialization and accumulation of the continuous loss time index. In the pre-loss state, the tracking state corresponding to the continuous loss time index exceeding the preset tolerance threshold is the target loss state; the box rationality index is used to characterize whether the detection bounding box and the tracking bounding box exceed the image boundary, and whether the aspect ratio of the bounding box is within the preset normal range.

3. The UAV photoelectric pod tracking and control system according to claim 1, characterized in that, The pattern management module is configured with pattern mapping relationships, specifically including: The anti-disturbance tracking mode is used when the tracking is stable and the current task scenario is a dense building scene. The tracking enhancement mode is corresponding to a stable tracking state and a current task scenario that is an open scene. For stable tracking and other task scenarios other than dense building scenes and open scenes, the corresponding tracking-parallel detection mode is used. The pre-loss state corresponds to the target correction mode; The search mode corresponds to the target loss status.

4. The UAV photoelectric pod tracking and control system according to claim 1, characterized in that, The target correction module includes: The position offset calculation unit is configured to output horizontal tracking detection position offset and vertical tracking detection position offset based on the detection results and tracking results; the detection results include at least the center position coordinates of the detected target and the position coordinates of the detected bounding box; the tracking results include at least the center position coordinates of the tracked target and the position coordinates of the tracked bounding box. The compensation amount generation unit is configured to output azimuth compensation amount and pitch compensation amount based on the horizontal tracking detection position offset and the vertical tracking detection position offset.

5. The UAV photoelectric pod tracking and control system according to claim 1, characterized in that, The master control decision layer employs different control strategies based on different tracking states, including: When the tracking state is stable, the corresponding angle control strategy is maintained, and the speed adaptive control module finely adjusts the motor speed control parameters. When the tracking status is in the pre-loss state, the corresponding correction control strategy is to dynamically adjust the gimbal angle control parameters of the angle adaptive control module. When the tracking status is that the target is lost, a corresponding re-acquisition control strategy is adopted, and the zoom adaptive control module dynamically adjusts the zoom control parameters to expand the gimbal scanning range.

6. A method for tracking and controlling an unmanned aerial vehicle (UAV) optoelectronic pod, characterized in that, Includes the following steps: S10: Receive ground control commands and acquire image information, identify target category and mission scenario, determine the tracking target based on the ground control commands and target category; perform parallel processing on the image information, continuously track the tracking target through the tracking bounding box and output the tracking result; The tracking target is detected by detecting bounding boxes, and the detection results are output. The tracking result includes at least the coordinates of the center position of the tracked target and the coordinates of the tracked bounding box; the detection result includes at least the coordinates of the center position of the detected target and the coordinates of the detected bounding box. S20, Generate tracking evaluation information based on the detection results and tracking results, calculate the overlap rate between the detected bounding box and the tracking bounding box based on the position coordinates of the detected bounding box and the tracking bounding box, and filter out abnormal tracking bounding boxes based on the overlap rate. The tracking status is obtained based on the tracking evaluation information; It determines the corresponding working mode based on the tracking status and task scenario, and switches from the current working mode to the next working mode through progressive control. S30, execute the corresponding control strategy according to the tracking status; The motor speed control parameters, gimbal angle control parameters, and zoom control parameters for the next control cycle are updated based on the detection results, tracking results, tracking status, and tracking evaluation information.

7. The UAV electro-optical pod tracking and control method according to claim 6, characterized in that, In step S20, the process of determining the working mode includes: When the tracking state is stable and the current task scenario is a dense building scenario, the system switches to anti-disturbance tracking mode; When the tracking status is stable and the current task scenario is an open scene, the system switches to tracking enhancement mode; When the tracking state is stable and the current task scenario is a task scenario other than dense building scenario and open space scenario, the system switches to tracking-parallel detection mode; When the tracking status is in the pre-loss state, the system switches to target correction mode; When the tracking status indicates that the target is lost, the system switches to search mode.

8. The UAV electro-optical pod tracking and control method according to claim 7, characterized in that, The workflow of the anti-interference tracking mode specifically includes: The target detection area is determined based on the coordinates of the tracking bounding box position. Increase the target detection frequency and anomaly tracking bounding box detection frequency in the target detection region; When the target confidence level is lower than the preset confidence threshold or the frame rationality index is lower than the preset rationality threshold, the anti-disturbance tracking mode is exited, and the target correction mode is switched to gradually through control.

9. The UAV electro-optical pod tracking and control method according to claim 6, characterized in that, Step S30 involves executing the corresponding control strategy based on the tracking status, including: When the tracking state is stable, switch the holding angle control strategy and finely adjust the motor speed control parameters; When the tracking status is in a pre-loss state, switch the correction control strategy and dynamically adjust the gimbal angle control parameters. When the tracking status is that the target is lost, switch the reacquisition control strategy and dynamically adjust the zoom control parameters to expand the gimbal scanning range.

10. The UAV electro-optical pod tracking and control method according to claim 9, characterized in that, The angle-maintaining control strategy calculates the target center position offset based on the target center position coordinates and the image center position coordinates, and generates an angular velocity control quantity based on the target center position offset; when the target center position offset is within the preset tracking dead zone range, the current attitude of the gimbal is maintained. When the target center position offset exceeds the preset tracking dead zone range, the motor speed control parameters are finely adjusted. The correction control strategy converts the tracking and detection position offset into azimuth and pitch compensation values ​​based on the current attitude of the gimbal, and dynamically adjusts the gimbal angle control parameters. The reacquisition control strategy determines the search center and initial search direction based on the target's movement direction before loss, the target's center position offset, and the gimbal's current attitude. It dynamically adjusts the zoom control parameters and controls the gimbal to expand the scanning range according to the preset scanning trajectory with the search center as the reference.

Images