An airborne tracking network capture system based on heterogeneous computing and a motion compensation method thereof

Abstract

The present application relates to airborne photoelectric detection and unmanned aerial vehicle countermeasure technical field, disclose a kind of based on heterogeneous computing's airborne tracking net capture system and its motion compensation method, wherein the system includes visual detector, visual processing unit, range finder, main control unit and holder system;Visual detector acquires image, visual processing unit exports target direction, main control unit fuses direction, distance and inertial data to realize motion compensation, trajectory prediction and holder control, holder system executes tracking and net capture.The present application adopts heterogeneous computing architecture to realize the efficient scheduling of computing power, offset the aircraft jitter by hardware level motion compensation, combined with multi-sensor space-time synchronization, inertial coordinate system trajectory smoothing and full-link delay preview compensation, effectively solve the defects of traditional airborne platform visual tracking lag, image jitter, target easy to lose and low interception hit rate, significantly improve the tracking stability, robustness and interception success rate of air-to-air unmanned aerial vehicle net capture system.

Description

Technical Field

[0001] This invention relates to the field of airborne optoelectronic detection and unmanned aerial vehicle (UAV) countermeasures technology. Background Technology

[0002] In recent years, with the gradual opening of low-altitude airspace, rotary-wing drones have been widely used in security patrols, counter-terrorism and bomb disposal, and air defense of key areas. However, the accompanying "black flight" drones have posed a significant threat to low-altitude public safety and the protection of important facilities. Targeting such targets with low collateral damage, such as netting, electronic jamming, and navigation deception, has become an urgent need in the field of low-altitude defense.

[0003] In airborne dynamic interception missions, the carrier platform relies on a gimbal-based optoelectronic system to perform real-time target detection and stable tracking, and calculate the pre-aiming position to capture and engage the target drone. To achieve precise tracking, the system needs to run deep learning target detection and trajectory filtering algorithms at high frequency. However, the computing power of traditional airborne main control units is limited, making it difficult to meet the requirements of parallel execution of high frame rate image processing and high real-time control.

[0004] While current airborne visual tracking technology has made some progress, it still has significant shortcomings in highly dynamic and highly interfered air-to-air capture scenarios. Firstly, the real-time performance of computing power scheduling is poor; deep learning algorithms and real-time control tasks share the same processor, easily leading to timing conflicts and large processing delays. Secondly, multi-sensor data is difficult to synchronize precisely in time and space, resulting in discrepancies between visual, distance, and inertial attitude information, leading to inaccurate target localization. Simultaneously, existing systems have weak motion compensation capabilities, failing to effectively counteract image jitter caused by aircraft flight vibrations and maneuvers, easily resulting in target loss and tracking lag. This leads to low hit rates and insufficient robustness of UAV countermeasure systems during air-to-air interception, making it difficult to meet the requirements for stable locking and accurate capture of maneuvering UAVs in complex low-altitude environments. Summary of the Invention

[0005] The purpose of this invention is to provide an airborne tracking and capture system and its motion compensation method based on heterogeneous computing. By separating computing power tasks through heterogeneous computing, offsetting aircraft jitter through hardware-level motion compensation, and using multi-sensor spatiotemporal synchronization and inertial trajectory prediction, the system solves the problems of airborne tracking lag, easy target loss, and low interception hit rate, and achieves stable locking and accurate capture in high dynamic scenarios.

[0006] To address the aforementioned technical problems, this invention provides an airborne tracking and capture system based on heterogeneous computing, comprising a visual detector, a visual processing unit, a rangefinder, a main control unit, and a gimbal system. The visual detector is connected to the visual processing unit, and both the visual processing unit and the rangefinder are connected to the main control unit, which is in turn connected to the gimbal system. The visual detector acquires target images, the visual processing unit extracts target features and outputs target orientation information, the rangefinder measures the target distance, the main control unit receives the target orientation information, target distance, and inertial measurement data, performs motion compensation and gimbal control, generates a target trajectory and outputs control signals, and the gimbal system performs target tracking and capture / interception.

[0007] The vision processing unit and the main control unit form a heterogeneous computing architecture. The vision processing unit performs image processing, and the main control unit performs deterministic real-time control tasks.

[0008] An inertial measurement unit is rigidly mounted on the back of the visual detector. Based on the data output by the inertial measurement unit, the main control unit sends motion compensation control signals to the gimbal system that satisfy the following relationship: in, For pitch compensation rate, For azimuth compensation rate, The pitch angular velocity, This represents the azimuth angular velocity.

[0009] The visual detector includes a visible light camera and an infrared camera, with the visible light camera integrating a zoom lens.

[0010] This invention also provides a motion compensation method for an airborne tracking and capture system based on heterogeneous computing, comprising the following steps: S1. Heterogeneous perception and alignment: The vision processing unit controls the vision detector to acquire images and detect targets, while the main control unit acquires inertial measurement data. The image detection frequency is lower than the inertial measurement data acquisition frequency. S2, Hardware-level motion compensation: When there is no target, the main control unit sends a motion compensation control signal to the gimbal system, causing the gimbal to rotate in the opposite direction relative to the carrier to stabilize the field of view. S3. Target Angle Calculation: When a target is detected, the vision processing unit calculates the target image coordinates and converts them into the angular deviation of the target relative to the camera optical axis. S4, Angle Compensation Alignment: The main control unit calculates the rotation offset within the image processing cycle based on inertial measurement data and compensates for the angle deviation. S5. Trajectory Prediction: The main control unit uses the compensated angle and target distance as measurement values, calculates the three-dimensional coordinates and velocity of the target inertial coordinate system through a filtering algorithm, and outputs the target predicted trajectory. S6. Interception Control: The main control unit calculates the interception lead angle, controls the pan-tilt unit to point to the pre-aiming position, and triggers the net capture device when the interception conditions are met.

[0011] Step S3 includes: S31: Calculate the pixel deviation between the target image coordinates and the image center coordinates: S32: Based on pixel deviation, image resolution, and the field of view of the lens corresponding to its focal length, calculate the angular deviation of the target relative to the camera's optical axis using the following formula: in, For horizontal pixel deviation, For vertical pixel deviation, For the target image coordinates, The coordinates of the image center are... For image height resolution, For image width resolution, Focal length The corresponding horizontal field of view, Focal length The corresponding vertical field of view, The pitch deviation angle, This is the azimuth deviation angle.

[0012] In step S4, the main control unit compensates and aligns the target deviation angle according to the following formula: in, To compensate for the angle, The original deviation angle, This represents the cumulative rotation offset within the image processing delay period.

[0013] In step S5, when the rangefinder cannot obtain the target distance, the vision processing unit runs a monocular depth estimation network to estimate the target distance and uses the estimated distance as the measurement value of the filtering algorithm.

[0014] In step S5, the filtering algorithm configures an adaptive process noise matrix and adjusts the parameters of the process noise matrix according to the target maneuvering state and the target detection confidence.

[0015] Step S6 includes: S61: Calculate the total system lead time: in, This is the total system lead time. To handle latency in the algorithm, The mechanical response time of the gimbal system. The time required for the drone capture net to fully deploy; S62: Based on the target's current position, velocity, and total lead time, calculate the position of the pre-aiming point in the inertial coordinate system: in, The position of the aiming point in the inertial coordinate system. Let the target's current position be in the inertial coordinate system. Let this be the target's current velocity in the inertial coordinate system; S63: Transform the position of the preview point in the inertial coordinate system to the camera coordinate system; S64: Based on the position of the preview point in the camera coordinate system, calculate the gimbal control angle and drive the gimbal to rotate.

[0016] Compared to existing technologies, this invention effectively solves the technical problems of visual tracking lag, image jitter, and target loss caused by insufficient computing power, poor real-time performance, asynchronous data from multiple sensors, and weak motion compensation capabilities in traditional airborne platforms by employing a heterogeneous computing architecture airborne tracking and capture system and its supporting motion compensation method. It separates the time-consuming image deep learning computation from the high-real-time gimbal control task, which is executed separately by the vision processing unit and the main control unit, significantly improving computing power scheduling efficiency and reducing overall system latency, enabling stable parallel operation of high-frame-rate target detection and high-frequency gimbal control. By rigidly mounting an inertial measurement unit on the back of the vision detector and combining it with hardware-level reverse motion compensation, it directly eliminates optical axis jitter and image shift caused by aircraft motion, ensuring the gimbal's field of view remains stable even under high-maneuverability flight conditions, fundamentally improving the problems of tracking jitter and target loss. Timestamp alignment, angle deviation compensation, and inertial coordinate system trajectory modeling enable precise spatiotemporal synchronization of visual data, distance data, and inertial attitude data, improving the target's 3D positioning accuracy and trajectory smoothness, and ensuring the accuracy and reliability of target motion state estimation. For range measurement failure scenarios, seamless continuation using monocular depth estimation is employed, and adaptive filtering dynamically adjusts weights to further enhance the system's robustness under conditions of strong interference and violent target maneuvers. Simultaneously, through end-link delay calculation and pre-aiming point extrapolation, the gimbal is pre-positioned at the predicted target location, accurately matching the timing of net capture and launch, significantly improving the hit rate of air-to-air interception. Overall, this invention overcomes the shortcomings of traditional airborne countermeasure systems in high-dynamic scenarios, such as poor stability, inaccurate positioning, and delayed response, achieving stable locking, precise tracking, and efficient net capture of aerial maneuvering UAVs, greatly improving the overall performance and combat capability of low-altitude UAV countermeasure systems.

[0017] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0018] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0019] Figure 1 This is a system principle block diagram of at least one embodiment of the present invention; Figure 2 This is a flowchart illustrating at least one embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the embodiments of this invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details are presented in the embodiments of this invention to facilitate a better understanding of this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments. The division of the following embodiments is for ease of description and should not constitute any limitation on the specific implementation of this invention. The embodiments can be combined with and referenced by each other without contradiction.

[0021] Example 1 like Figure 1 The airborne tracking and capture system based on heterogeneous computing shown includes a visual detector, a visual processing unit, a rangefinder, a main control unit, and a gimbal system; the visual detector is connected to the visual processing unit, the visual processing unit and the rangefinder are both connected to the main control unit, and the main control unit is connected to the gimbal system; Visual detectors are used to acquire target images, providing raw data sources for target detection and forming the foundation for the entire system to achieve target perception. The vision processing unit is used to extract target features and output target orientation information, complete target recognition and positioning, and convert image information into orientation data that can be used for control. Rangefinders are used to measure the distance to a target, providing distance dimension information of the target's spatial location, which, together with orientation information, forms the basis of three-dimensional positioning; The main control unit is used to receive target azimuth information, target distance and inertial measurement data, perform motion compensation and gimbal control, generate target trajectory and output control signals, and undertake the core scheduling functions of system data fusion, motion compensation, trajectory prediction and command output; The PTZ system is used to perform target tracking and net capture interception, and completes line-of-sight stabilization, target following and physical net capture execution. It is the execution terminal for the interaction between the system and the target.

[0022] Thus, through a modular and hierarchical hardware architecture and clear data flow, a complete UAV interception system suitable for air-to-air scenarios is constructed. It can independently complete the entire process of target detection, feature extraction, distance measurement, attitude compensation, trajectory generation, and capture net launch. The architecture has strong versatility and high hardware feasibility, which can effectively solve the problems of traditional airborne countermeasure systems, such as dispersed functions, poor coordination, and difficulty in forming a complete interception loop. It provides reliable hardware support for stable tracking and precise capture of UAVs under high dynamic flight conditions.

[0023] Example 2 Based on Embodiment 1, the vision processing unit and the main control unit constitute a heterogeneous computing architecture. The vision processing unit performs image calculations, and the main control unit performs deterministic real-time control tasks. This separates the image calculations, which consume a lot of computing power and have a non-fixed time consumption, from the control tasks, which have high real-time requirements and fixed timing, thereby avoiding computing power preemption and timing conflicts and ensuring the real-time response of the system.

[0024] Furthermore, an inertial measurement unit is rigidly mounted on the back of the visual detector. Based on the data output by the inertial measurement unit, the main control unit sends motion compensation control signals to the gimbal system that satisfy the following relationship: in, For pitch compensation rate, For azimuth compensation rate, The pitch angular velocity, The azimuth angular velocity; rigid installation can eliminate measurement errors caused by installation gaps, and reverse rate compensation can directly offset the optical axis jitter caused by the movement of the carrier aircraft, thus stabilizing the field of view from a hardware perspective.

[0025] Furthermore, the visual detector includes a visible light camera and an infrared camera, with the visible light camera integrating a zoom lens; the dual-camera fusion can adapt to complex environments such as day and night, strong light, and backlight, and the zoom lens can adjust the field of view according to the target distance, improving the ability to detect distant targets.

[0026] Thus, through three core improvements—heterogeneous computing architecture, hardware-level motion compensation, and multi-segment adaptive sensing—the system simultaneously overcomes the technical bottlenecks of traditional airborne tracking systems, such as insufficient computing power, chaotic timing, tracking jitter, and weak environmental adaptability. The heterogeneous architecture enables efficient scheduling of computing power, reducing overall system latency. By utilizing IMU data to control the gimbal to rotate in the opposite direction relative to the carrier aircraft at the same rate, the system directly eliminates image jitter caused by carrier aircraft maneuvers, significantly improving field-of-view stability. The dual-light imaging and zoom design greatly expands the applicable scenarios and detection range, enabling the system to continuously and stably acquire target information under complex weather and highly dynamic flight conditions, laying the hardware foundation for subsequent accurate tracking and interception.

[0027] Example 3 like Figure 2 The motion compensation method for an airborne tracking and capture system based on heterogeneous computing, as shown, includes the following steps: S1. Heterogeneous Perception and Alignment: The vision processing unit controls the vision detector to acquire images and detect targets, while the main control unit acquires inertial measurement data. The image detection frequency is lower than the inertial measurement data acquisition frequency. High-frequency acquisition of inertial data can accurately capture changes in the carrier's motion, while low-frequency vision detection balances computing power consumption. The combination of high and low frequencies takes into account both accuracy and efficiency.

[0028] S2. Hardware-level motion compensation: When there is no target, the main control unit sends a motion compensation control signal to the gimbal system to drive the gimbal to rotate in the opposite direction to stabilize the field of view; when there is no target, the field of view is continuously stabilized, which can avoid the inability to quickly capture the target when it enters the field of view due to screen shaking.

[0029] S3. Target Angle Calculation: When a target is detected, the vision processing unit calculates the target image coordinates and converts them into the angular deviation of the target relative to the camera optical axis; the pixel coordinates are converted into the angular deviation, and the image information is converted into angle commands that can be used for gimbal control.

[0030] S4. Angle Compensation Alignment: The main control unit calculates the rotational offset of the carrier aircraft within the image processing cycle based on inertial measurement data and compensates for the angle deviation; it eliminates the angle error caused by image processing delay and ensures the real-time accuracy of target positioning.

[0031] S5. Trajectory Prediction: The main control unit uses the compensated angle and target distance as measurement values, calculates the three-dimensional coordinates and velocity of the target inertial coordinate system through a filtering algorithm, and outputs the target predicted trajectory. Since the inertial coordinate system is fixed, the coordinate calculation in this coordinate system can shield the influence caused by the shaking of the carrier aircraft, and output a smooth and continuous target motion trajectory.

[0032] S6. Interception Control: The main control unit calculates the interception lead angle and controls the PTZ to point to the pre-aiming position. When the interception conditions are met, the net capture device is triggered; it points to the predicted position of the target in advance to make up for system delay and improve the probability of net capture hit.

[0033] Thus, a stable tracking mechanism is constructed through heterogeneous asynchronous sensing, hardware image stabilization, image processing delay compensation, inertial trajectory prediction, and dynamic forward interception. A closed loop is formed from data acquisition, motion stabilization, target localization, trajectory smoothing to interception decision-making. This significantly reduces the tracking lag and target loss risk caused by high-maneuverability flight, enabling the system to maintain continuous locking capability in air-to-air interception scenarios with strong interference and high dynamics, and greatly improving tracking stability and interception reliability.

[0034] Example 4 Based on Example 3, step S3 includes: S31: Calculate the pixel deviation between the target image coordinates and the image center coordinates: Calculating the deviation using the image center as a reference directly reflects the degree of deviation between the target and the optical axis, providing a benchmark for angle conversion.

[0035] S32: Based on pixel deviation, image resolution, and the field of view of the lens corresponding to its focal length, calculate the angular deviation of the target relative to the camera's optical axis using the following formula: in, For horizontal pixel deviation, For vertical pixel deviation, For the target image coordinates, The coordinates of the image center are... For image height resolution, For image width resolution, Focal length The corresponding horizontal field of view, Focal length The corresponding vertical field of view, The pitch deviation angle, This is the azimuth deviation angle; by establishing a mapping relationship between pixels and angles through resolution and field of view, precise quantification of target positioning can be achieved.

[0036] Furthermore, in step S4, the main control unit compensates for and aligns the target deviation angle according to the following formula: in, To compensate for the angle, The original deviation angle, This represents the cumulative rotational offset within the image processing delay period; the motion offset of the carrier aircraft during the delay period is directly deducted to ensure that the target angle data matches the current optical axis attitude in real time.

[0037] Furthermore, in step S5, when the rangefinder cannot obtain the target distance, the vision processing unit runs a monocular depth estimation network to estimate the target distance and uses the estimated distance as the measurement value of the filtering algorithm; in the event of rangefinder failure, the distance data is kept continuous to avoid positioning interruption leading to gimbal malfunction.

[0038] Furthermore, in step S5, the filtering algorithm configures an adaptive process noise matrix and adjusts the parameters of the process noise matrix according to the target maneuvering state and the target detection confidence; it dynamically adjusts the filtering weights to increase the confidence in the measured data when the target is maneuvering violently, and avoids lag caused by excessive trajectory smoothing.

[0039] Furthermore, step S6 includes: S61: Calculate the total system lead time: in, This is the total system lead time. To handle latency in the algorithm, The mechanical response time of the gimbal system. The time required for the drone capture net to fully deploy is calculated; by considering the overall latency, a precise advance prediction time is determined.

[0040] S62: Based on the target's current position, velocity, and total lead time, calculate the position of the pre-aiming point in the inertial coordinate system: in, The position of the aiming point in the inertial coordinate system. Let the target's current position be in the inertial coordinate system. The target's current velocity in the inertial coordinate system is used to extrapolate the target's future position based on the uniform velocity model, providing the gimbal with precise target pointing.

[0041] S63: Transform the position of the preview point in the inertial coordinate system to the camera coordinate system; unify the coordinate system so that the preview point can be directly used for gimbal angle calculation.

[0042] S64: Based on the position of the preview point in the camera coordinate system, calculate the gimbal control angle and drive the gimbal to rotate; convert the spatial position into the motor control angle to achieve precise gimbal pointing.

[0043] Thus, through precise formulaic calculations, seamless switching of multi-sensor data, adaptive filtering correction, and full-link delay pre-compensation, the entire process of fine-grained control from coarse target tracking to fine tracking and then to precise interception is achieved. This completely solves the problems of inaccurate positioning, large delays, loss of targets due to ranging interruptions, and lack of interception prediction in traditional systems. It makes the target trajectory smoother, the gimbal movement more stable, and the timing of net capture and launch more precise. Even under complex working conditions such as high maneuverability, strong interference, and long distances, it can still maintain extremely high tracking robustness and interception success rate, comprehensively improving the overall performance of the airborne UAV net capture system.

[0044] Example 5 Based on the above embodiments, the main aspects involved are as follows: Figure 1 The airborne tracking and capture system based on heterogeneous computing, as shown, features a visual processing unit and a main control unit that are heterogeneously connected. The former performs target detection and outputs the target position at a non-fixed frequency, while the latter runs filtering algorithms and motion compensation at a fixed frequency. Based on this heterogeneous computing architecture, it achieves airborne high-dynamic target recognition, accurate trajectory prediction, and physical capture and interception. This includes: The visual detector, which includes a visible light camera and an infrared camera, can reliably acquire images of targets in environments such as strong light, smoke, night, and backlight. The vision processing unit is used to run deep learning algorithms to extract features of the target UAV and adjust the weights of visible light and infrared images in target detection in real time according to the lighting environment, and has the function of friend or foe identification. A rangefinder is used to measure the distance to a target. The main control unit, heterogeneously connected to the vision processing unit, is used to execute motion compensation algorithms and gimbal control. Based on the target orientation information returned by the vision processing unit, the target distance information returned by the rangefinder, and the angular velocity data returned by the inertial measurement unit (IMU), it uses a filtering algorithm to perform state estimation, generates a real-time smooth trajectory of the target, sends motion control signals to the gimbal system at high frequency, and determines whether the target is within the range of the net capture device. The gimbal system consists of a pitch and azimuth module and a drone net capture device. A visual detector is mounted on the gimbal system and moves with the gimbal to track the target. Preferably, the vision processing unit and the main control unit constitute a heterogeneous computing architecture, wherein the vision processing unit is responsible for nondeterministic and time-consuming image processing, and the main control unit is responsible for deterministic and time-series hard real-time control tasks. Preferably, the visible light camera in the visual detector integrates a zoom lens that automatically adjusts the focal length according to the distance to the target, thereby increasing the target detection range; Preferably, an IMU is rigidly mounted on the back of the visual detector to sense changes in the spatial orientation of the camera's optical axis. The main control unit sends a high-frequency reverse motion signal to the gimbal system based on the changes in orientation to perform motion compensation and reduce image jitter caused by the movement of the carrier aircraft.

[0045] Example 6 Based on Example 5, it mainly involves, as follows Figure 2 The motion compensation method for the airborne tracking and capture system based on heterogeneous computing, as shown, includes the following steps: S1: Heterogeneous Perception and Alignment: The visual processing unit controls the visual detector to acquire images at a frequency H C Whether a target drone is present in the images acquired by the computer vision detector is simultaneously determined by the main control unit at frequency H. I Acquire motion data from the IMU (a X a Y a Z ω X ω Y ω Z T I ), where H I >H C T I For the timestamp of motion data acquisition, (a X a Y a Z ) and (ω X ω Y ω Z These are acceleration and angular velocity data, respectively. S2: Hardware-level motion compensation: When there is no UAV target in the field of view, the main control unit does not receive the original UAV image coordinate data sent by the vision processing unit. At this time, the main control unit sends a signal to the gimbal system at frequency H. I Send motion compensation control signals, including pitch compensation rate V. CP and azimuth compensation rate V CY V CP= -ω Y V CY= -ω Z This causes the gimbal motors to rotate in the opposite direction relative to the carrier at the same speed, maintaining the stability of the image; S3 UAV Target Detection: When a UAV target appears in the field of view, the visual computing unit first calculates the original image coordinates (X...) of the UAV. U Y U Record the timestamp T of this image acquisition. C Then, based on the lens's focal length and field of view, calculate the angular deviation (Δψ, Δθ) between the target and the camera's optical axis, where X... U and Y U Δψ and Δθ are the horizontal and vertical coordinates of the target UAV in the image, respectively, and Δψ and Δθ are the azimuth and pitch deviation angles, respectively. S4 Motion Angle Compensation: After the vision processing unit detects the target, it modifies the target deflection angle with the timestamp (Δψ, Δθ, T).C The data is sent to the main control unit, which calculates the cumulative rotational offset (ψ) of the visual detector within the image processing delay period based on the motion data. O θ O The target deviation angle is compensated and aligned to obtain the compensated angle (Δψ). O , Δθ O ): , ; S5 Target Tracking and Trajectory Prediction: The main control unit will calculate the compensated target angle and the target distance (Δψ) returned by the rangefinder. O , Δθ O The target vector (R) is used as a measurement value to correct the current value of the prediction filtering algorithm (such as KF, EKF, UKF, etc.). The filtering prediction algorithm uses the target coordinates in the inertial coordinate system Sn for calculation. In this coordinate system, the target vector is absolute. Even if the base of the carrier aircraft is shaking, the change in the target's coordinates is smooth, which can obtain an accurate velocity estimate. The calculation is first based on the angular deviation (Δψ) of the target relative to the optical axis after compensation. O , Δθ O From the range value R, we obtain the three-dimensional coordinates (x, y) of the target in the camera coordinate system. c y c , z c ): Then convert it to a point in the inertial coordinate system: Where (x) n ,y n ,z n Let R be the three-dimensional coordinates of the target in the inertial coordinate system. cn Let t be the rotation matrix and t be the translation matrix. Both are calculated using the aircraft displacement and IMU data. To describe the motion of the target UAV in the inertial coordinate system and predict its trajectory, the state variables must include position and velocity. Therefore, the state vector is: Among them (v) xn ,v yn ,v zn The target's three-dimensional velocity in the inertial coordinate system is used. A filtering algorithm is applied to this state vector to output the high-frequency real-time predicted trajectory of the target UAV within the visual processing interval. The trajectory is then updated when the measured value is received. S6 interception strategy generation: The main control unit calculates the interception lead angle at high frequency, drives the gimbal system to point to the position where the target is about to arrive in advance, and controls the drone net capture device to launch when the preset interception envelope conditions are met; Preferably, in step S3, the original image coordinates (X...) of the UAV are... U Y U When converting to angular deviations (Δψ, Δθ) from the camera's optical axis, first calculate the original UAV image coordinates (X). U Y U The difference between the coordinates of the image center (u0, v0) and the coordinates of the image center (u0, v0): Where Δu is the horizontal pixel deviation between the original UAV image coordinates and the image center, and Δv is the vertical pixel deviation between the original UAV image coordinates and the image center, then the angular deviation (Δψ, Δθ) between the target and the camera optical axis is calculated: Among them, W p H represents the resolution in the image width direction. p FOV is the resolution in the height direction of the image. H f FOV is the horizontal field of view of a lens at its focal length f. V f The vertical field of view is the angle of view when the lens focal length is f. Preferably, in step S4, the main control unit receives the target data packet sent by the vision processing unit using direct memory access (DMA) and triggers data parsing using serial port idle interrupt, recording the vision frame timestamp corresponding to the data packet, thereby achieving strict alignment of vision data and IMU data on the time axis. Preferably, in step S5, since the rangefinder is installed next to the camera and moves synchronously with it, when the target UAV deviates significantly from the camera's optical axis, the rangefinder cannot obtain the target distance R. In this case, the vision processing unit runs a monocular depth estimation network (such as DepthAnything, MiDaS, FastDepth, etc.) to estimate the distance R' of the target UAV, and uses it as the measurement value for the filtering algorithm. Preferably, the filtering algorithm in step S5 includes an adaptive process noise matrix Q. When the IMU detects that the aircraft is undergoing violent maneuvers or the confidence of target detection decreases, the Q value is dynamically increased to increase the confidence weight of the visual measurement value. Preferably, the interception envelope conditions in step S6 include: the relative distance to the target is within the effective range of the UAV net-catching device; the angle error between the current real-time pointing of the gimbal system and the calculated forward angle aiming line is less than a preset threshold, the preset threshold being determined based on the effective range of the UAV net-catching device; and the precise position P output by the filtering algorithm. now and speed V now This allows for the prediction of the target's imminent location and the control of the gimbal to point at that location. The specific steps include: S601 calculates advance time T lead : Among them, T process For algorithm latency processing, T mechanical T is the mechanical response time of the gimbal system. flight The time required for the drone capture net to fully deploy; S602 calculates target drones in T lead The target point that will be moved to after a certain time, i.e., the pre-aiming point position P in the inertial coordinate system. aim n : S603 performs a coordinate transformation, converting the coordinates P in the inertial coordinate system. aim n Convert to camera coordinate system P aim c : S604 calculates the gimbal control angle and sets the aiming point P. aim c (x) aim c ,y aim c ,z aim c Switch back to the azimuth and pitch control angles of the gimbal system to control its rotation: , .

[0046] Therefore, the present invention: 1) By adopting a heterogeneous computing architecture, the system processing latency is significantly reduced. By deploying the computationally intensive deep learning visual detection algorithm and monocular depth estimation algorithm in the visual processing unit, and deploying the high-frequency real-time requirements of attitude calculation, filtering prediction and motor closed-loop control in the main control unit, efficient computing power scheduling is achieved, which effectively solves the contradiction between the limited computing power of general airborne platforms and the high latency of visual algorithms. 2) By using the absolute attitude of the gimbal's independent IMU, the visual deviation and ranging data of the target in the camera coordinate system are converted into three-dimensional spatial coordinates in the inertial system in one step. This eliminates the need for a chain conversion between the gimbal encoder and the aircraft's IMU, effectively decoupling the attitude disturbances caused by the high maneuverability of the carrier aircraft, eliminating multi-level cumulative errors, and providing high anti-interference capability and dynamic response speed. 3) It breaks through the limitations of traditional two-dimensional image tracking, establishes a real motion model of the target in an absolutely static inertial coordinate system, introduces an adaptive observation matrix, uses a monocular depth estimation network to predict the target distance when the rangefinder data is missing, and seamlessly switches after the range value is valid, avoiding gimbal oscillation caused by sudden changes in sensor data, and realizing smooth sliding and stable locking of the target motion trajectory. 4) A dynamic pre-aiming compensation mechanism incorporating end-to-end latency is proposed, significantly improving the interception success rate of airborne net capture missions. Addressing the issue of flight time in UAV capture nets and the need for pre-firing for physical interception, this invention comprehensively considers data computation time, gimbal mechanical response time, and capture net deployment time to calculate the total system hysteresis. Utilizing the target's three-dimensional absolute velocity and position output by a filtering algorithm, spatial extrapolation in the time dimension is performed in the inertial coordinate system to calculate a high-precision pre-aiming point, which is then inversely solved to obtain the control angle of the gimbal motors.

[0047] Those skilled in the art will understand that the above embodiments can be modified in form and detail in practical applications without departing from the spirit and scope of the invention.

Claims

1. An airborne tracking and capture system based on heterogeneous computing, characterized in that, It includes a visual detector, a visual processing unit, a rangefinder, a main control unit, and a gimbal system; The visual detector is connected to the visual processing unit, and the visual processing unit and the rangefinder are both connected to the main control unit. The main control unit is connected to the gimbal system. The visual detector acquires target images, the visual processing unit extracts target features and outputs target orientation information, the rangefinder measures the target distance, the main control unit receives the target orientation information, target distance and inertial measurement data, performs motion compensation and gimbal control, generates the target trajectory and outputs control signals, and the gimbal system performs target tracking and net capture and interception.

2. The airborne tracking and capture system based on heterogeneous computing as described in claim 1, characterized in that, The vision processing unit and the main control unit form a heterogeneous computing architecture. The vision processing unit performs image processing, and the main control unit performs deterministic real-time control tasks.

3. The airborne tracking and capture system based on heterogeneous computing as described in claim 1, characterized in that, An inertial measurement unit is rigidly mounted on the back of the visual detector. Based on the data output by the inertial measurement unit, the main control unit sends motion compensation control signals to the gimbal system that satisfy the following relationship: in, For pitch compensation rate, For azimuth compensation rate, The pitch angular velocity, This represents the azimuth angular velocity.

4. The airborne tracking and capture system based on heterogeneous computing as described in claim 1, characterized in that, The visual detector includes a visible light camera and an infrared camera, with the visible light camera integrating a zoom lens.

5. A motion compensation method for an airborne tracking and capture system based on heterogeneous computing, characterized in that, Includes the following steps: S1. Heterogeneous perception and alignment: The vision processing unit controls the vision detector to acquire images and detect targets, while the main control unit acquires inertial measurement data. The image detection frequency is lower than the inertial measurement data acquisition frequency. S2, Hardware-level motion compensation: When there is no target, the main control unit sends a motion compensation control signal to the gimbal system, causing the gimbal to rotate in the opposite direction relative to the carrier to stabilize the field of view. S3. Target Angle Calculation: When a target is detected, the vision processing unit calculates the target image coordinates and converts them into the angular deviation of the target relative to the camera optical axis. S4, Angle Compensation Alignment: The main control unit calculates the rotation offset within the image processing cycle based on inertial measurement data and compensates for the angle deviation. S5. Trajectory Prediction: The main control unit uses the compensated angle and target distance as measurement values, calculates the three-dimensional coordinates and velocity of the target inertial coordinate system through a filtering algorithm, and outputs the target predicted trajectory. S6. Interception Control: The main control unit calculates the interception lead angle, controls the pan-tilt unit to point to the pre-aiming position, and triggers the net capture device when the interception conditions are met.

6. The motion compensation method for an airborne tracking and capture system based on heterogeneous computing as described in claim 5, characterized in that, Step S3 includes: S31: Calculate the pixel deviation between the target image coordinates and the image center coordinates: S32: Based on pixel deviation, image resolution, and the field of view of the lens corresponding to its focal length, calculate the angular deviation of the target relative to the camera's optical axis using the following formula: in, For horizontal pixel deviation, For vertical pixel deviation, For the target image coordinates, The coordinates of the image center are... For image height resolution, For image width resolution, Focal length The corresponding horizontal field of view, Focal length The corresponding vertical field of view, The pitch deviation angle, This is the azimuth deviation angle.

7. The motion compensation method for an airborne tracking and capture system based on heterogeneous computing as described in claim 5, characterized in that, In step S4, the main control unit compensates and aligns the target deviation angle according to the following formula: in, To compensate for the angle, The original deviation angle, This represents the cumulative rotation offset within the image processing delay period.

8. The motion compensation method for an airborne tracking and capture system based on heterogeneous computing as described in claim 5, characterized in that, In step S5, when the rangefinder cannot obtain the target distance, the vision processing unit runs a monocular depth estimation network to estimate the target distance and uses the estimated distance as the measurement value of the filtering algorithm.

9. The motion compensation method for an airborne tracking and capture system based on heterogeneous computing as described in claim 5, characterized in that, In step S5, the filtering algorithm configures an adaptive process noise matrix and adjusts the parameters of the process noise matrix according to the target maneuvering state and the target detection confidence.

10. The motion compensation method for an airborne tracking and capture system based on heterogeneous computing as described in claim 5, characterized in that, Step S6 includes: S61: Calculate the total system lead time: in, This is the total system lead time. To handle latency in the algorithm, The mechanical response time of the gimbal system. The time required for the drone capture net to fully deploy; S62: Based on the target's current position, velocity, and total lead time, calculate the position of the pre-aiming point in the inertial coordinate system: in, The position of the aiming point in the inertial coordinate system. Let the target's current position be in the inertial coordinate system. Let this be the target's current velocity in the inertial coordinate system; S63: Transform the position of the preview point in the inertial coordinate system to the camera coordinate system; S64: Based on the position of the preview point in the camera coordinate system, calculate the gimbal control angle and drive the gimbal to rotate.

Images