An autonomous target tracking system with panoramic multi-view perception and high-precision servo control

Abstract

The application discloses an autonomous target tracking system with panoramic multi-view perception and high-precision servo control, relates to the technical field of autonomous target tracking, and comprises a hardware unit layer including a panoramic multi-view perception unit, a single-path two-axis servo tracking unit, a variable-power optical unit, a laser emission unit and a heterogeneous embedded main control unit; the heterogeneous embedded main control unit is internally integrated with an NPU processor, an FPGA processor and an ARM processor; an embedded scheduling logic layer comprises a composite urgency arbitration module, a time-sharing polling scheduling module, an EKF threat pre-judgment module, a differentiated channel recycling module and a shielding adaptive dual-source fusion module; the scheduling defects of traditional devices, such as random allocation of tracking objects, high-emergency target leakage control and risk assessment depending on only instantaneous pictures, are solved, and the parallel control capability for multiple intrusion targets is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous target tracking technology, specifically an autonomous target tracking system with panoramic multi-view perception and high-precision servo control. Background Technology

[0002] Currently, in the fields of low-altitude security, substation, and park perimeter protection, integrated monitoring equipment with panoramic wide-angle cameras and single servo tracking PTZ is commonly used. It relies on image recognition to detect intrusion targets and classifies the danger level of targets through simple distance and speed parameters. This is the mainstream security monitoring solution at present.

[0003] However, such devices generally suffer from common industry shortcomings such as fragmented functions and simple scheduling logic: existing devices can only rely on real-time observation data to determine target risk, lack standardized methods for quantifying and ranking multiple targets in urgency. When multiple highest-risk targets appear at the same time, the servo channel randomly assigns tracking objects, and high-urgency targets are prone to not being observed in detail for a long time. The devices do not have a time-sharing scheduling mechanism for servo hardware, cannot periodically poll multiple high-risk targets, and can only make risk judgments based on the current image. They cannot make long-term motion predictions for medium-risk targets, and resource contention is very likely to occur when multiple targets simultaneously appear in a high-risk state.

[0004] Meanwhile, existing dual-source data fusion methods mostly use fixed weighting coefficients, failing to consider observation distortions caused by trees and towers, resulting in severe target trajectory drift during polling intervals; using a unified channel release logic for different scenarios such as target occlusion, temporary loss, and complete evacuation leads to frequent reciprocating rotation of the gimbal, exacerbating mechanical wear; laser deterrence methods lack hierarchical control logic, using the same laser output strategy for drones, birds, and personnel, posing safety and compliance risks; embedded main control lacks a dynamic computing power adjustment mechanism, operating at full load across all modules around the clock, resulting in high power consumption and limited battery life for equipment deployed in the field. Summary of the Invention

[0005] The purpose of this invention is to provide an autonomous target tracking system with panoramic multi-view perception and high-precision servo control to solve the problems raised in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an autonomous target tracking system with panoramic multi-view perception and high-precision servo control, the system comprising a hardware unit layer and an embedded scheduling logic layer; The hardware unit layer includes a panoramic multi-view perception unit, a single-channel two-axis servo tracking unit, a zoom optical unit, a laser emission unit, and a heterogeneous embedded main control unit. The heterogeneous embedded main control unit integrates an NPU processor, an FPGA processor, and an ARM processor. The heterogeneous embedded main control unit can calculate and output the real-time threat level L0 of the target based on the acquired image information. The calculation result is constrained to an integer range of 1-5 by a limiting function, with the highest threat level set to level 5. The embedded scheduling logic layer includes a composite urgency arbitration module, a time-sharing polling scheduling module, an EKF threat prediction module, a differentiated channel recycling module, and an occlusion adaptive dual-source fusion module. The five functional modules are linked in series according to a fixed 40ms running cycle and the preceding and following time sequence, and the five functional modules cooperate with each other to form a closed-loop computing system. The composite urgency arbitration module is used to quantify, score, and sort all L0=5 targets to generate a descending high-risk queue. New high-risk targets that meet the score threshold can be forcibly preempted from the servo tracking channel. The time-sharing polling scheduling module is used to time-multiplex the servo hardware channel when the number of high-risk targets is ≥2, and to track the high-risk targets in the queue in sequence and output pulse laser warnings. The EKF threat prediction module is used to extrapolate the 2-second and 5-second long-term motion parameters of medium-risk targets at level 3-4, calculate the predicted threat level, and mark pre-high-risk targets to be sent to the high-risk queue. The differentiated channel recovery module is used to release servo channels differently based on three states: temporary obstruction, short-term loss, and complete disappearance of the target. When the target moves away from the protected area, the urgency score is smoothly attenuated. The occlusion adaptive dual-source fusion module is used to poll and intermittently fuse servo high-precision observation and panoramic coarse observation data, introduces target occlusion confidence to dynamically correct fusion weights, and continuously updates the motion trajectories of all high-risk targets.

[0007] Furthermore, the process of quantitative scoring in the composite urgency arbitration module includes the following: Obtain the remaining time Tc for the target to reach the core protected area; when Tc < 10s, the overall value of this item is doubled; obtain the area hazard weight constant A, the area includes the core protected area and the outer buffer zone, and the hazard weight constant of the core protected area is greater than that of the outer buffer zone; obtain the target's radial approach velocity vr; using the formula: Sc = α*(1 / Tc) + β*A + γ*vr; Calculate the composite urgency score Sc; the larger the value, the more urgent the target risk, and the high-risk queue is arranged in descending order of this value; 1 / Tc represents the time urgency factor, and the shorter the remaining time, the larger the value of this item; where α, β, and γ represent the corresponding normalized balance coefficients, α+β+γ=1. The system can configure corresponding fixed coefficients for three different security scenarios: airport, substation, and park respectively; the determination condition for channel preemption is that the score of a newly emerging high-risk target is greater than or equal to n times the score of the currently tracked target, where n>1.

[0008] Further, the time-sharing polling scheduling module sequentially tracks high-priority targets in the queue and outputs pulsed laser warning, which includes the following contents: The time-sharing polling scheduling module is internally configured with a fixed duration parameter, which is the main target continuous tracking duration T1. When a secondary high-risk target exists, the patrol duration will be adaptively adjusted according to the remaining intrusion time. The adjustment methods include: when Tc is less than 10s, the patrol duration for the secondary high-risk target is set to T1-m1 seconds; when Tc is in the range of 10s to 30s, the patrol duration for the secondary high-risk target is set to T1-m2 seconds; when Tc is greater than 30s, the patrol duration for the secondary high-risk target is set to T1-m3 seconds; where m1<m2<m3<T1; a smaller Tc value indicates a faster target approaching speed and higher immediate risk, so a longer patrol observation window is allocated; During the polling operation, the system only outputs pulsed laser with a low duty cycle, and will not turn on high-power long-duration continuous laser; if an emergency target with a higher score is identified during the patrol, the current patrol process will be immediately interrupted and the servo tracking object will be switched; the system is equipped with a three-layer laser linkage strategy, which implements differentiated laser control methods for three types of targets: birds, drones, and personnel.

[0009] Further, the EKF threat prediction module fixes a single iteration step, and performs the following operations in sequence, including: When a target is detected for the first time, construct a nine-dimensional target state vector x, a four-dimensional observation vector z, and initialize the basic process noise matrix Qb, the basic observation noise matrix Rb, and the initial error covariance matrix P0; Based on the optimal posterior target state x obtained after observation correction in the previous cycle k-1|k-1 , complete the one-step state prediction operation x with the help of the state transition matrix F k|k-1 =F*x k-1|k-1 , where x k|k-1 represents the priori target state at time k, which is derived only from the previous state deduction without using the observation data at the current moment; the state transition matrix F is a discrete-time transformation matrix constructed based on the uniform acceleration motion model, used to describe the evolution law of the target motion state in adjacent iteration cycles; synchronously update the priori error covariance P k|k-1 =F*P k-1|k-1 *F T +Qk; where Qk represents the adaptively adjusted process noise matrix at time k, used to characterize the state prediction uncertainty caused by target maneuver and environmental disturbance; Collect panoramic fusion observations for this period and calculate the observation residual ek = zk - H*x k|k-1 H is the observation mapping matrix, which maps the nine-dimensional target state space to the four-dimensional observation space, achieving dimensional matching between the predicted state and the actual observation data. Based on the residual, noise adaptive update logic is executed to obtain real-time Rk and real-time Qk. k Let K represent the adaptively adjusted observation noise matrix at time k, and then solve for the Kalman gain matrix K. k =P k|k-1 *H T (H*P k|k-1 *H T +R k ) -1 The optimal posterior state x for this period is obtained by correction. k|k =x k|k-1 +K k *ek, synchronously update the posterior error covariance P k|k =(IK k H)P k|k-1 I represents the identity matrix; Using the optimal posterior state output above as the initial value, the state transition equation is iterated N2s and N5s times respectively. N2s represents the total number of iterations corresponding to a prediction duration of 2s and N5s represents the total number of iterations corresponding to a prediction duration of 5s. The entire extrapolation process does not introduce the current period observation data for correction. Each iteration relies on the uniformly accelerated discrete motion model to recursively deduce all motion state components of the target. After the final iteration, the third dimension value is extracted from the nine-dimensional state vector obtained by the final deduction as the predicted distance rp and the sixth dimension value is extracted as the predicted radial velocity vp. Substituting the extrapolated RP and VP values ​​into the threat prediction formula, the 2-second predicted threat level L is calculated. p2s 5-second threat level prediction L p5s When the module determines that Lp2s≥5 or Lp5s≥5, it marks the target as a pre-high-risk target and sends it to the high-risk target queue.

[0010] Furthermore, the EKF threat prediction module uses the following formula to calculate the predicted threat level: Lp=clip(⌊Ks*(w1*(d / rp)+w2*(|vp| / vf)+w3*Ct)⌋,1,5), where Lp represents the predicted threat level of the target, Ks represents the threat scaling factor, w1 represents the risk weight coefficient corresponding to the target distance dimension, w2 represents the risk weight coefficient corresponding to the target radial velocity dimension, w3 represents the risk weight coefficient corresponding to the target type, w1+w2+w3=1, and d represents the distance reference base. The reference value is defined as follows: vf represents the radial velocity reference value, Ct represents the target type coefficient, which is assigned a fixed value according to the target category and is used to distinguish the inherent danger level of different intrusion targets such as drones, birds, and personnel; clip(·,1,5) represents the clipping function, and the corresponding operation rule is that when the calculation result inside the function is less than 1, the value 1 is output; when the calculation result is between 1 and 5, the original integer result is directly output; when the calculation result is greater than 5, the value 5 is forcibly output. Its function is to constrain the final predicted threat level within the integer range of 1 to 5.

[0011] Furthermore, the differentiated channel recycling module includes three sets of independent decision-making logic; as detailed below: The first logic is that if a target cannot be identified for 1 to 3 consecutive frames, it is considered a temporary obstruction, and the system will not release the servo tracking channel. The second logic is that if a target cannot be identified for 1 to 3 consecutive seconds, it is considered a short-term loss, and the system will downgrade the current primary target to a polling target and release the servo channel. The third logic is that if a target cannot be identified for more than 3 consecutive seconds and the real-time threat level is below level 3, it is considered a complete disappearance, and the system will remove the target from the high-risk queue and completely release the channel. When the target continues to move outward from the protected area, the module will linearly reduce the target's composite urgency Sc in each running cycle.

[0012] Furthermore, the occlusion-adaptive dual-source fusion module builds an adaptive fusion model, specifically as follows: Xf = ws * Xs + (1 - So) * wp * Xp; Where Xf represents the fused target motion state vector; So represents the target occlusion confidence, calculated from the proportion of effective unoccluded pixels of the target in the panoramic image to the total number of pixels of the target; ws represents the basic weighting coefficient corresponding to the servo observation data, representing the confidence ratio of the servo high-precision observation results in the fusion calculation process; when the time since the last servo inspection is less than 2 seconds, this coefficient is fixed at 0.7; when the time since the last servo inspection is more than 2 seconds, this coefficient is fixed at 0.3; Xs represents the servo observation state vector, which is collected during the inspection process of the servo tracking unit and processed by the servo observation data. The target's nine-dimensional motion state data after EKF filtering correction; wp represents the basic weighting coefficient corresponding to the panoramic observation data, representing the basic credibility ratio of the panoramic wide-angle observation results in the fusion calculation process; when the time since the last servo inspection is less than 2 seconds, this coefficient is fixed at 0.3; when the time since the last servo inspection is more than 2 seconds, this coefficient is fixed at 0.7; both sets of scenarios satisfy the normalization constraint condition ws+wp=1; Xp represents the panoramic observation state vector, which is the target's nine-dimensional motion state data obtained by the panoramic multi-view perception unit from the wide-angle image detection; The target motion vector after fusion is directly used as the initial parameter for servo gimbal aiming.

[0013] Furthermore, the embedded scheduling logic layer is equipped with three levels of computing power energy-saving control logic. When the system is expanded with multiple servo hardware, a load balancing mechanism is enabled. When the highest target threat level in the system is greater than or equal to level 4, all five functional modules are fully operational. When the highest target threat level is between level 2 and level 3, only 2 seconds of short-term target inference calculation is retained, and 5 seconds of long-term inference is disabled. When all target threat levels are less than or equal to level 1, polling and fusion calculations are suspended to reduce the overall power consumption of the hardware. When the system is equipped with multiple servo channels and all channels are occupied, the time-sharing polling scheduling module is reused to handle the remaining high-risk targets.

[0014] Furthermore, after each round of computation is completed, the heterogeneous embedded main control unit will synchronously transmit the high-risk target queue, the target polling order, and the laser control commands to the human-machine interaction unit and the superior command unit; at the same time, the device will store the scheduling log of each round of computation locally, and the log can be used for subsequent security risk backtracking analysis.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This invention constructs an integrated time-series closed-loop scheduling system. It achieves standardized priority ranking of multiple high-risk targets through a three-dimensional composite urgency scoring mechanism, and realizes periodic fine-grained inspection of multiple targets on a single servo channel by combining adaptive time-division polling logic. At the same time, it is equipped with an EKF prediction module with adaptive noise adjustment to predict the long-term motion status of targets and include them in the pre-high-risk queue in advance. This fundamentally alleviates the channel preemption conflict caused by multiple concurrent targets, solves the scheduling defects of traditional equipment such as random allocation of tracking objects, missed control of high-urgent targets, and reliance on instantaneous images to assess risks, and significantly improves the parallel control capability of multiple intrusion targets. This invention optimizes target trajectory estimation and hardware resource recovery logic for complex outdoor imaging environments. It introduces dynamic attenuation of panoramic observation weights based on occlusion confidence, effectively suppressing target trajectory drift under occlusion conditions and shortening the time required for gimbal reacquisition. At the same time, it sets differentiated channel recovery rules for three types of scenarios: temporary occlusion, short-term loss, and complete disappearance, and smoothly attenuates the risk score for targets far from the protected area, avoiding meaningless frequent start-stop of the gimbal, reducing mechanical wear of the servo transmission structure, and improving tracking stability and equipment lifespan in complex scenarios.

[0016] This invention balances scenario compliance with the need for low-power operation of equipment, and designs a three-layered laser linkage strategy. Differentiated laser power, illumination methods, and voice linkage schemes are matched for drones, birds, and personnel respectively, eliminating the personal safety and light pollution compliance risks caused by uniform laser output. In addition, a three-level computing power graded energy-saving control logic is set up to dynamically adjust the computing load according to the real-time threat level in the field, reducing the long-term operating power consumption of unattended equipment in the field. The five major modules work together to achieve autonomous coordination of the entire chain of perception, assessment, tracking, handling, and energy consumption, significantly improving the integrity and practicality of overall security management. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of an autonomous target tracking system with panoramic multi-view perception and high-precision servo control according to the present invention. Detailed Implementation

[0018] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example: Figure 1 As shown, the present invention provides an autonomous target tracking system with panoramic multi-view perception and high-precision servo control. The system includes a hardware unit layer and an embedded scheduling logic layer. The hardware unit layer includes a panoramic multi-view perception unit, a single-channel two-axis servo tracking unit, a zoom optical unit, a laser emission unit, and a heterogeneous embedded main control unit. The heterogeneous embedded main control unit integrates an NPU processor, an FPGA processor, and an ARM processor. The heterogeneous embedded main control unit can calculate and output the real-time threat level L0 of the target based on the acquired image information. The calculation result is constrained to an integer range of 1-5 by a limiting function, with the highest threat level set to level 5. The embedded scheduling logic layer includes a composite urgency arbitration module, a time-sharing polling scheduling module, an EKF threat prediction module, a differentiated channel recycling module, and an occlusion adaptive dual-source fusion module. These five functional modules are linked in series according to a fixed 40ms running cycle and sequential timing, and they cooperate with each other to form a closed-loop computing system. The calculation of the target's real-time threat level is achieved through panoramic image target detection and ranging calculation, which obtains three basic parameters of the target at the current moment: target type coefficient, target current measured distance, and target current measured radial velocity. These are the raw measured data directly observed by the panoramic camera in the current frame, without EKF filtering or future time extrapolation processing. The composite urgency arbitration module is used to quantify, score, and sort all L0=5 targets to generate a descending high-risk queue. New high-risk targets that meet the score threshold can be forcibly preempted from the servo tracking channel. The time-sharing polling scheduling module is used to time-multiplex the servo hardware channel when the number of high-risk targets is ≥2, and to track the high-risk targets in the queue in sequence and output pulse laser warnings. The EKF threat prediction module is used to extrapolate the 2-second and 5-second long-term motion parameters of medium-risk targets at level 3-4, calculate the predicted threat level, and mark pre-high-risk targets to be sent to the high-risk queue. The differentiated channel recovery module is used to release servo channels differently based on three states: temporary obstruction, short-term loss, and complete disappearance of the target. When the target moves away from the protected area, the urgency score is smoothly attenuated. The occlusion adaptive dual-source fusion module is used to poll and intermittently fuse servo high-precision observation and panoramic coarse observation data, introduces target occlusion confidence to dynamically correct fusion weights, and continuously updates the motion trajectories of all high-risk targets.

[0020] The process of quantitative scoring in the composite urgency arbitration module includes the following: Obtain the remaining time Tc for the target to reach the core protected area; when Tc < 10s, the overall value of this item is doubled; obtain the area hazard weight constant A, the area includes the core protected area and the outer buffer zone, and the hazard weight constant of the core protected area is greater than that of the outer buffer zone; obtain the target's radial approach velocity vr; using the formula: Sc = α*(1 / Tc) + β*A + γ*vr; Calculate the composite urgency score Sc; a larger value indicates a more urgent target risk, and the high-risk queue is sorted in descending order by this value; 1 / Tc represents the time urgency factor, and the shorter the remaining time, the larger this value; where α, β, and γ represent the corresponding normalized balance coefficients, and α+β+γ=1; Fixed configuration according to actual application scenarios: For birds in the airport scenario, α=0.5, β=0.2, γ=0.3; For drones in the substation scenario, α=0.4, β=0.4, γ=0.2; For ground security in the park scenario, α=0.3, β=0.5, γ=0.2; The system can configure corresponding fixed coefficients respectively according to three different security scenarios: airport, substation and park; the determination condition for channel preemption is that the score of a newly emerging high-risk target is greater than or equal to n times the score of the currently tracked target, where n>1, and n is generally set to 1.3.

[0021] The time-sharing polling scheduling module sequentially tracks high-risk targets in the queue and outputs pulsed laser warning, including the following contents: The time-sharing polling scheduling module is internally configured with a fixed duration parameter, which is the main target continuous tracking duration T1. When there is a secondary high-risk target, the patrol duration will be adaptively adjusted according to the remaining intrusion time. The adjustment method includes that when Tc is less than 10s, the patrol duration for the secondary high-risk target is T1-m1 seconds, when Tc is in the range of 10s to 30s, the patrol duration for the secondary high-risk target is T1-m2 seconds, and when Tc is greater than 30s, the patrol duration for the secondary high-risk target is T1-m3 seconds; where m1<m2<m3<T1; a smaller Tc value indicates a faster target approaching speed and higher immediate risk, so a longer patrol observation window is allocated; During the polling operation, the system only outputs pulsed laser with a low duty cycle, and will not turn on high-power long-lasting continuous laser; if an emergency target with a higher score is identified during the patrol process, the current patrol process will be immediately interrupted and the servo tracking object will be switched; the system is equipped with a three-layer laser linkage strategy, which implements differentiated laser control methods for three types of targets: birds, drones and personnel.

[0022] As shown in the embodiment: the first layer is the polling warning layer, the second layer is the stable alert layer, and the third layer is the continuous jamming layer; For bird targets: only the first and second layer laser strategies are executed, the laser spot is projected into the airspace in front of the bird, direct irradiation on the bird body is prohibited, and the maximum laser power is limited to 0.5W; For drone targets: all three layers of laser strategies are enabled. After stable tracking for 2 seconds, it can enter the third layer of continuous jamming layer, the laser spot is aimed at the airborne photoelectric load of the drone, and the maximum laser power is limited to 2W; Personnel objective: Only execute the first-level pulse warning, lock and shut down all second and third-level lasers, and simultaneously link with the external high-volume voice alarm device to play a drive-away voice message.

[0023] The EKF threat prediction module has a fixed single iteration step size and performs the following operations in sequence, including: Upon initial target detection, a nine-dimensional target state vector x, a four-dimensional observation vector z, an initial basic process noise matrix Qb, a basic observation noise matrix Rb, and an initial error covariance matrix P0 are constructed. The nine dimensions represent, in order: the first dimension is the target azimuth, the second is the target pitch, the third is the target range, the fourth is the target azimuth velocity, the fifth is the target pitch velocity, the sixth is the target radial velocity, the seventh is the target azimuth acceleration, the eighth is the target pitch acceleration, and the ninth is the target radial acceleration. The four dimensions represent, in order: the first is the observed azimuth, the second is the observed pitch, the third is the observed target range, and the fourth is the observed radial velocity. Based on the optimal posterior target state x obtained after observation correction in the previous period. k-1|k-1 The state prediction operation x is completed in one step using the state transition matrix F. k|k-1 =F*x k-1|k-1 , where x k|k-1 The state at time k represents the prior target state obtained solely from the derivation of previous states without using the observation data at the current time; the state transition matrix F is a discrete-time transformation matrix constructed based on the uniform acceleration motion model, used to describe the evolution of the target motion state within adjacent iteration cycles; the prior error covariance P is updated synchronously. k|k-1 =F*P k-1|k-1 *F T +Qk; where Qk represents the adaptively adjusted process noise matrix at time k, used to characterize the uncertainty in state prediction caused by target maneuvering and environmental disturbances; Collect panoramic fusion observations for this period and calculate the observation residual ek = zk - H*x k|k-1 H is the observation mapping matrix, which maps the nine-dimensional target state space to the four-dimensional observation space, achieving dimensional matching between the predicted state and the actual observation data. Based on the residual, noise adaptive update logic is executed to obtain real-time Rk and real-time Qk. k Let K represent the adaptively adjusted observation noise matrix at time k, and then solve for the Kalman gain matrix K. k =P k|k-1 *H T (H*P k|k-1 *H T +R k ) -1 The optimal posterior state x for this period is obtained by correction. k|k =x k|k-1 +Kk *ek, synchronously update the posterior error covariance P k|k =(IK k H)P k|k-1 I represents the identity matrix; Using the optimal posterior state output above as the initial value, the state transition equation is iterated N2s and N5s times respectively. N2s represents the total number of iterations corresponding to a prediction duration of 2s and N5s represents the total number of iterations corresponding to a prediction duration of 5s. The entire extrapolation process does not introduce the current period observation data for correction. Each iteration relies on the uniformly accelerated discrete motion model to recursively deduce all motion state components of the target. After the final iteration, the third dimension value is extracted from the nine-dimensional state vector obtained by the final deduction as the predicted distance rp and the sixth dimension value is extracted as the predicted radial velocity vp. Substituting the extrapolated RP and VP values ​​into the threat prediction formula, the 2-second predicted threat level L is calculated. p2s 5-second threat level prediction L p5s When the module determines that Lp2s≥5 or Lp5s≥5, it marks the target as a pre-high-risk target and sends it to the high-risk target queue.

[0024] The complete adjustment process of the noise adaptive update logic is as follows: First, calculate the mean absolute value of the observation residuals in the current period, mean(|ek|), and set a fixed residual threshold ξ; if mean(|ek|≤ξ), it means that the current imaging environment is clear, and the noise matrix maintains the basic values ​​Rk=Rb and Qk=Qb; if mean(|ek|>ξ), it means that backlighting, rain, snow, or occlusion have increased the observation noise, and the real-time observation noise matrix is ​​updated according to the formula Rk=Rb*(1+μ*mean(|ek|)), with an adjustment coefficient μ=0.2; when the mean residual value is greater than the threshold ξ for three consecutive 40ms iterations, the process noise Qk=Qb*1.2 is amplified synchronously; when the mean residual value falls back to within the threshold, the noise matrix is ​​completely restored to the basic matrix; the adaptively adjusted Rk and Qk are substituted into the time update and observation correction calculations of the current period to reduce the target motion prediction error in complex environments.

[0025] The EKF threat prediction module uses the following formula to calculate the predicted threat level: Lp=clip(⌊Ks*(w1*(d / rp)+w2*(|vp| / vf)+w3*Ct)⌋,1,5), where Lp represents the predicted threat level, Ks represents the threat scaling factor (fixed at 6), w1 represents the risk weight coefficient corresponding to the target distance dimension, w2 represents the risk weight coefficient corresponding to the target radial velocity dimension, w3 represents the risk weight coefficient corresponding to the target type, w1+w2+w3=1, and d represents the distance reference value (fixed at 1). 00m, vf represents the radial velocity reference value, fixed at 10 meters per second, Ct represents the target type coefficient, which is assigned a fixed value according to the target category and is used to distinguish the inherent danger level of different intrusion targets such as drones, birds, and personnel; clip(·,1,5) represents the clipping function, and the corresponding operation rule is that when the calculation result inside the function is less than 1, the value 1 is output; when the calculation result is between 1 and 5, the original integer result is directly output; when the calculation result is greater than 5, the value 5 is forcibly output. Its function is to constrain the final predicted threat level within the integer range of 1 to 5.

[0026] The differentiated channel recycling module contains three sets of independent decision-making logic; as follows: The first logic is that if a target cannot be identified for 1 to 3 consecutive frames, it is considered a temporary obstruction, and the system will not release the servo tracking channel. The second logic is that if a target cannot be identified for 1 to 3 consecutive seconds, it is considered a short-term loss, and the system will downgrade the current primary target to a polling target and release the servo channel. The third logic is that if a target cannot be identified for more than 3 consecutive seconds and the real-time threat level is below level 3, it is considered a complete disappearance, and the system will remove the target from the high-risk queue and completely release the channel. When the target continues to move outward from the protected area, the module will linearly reduce the target's composite urgency Sc in each running cycle.

[0027] The occlusion-adaptive dual-source fusion module builds an adaptive fusion model, specifically as follows: Xf = ws * Xs + (1 - So) * wp * Xp; Where Xf represents the fused target motion state vector, which is the optimal target state estimation result obtained after weighted fusion of two types of observation data. This vector is ultimately used as the initial parameter preset for servo gimbal aiming; So represents the target occlusion confidence, calculated from the proportion of effective unoccluded pixels of the target in the panoramic image to the total number of pixels of the target; ws represents the basic weighting coefficient corresponding to the servo observation data, representing the confidence ratio of the servo high-precision observation results in the fusion calculation process; when the time since the last servo inspection is less than 2 seconds, this coefficient is fixed at 0.7; when the time since the last servo inspection is more than 2 seconds, this coefficient is fixed at 0.3; Xs represents The servo observation state vector is the target's nine-dimensional motion state data collected during the servo tracking unit's inspection process and corrected by EKF filtering; wp represents the basic weighting coefficient corresponding to the panoramic observation data, representing the basic credibility ratio of the panoramic wide-angle observation results in the fusion calculation process; when the time since the last servo inspection is less than 2 seconds, this coefficient is fixed at 0.3; when the time since the last servo inspection is more than 2 seconds, this coefficient is fixed at 0.7; both sets of scenarios satisfy the normalization constraint condition ws+wp=1; Xp represents the panoramic observation state vector, which is the target's nine-dimensional motion state data obtained by the panoramic multi-view perception unit from the wide-angle image detection; The target motion vector after fusion is directly used as the initial parameter for servo gimbal aiming.

[0028] An attenuation coefficient 1-So is introduced before the panoramic base weight wp. When the target occlusion degree is higher and the So value is larger, the actual weighting ratio corresponding to the panoramic observation data will be attenuated and reduced synchronously, weakening the interference of occluded and distorted observation data on the target state fusion result. When the target is completely unoccluded, So=0, and the panoramic observation weight remains unchanged at the original base value. The Xf obtained after fusion is used as the initial state parameter for the next servo gimbal preset aiming, which can significantly shorten the time for the servo to reacquire the target and suppress the estimation drift error of the target motion trajectory during the polling interval.

[0029] The embedded scheduling logic layer is equipped with three levels of computing power energy-saving control logic. When the system is expanded with multiple servo hardware, a load balancing mechanism is enabled. When the highest target threat level in the system is greater than or equal to level 4, all five functional modules are fully operational. When the highest target threat level is between level 2 and level 3, only 2 seconds of short-term target inference calculation is retained, and 5 seconds of long-term inference is disabled. When all target threat levels are less than or equal to level 1, polling and fusion calculations are suspended to reduce the overall power consumption of the hardware. When the system is equipped with multiple servo channels and all channels are occupied, the time-sharing polling scheduling module is reused to handle the remaining high-risk targets.

[0030] After each cycle of computation is completed, the heterogeneous embedded main control unit will synchronously transmit the high-risk target queue, the target polling order, and the laser control commands to the human-machine interaction unit and the superior command unit. At the same time, the device will store the scheduling log of each cycle locally, which can be used for subsequent safety risk retrospective analysis.

[0031] As shown in the example: Panoramic multi-view perception unit: 12 wide-angle cameras arranged in a ring, 30° single camera field of view, global synchronization 40ms acquisition cycle, built-in image ranging and target classification algorithms; Single-channel two-axis servo tracking unit: 360° horizontal and ±90° pitch servo gimbal, equipped with a 30x zoom optical unit and a 2W adjustable laser emission unit; Heterogeneous embedded main control unit: ARM main controller + NPU image processor + FPGA parallel computing unit, which completes a round of full-process scheduling calculation every 40ms; Peripheral equipment includes: human-machine interface display screen, superior command and communication module, local log storage hard drive, and high-volume voice alarm device.

[0032] Scene parameter configuration: Substation scene weight coefficients α=0.4, β=0.4, γ=0.2, preemption threshold n=1.3; The system identified three targets in this round of image detection, which were labeled as target A, target B, and target C. The main controller first uniformly calculated the real-time threat level L0. Target A is a small intrusion drone: measured distance = 60m, radial velocity = -8m / s (rapid approach), target coefficient Ct = 1.5; calculated weighted terms: 100 / 60 + 8 / 10 + 1.5 ≈ 1.667 + 0.8 + 1.5 = 3.967, scaled and rounded to 11, after limiting L0 = 5; determined to be the highest risk at present; Target B is a low-flying bird with a measured distance of 120m, radial velocity of -3m / s, and Ct of 0.8. The weighted average is calculated as: 100 / 120 + 3 / 10 + 0.8 ≈ 0.833 + 0.3 + 0.8 = 1.933. The scaling factor is rounded to 11, and the amplitude limit is L0 = 5. The target is classified as a second-highest risk. Target C is a person loitering at a distance, measured distance = 220m, radial velocity = -1.2m / s, Ct = 0.3; calculated weighted terms: 100 / 220 + 1.2 / 10 + 0.3 ≈ 0.454 + 0.12 + 0.3 = 0.874, scaled and rounded to 5, after limiting L0 = 3; medium risk, only triggers EKF prediction; Output results: Real-time threat L0=5 for targets A and B, proceed to the composite urgency arbitration module; CL0=3 for target A, proceed to the EKF threat prediction module. Obtain the remaining time for targets A and B to invade the core area. The weight of region A is: 1.0 for the core area and 0.4 for the buffer zone. Target A: Sc = 0.4 × (1 / 7) × 2 + 0.4 × 1.0 + 0.2 × 8 ≈ 0.114 + 0.4 + 1.6 = 2.114; Target B: Sc = 0.4 × (1 / 18) + 0.4 × 0.4 + 0.2 × 3 ≈ 0.022 + 0.16 + 0.6 = 0.782; High-risk queue, sorted by score in descending order: [Target A, Target B] The current servo channel is empty, so the target with the highest score, A, is directly assigned as the main tracking target, and the tracking duration is T1=3s.

[0033] The EKF threat prediction module provides a complete simulation of medium-risk target C. The iterative base step size is 40ms, N2s=50, N5s=125 steps, residual threshold ξ, and noise adjustment coefficient μ=0.2; extract the predicted distance rp=92m and the predicted radial velocity vp=-7.5m / s in 2s; calculate the predicted threat Lp=5, determine Lp2s≥5, mark target C as a pre-high-risk target, insert it at the tail of the high-risk queue, and update the queue: [A,B,C].

[0034] After all calculations are completed in this 40ms cycle, the main control unit synchronously pushes data to the human-machine interface and the superior command unit. High-risk queue ranking: A (2.114), B (0.782), pre-high-risk C; Polling allocation record: A continuously tracks for 3 seconds → B inspects for 1.5 seconds; Laser control command: Drone A activates 2W continuous interference laser, Drone B issues pulse warning; The local hard drive persistently stores complete scheduling logs, including threat values ​​for each target, EKF simulation parameters, and PTZ switching sequence, for post-event security risk tracing.

[0035] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An autonomous target tracking system with panoramic multi-view perception and high-precision servo control, characterized in that: The system includes a hardware unit layer and an embedded scheduling logic layer; The hardware unit layer includes a panoramic multi-view perception unit, a single-channel two-axis servo tracking unit, a variable magnification optical unit, a laser emission unit, and a heterogeneous embedded main control unit. Heterogeneous The embedded main control unit integrates an NPU processor, an FPGA processor, and an ARM processor. The heterogeneous embedded main control unit can calculate and output the real-time threat level L0 of the target based on the acquired image information; the calculation result is constrained to an integer range of 1-5 by a limiting function, with the highest threat level set to level 5. The embedded scheduling logic layer includes a composite urgency arbitration module, a time-sharing polling scheduling module, an EKF threat prediction module, a differentiated channel recycling module, and an occlusion adaptive dual-source fusion module. The composite urgency arbitration module is used to quantify, score and sort all L0=5 targets to generate a descending high-risk queue. New high-risk targets that meet the score threshold can be forcibly preempted from the servo tracking channel. The time-sharing polling scheduling module is used to time-multiplex the servo hardware channel when the number of high-risk targets is ≥2, and to track the high-risk targets in the queue in sequence and output pulse laser warnings. The EKF threat prediction module is used to extrapolate the 2s and 5s long-term motion parameters of medium-risk targets of level 3-4, calculate the predicted threat level, and mark pre-high-risk targets to be sent to the high-risk queue. The differentiated channel recovery module is used to release servo channels differently based on three states: temporary obstruction, short-term loss, and complete disappearance of the target. When the target moves away from the protection zone, the urgency score is smoothly attenuated. The occlusion adaptive dual-source fusion module is used to poll and intermittently fuse servo high-precision observation and panoramic coarse observation data, introduce target occlusion confidence to dynamically correct fusion weights, and continuously update the motion trajectories of all high-risk targets.

2. The autonomous target tracking system with panoramic multi-view perception and high-precision servo control according to claim 1, characterized in that: The process of quantitative scoring performed by the composite urgency arbitration module includes the following: Obtain the remaining time Tc for the target to reach the core protected area; when Tc < 10s, the overall value of this item is doubled; obtain the area hazard weight constant A, where the area includes the core protected area and the outer buffer zone, and the hazard weight constant of the core protected area is greater than that of the outer buffer zone; obtain the target radial approach velocity vr; using the formula: Sc = α*(1 / Tc) + β*A + γ*vr; Calculate the composite urgency score Sc; 1 / Tc represents the time urgency factor; where α, β, and γ represent the corresponding normalized equilibrium coefficients, and α+β+γ=1; The system can configure corresponding fixed coefficients for three different security scenarios: airports, substations, and industrial parks. The criteria for determining channel preemption are that the score of a newly emerging high-risk target is greater than or equal to n times the score of the currently tracked target, where n>1.

3. The autonomous target tracking system with panoramic multi-view perception and high-precision servo control according to claim 1, characterized in that: The time-sharing polling scheduling module sequentially tracks high-ranking targets within the queue and outputs pulsed laser warnings, including the following: The time-sharing polling scheduling module is internally configured with a fixed duration parameter. The fixed duration parameter is the continuous tracking duration T1 for the primary target. When a secondary high-risk target exists, the inspection duration will be adaptively adjusted according to the remaining intrusion time. The adjustment method includes: when Tc is less than 10s, the inspection duration for the secondary high-risk target is T1-m1 seconds; when Tc is between 10s and 30s, the inspection duration for the secondary high-risk target is T1-m2 seconds; and when Tc is greater than 30s, the inspection duration for the secondary high-risk target is T1-m3 seconds; where m1 <m2<m3<T1; During polling operation, the system only outputs low duty cycle pulsed laser and will not activate high-power, long-duration continuous laser. If an emergency target with a higher score is identified during the inspection process, the current inspection process will be immediately interrupted and the servo tracking target will be switched. The system is equipped with a three-layer laser linkage strategy, which implements differentiated laser control methods for three types of targets: birds, drones, and personnel.

4. The autonomous target tracking system with panoramic multi-view perception and high-precision servo control according to claim 1, characterized in that: The EKF threat prediction module has a fixed single iteration step size and performs the following operations in sequence, including: When the target is detected for the first time, construct a nine-dimensional target state vector x, a four-dimensional observation vector z, initialize the basic process noise matrix Qb, the basic observation noise matrix Rb, and the initial error covariance matrix P0; Based on the optimal posterior target state x obtained after observation correction in the previous period. k-1|k-1 The state prediction operation x is completed in one step using the state transition matrix F. k|k-1 =F*x k-1|k-1 , where x k|k-1 The state at time k represents the prior target state obtained solely from the derivation of previous states without using the observation data at the current time; the state transition matrix F is a discrete-time transformation matrix constructed based on the uniform acceleration motion model, used to describe the evolution of the target motion state within adjacent iteration cycles; the prior error covariance P is updated synchronously. k|k-1 =F*P k-1|k-1 *F T +Qk; where Qk represents the adaptively adjusted process noise matrix at time k; Collect panoramic fusion observations for this period and calculate the observation residual ek = zk - H*x k|k-1 Where H is the observation mapping matrix, and real-time Rk and real-time Qk are obtained by performing noise adaptive update logic based on the residual. k Let K represent the adaptively adjusted observation noise matrix at time k, and then solve for the Kalman gain matrix K. k =P k|k-1 *H T (H*P k|k-1 *H T +R k ) -1 The optimal posterior state x for this period is obtained by correction. k|k =x k|k-1 +K k *ek, synchronously update the posterior error covariance P k|k =(IK k H)P k|k-1 I represents the identity matrix; Using the optimal posterior state output above as the initial value, the state transition equation is iterated N2s and N5s times respectively. N2s represents the total number of iterations corresponding to a prediction duration of 2s and N5s represents the total number of iterations corresponding to a prediction duration of 5s. The entire extrapolation process does not introduce the current period observation data for correction. Each iteration relies on the uniformly accelerated discrete motion model to recursively deduce all motion state components of the target. After the final iteration, the third dimension value is extracted from the nine-dimensional state vector obtained by the final deduction as the predicted distance rp and the sixth dimension value is extracted as the predicted radial velocity vp. Substituting the extrapolated RP and VP values ​​into the threat prediction formula, the 2-second predicted threat level L is calculated. p2s 5-second threat level prediction L p5s When the module determines that Lp2s≥5 or Lp5s≥5, it marks the target as a pre-high-risk target and sends it to the high-risk target queue.

5. The autonomous target tracking system with panoramic multi-view perception and high-precision servo control according to claim 4, characterized in that: The EKF threat prediction module uses the following formula to calculate the predicted threat level: Lp=clip(⌊Ks*(w1*(d / rp)+w2*(|vp| / vf)+w3*Ct)⌋,1,5), where Lp represents the predicted threat level, Ks represents the threat scaling factor, w1 represents the risk weight factor corresponding to the target distance dimension, w2 represents the risk weight factor corresponding to the target radial velocity dimension, w3 represents the risk weight factor corresponding to the target type, w1+w2+w3=1, d represents the distance reference value, vf represents the radial velocity reference value, and Ct represents the target type coefficient, which is assigned a fixed value according to the target category to distinguish the inherent danger of different intrusion targets such as drones, birds, and personnel; clip(·,1,5) represents the clipping function, and the corresponding operation rule is that when the calculation result inside the function is less than 1, the value 1 is output; when the calculation result is between 1 and 5, the original integer result is directly output; and when the calculation result is greater than 5, the value 5 is forcibly output. Its function is to constrain the final predicted threat level within the integer range of 1 to 5.

6. The autonomous target tracking system with panoramic multi-view perception and high-precision servo control according to claim 1, characterized in that: The differentiated channel recycling module contains three sets of independent decision-making logic; as follows: The first logic is that if a target cannot be identified for 1 to 3 consecutive frames, it is considered a temporary obstruction, and the system will not release the servo tracking channel. The second logic is that if a target cannot be identified for 1 to 3 consecutive seconds, it is considered a short-term loss, and the system will downgrade the current primary target to a polling target and release the servo channel. The third logic is that if a target cannot be identified for more than 3 consecutive seconds and the real-time threat level is below level 3, it is considered a complete disappearance, and the system will remove the target from the high-risk queue and completely release the channel. When the target continues to move outward from the protected area, the module will linearly reduce the target's composite urgency Sc in each running cycle.

7. The autonomous target tracking system with panoramic multi-view perception and high-precision servo control according to claim 1, characterized in that: The occlusion-adaptive dual-source fusion module builds an adaptive fusion model, specifically as follows: Xf = ws * Xs + (1 - So) * wp * Xp; Where Xf represents the fused target motion state vector, So represents the target occlusion confidence, calculated from the proportion of effective unoccluded pixels of the target in the panoramic image to the total number of pixels of the target; ws represents the basic weighting coefficient corresponding to the servo observation data, representing the confidence ratio of the servo high-precision observation results in the fusion calculation process; when the time since the last servo inspection is less than 2 seconds, this coefficient is fixed at 0.7; when the time since the last servo inspection is more than 2 seconds, this coefficient is fixed at 0.3; Xs represents the servo observation state vector, which is the nine-dimensional motion state data of the target collected during the servo tracking unit inspection and corrected by EKF filtering; wp represents the basic weighting coefficient corresponding to the panoramic observation data, representing the basic confidence ratio of the panoramic wide-angle observation results in the fusion calculation process; when the time since the last servo inspection is less than 2 seconds, this coefficient is fixed at 0.3; when the time since the last servo inspection is more than 2 seconds, this coefficient is fixed at 0.7; both sets of scenarios satisfy the normalization constraint condition ws+wp=1=. Xp represents the panoramic observation state vector, which is the nine-dimensional motion state data of the target obtained by the panoramic multi-view perception unit from the wide-angle image detection. The target motion vector after fusion is directly used as the initial parameter for servo gimbal aiming.

8. The autonomous target tracking system with panoramic multi-view perception and high-precision servo control according to claim 1, characterized in that: The embedded scheduling logic layer is internally configured with three levels of computing power energy-saving control logic. When the system expands to multiple servo hardware, a load balancing mechanism is activated. When the highest target threat level in the system is greater than or equal to level 4, all five functional modules are fully operational. When the highest target threat level is between level 2 and level 3, only 2 seconds of short-term target deduction calculation is retained, and 5 seconds of long-term deduction is disabled. When all target threat levels are less than or equal to level 1, polling and fusion calculations are suspended to reduce overall hardware power consumption. When the system is equipped with multiple servo channels and all channels are occupied, the time-sharing polling scheduling module is reused to handle the remaining high-risk targets.

9. The autonomous target tracking system with panoramic multi-view perception and high-precision servo control according to claim 1, characterized in that: After each cycle of computation is completed, the heterogeneous embedded main control unit will synchronously transmit the high-risk target queue, the target polling order, and the laser control commands to the human-machine interaction unit and the superior command unit. At the same time, the device will store the scheduling log of each cycle locally, which can be used for subsequent safety risk retrospective analysis.

Images