Multi-source data fusion sea target real-time positioning and tracking system and method

Abstract

The invention provides a marine target real-time positioning and tracking system and method based on multi-source data fusion, and belongs to the field of target positioning and tracking. The sensor portion collects raw data. The data acquisition and time synchronization module is responsible for receiving and aligning data of each sensor; the multi-stage coordinate system registration and transformation module unifies all observation data to a world coordinate system by using IMU data and preset calibration parameters; the AI visual target detection module processes the video stream to identify a target; the adaptive federated Kalman filtering fusion module receives each path of processed data and outputs optimal target state estimation; and the dynamic task and resource scheduling module performs optimal allocation on system resources according to the current tracking state to form closed-loop control. Through deep fusion of high-precision RTK positioning data, laser ranging data, photoelectric pod attitude information and an AI vision algorithm, the positioning precision, tracking robustness and system real-time performance of a sea target under a complex dynamic sea condition are improved.

Description

TECHNICAL FIELD

[0001] The application provides a multi-source data fusion real-time positioning and tracking system and method for marine targets, and belongs to the field of target positioning and tracking TECHNICAL FIELD BACKGROUND

[0002] The accurate real-time positioning and tracking of marine targets has a key significance in the fields of maritime supervision, national defense security, emergency rescue and marine resource development, etc. However, the existing technologies mostly rely on single or limited combination of sensors, and their performance has significant limitations.

[0003] The traditional tracking methods mainly include:

[0004] Positioning method based on global navigation satellite system (GNSS): such methods, like using the automatic identification system (AIS) or independent GNSS receiver, can provide the absolute geographic position of the target. However, the AIS information has a low update frequency, is easily interfered by deceptive interference, and may be actively closed in specific scenarios. When using GNSS alone, it is easily affected by signal shielding and multipath effect near the shore, island or large facility, resulting in decreased positioning accuracy and insufficient reliability.

[0005] Radar-based tracking method: as an active detection device, radar is less affected by light and bad weather, and is the traditional core device for marine target detection. However, the resolution of radar is limited, and it is difficult to accurately identify small targets or dense target groups; at the same time, sea wave clutter will seriously interfere with the radar echo, especially in high sea states, which easily leads to increased false alarm rate and tracking loss.

[0006] Optical / visual-based tracking method: with the development of artificial intelligence (AI) technology, AI visual algorithms have made significant progress in target detection and identification. Using visible light or infrared cameras for target tracking has the advantages of high resolution and rich target features. However, the visual system is a passive detection system, and its performance is easily affected by changes in light, fog, rain, snow and other weather conditions; more importantly, in dynamic sea conditions, sea wave fluctuation, water surface reflection and the sway, pitch and yaw motion of the carrier itself will pose a serious challenge to the stability and accuracy of visual tracking.

[0007] Preliminary multi-source data fusion attempt: although existing multi-source data fusion technologies have been applied to marine target tracking, they mostly stay at the shallow fusion level, and have the following limitations in complex dynamic sea conditions:

[0008] (1) Insufficient positioning accuracy: the fusion model cannot effectively compensate for the combined errors caused by the dramatic changes in the carrier attitude and sea wave interference.

[0009] (2) Robustness is weak: when the data quality of a key sensor (such as GNSS) decreases or is temporarily interrupted, the overall performance of the system will deteriorate sharply, and even cause tracking failure.

[0010] (3) Limited real-time performance: complex fusion algorithms usually have large computational load, and it is difficult to realize real-time and stable tracking and processing of multiple targets on resource-limited embedded platforms.

[0011] Therefore, there is an urgent need for a new method of offshore target positioning and tracking that can deeply fuse data from multiple heterogeneous sensors, effectively cope with complex dynamic sea conditions, and balance high precision, strong robustness, and high real-time performance. SUMMARY

[0012] The present application aims to overcome the defects of the prior art and provide a multi-source data fusion offshore target real-time positioning and tracking system and method. By deeply fusing high-precision RTK positioning data, laser ranging data, optical pod attitude information, and AI vision algorithms, the positioning accuracy, tracking robustness, and system real-time performance of offshore targets in complex dynamic sea conditions (such as Beaufort wind scale 5 or above) are significantly improved.

[0013] To achieve the above objectives, the present application needs to solve the following key technical problems:

[0014] (1) Modeling and compensation of compound errors in dynamic sea conditions: the carrier (such as a drone or an unmanned ship) will produce violent rolling, pitching, and heaving movements in the wind and waves, directly leading to changes in the spatial position and attitude of the RTK antenna, laser range finder, and camera. How to accurately unify the measurement data of each sensor to a stable world coordinate system and compensate for the nonlinear, time-varying errors caused by the coupling of sea wave disturbance and carrier motion is the primary challenge to achieve high-precision positioning.

[0015] (2) Spatio-temporal registration and adaptive weighted fusion of heterogeneous sensor data: the sensor data involved in the present application comes from various sources (RTK position, laser distance, IMU attitude, camera image), and their data formats, coordinate systems, update frequencies, and error characteristics differ significantly. Achieving nanosecond-level time synchronization and millimeter-level spatial coordinate transformation is the basis for data fusion; how to dynamically adjust the weight of each sensor in the fusion algorithm according to its real-time "health status" (such as RTK solution status, AI vision confidence, laser echo signal-to-noise ratio) at different times is the core of improving system robustness and accuracy.

[0016] (3) Continuity and stability of target tracking when sensor data is partially lost: In real sea environment, any single sensor may have a short-term failure or performance drop (such as RTK loss of lock, visual target blocked by sea waves or obstacles). It is a key indicator to measure the robustness of the system to rely on other effective data sources and high-fidelity motion models to smooth prediction and maintain target trajectory when partial data is missing, avoiding tracking interruption or trajectory jump.

[0017] (4) Real-time optimization and scheduling of computing resources in multi-target tracking: When tracking multiple sea targets simultaneously, especially when AI visual detection and complex fusion algorithms run in parallel, the computing power of the processing unit is extremely high. How to design a dynamic task allocation and resource scheduling mechanism, intelligently allocate computing resources (such as adjusting AI inference frequency, laser ranging frequency) according to target threat level, motion state and other factors, while maintaining the real-time response capability of the system, is the technical bottleneck to realize the engineering application of the invention.

[0018] To solve the above technical challenges,

[0019] The present invention first provides a multi-source data fusion real-time positioning and tracking system for sea targets, characterized by comprising a sensor part and a central data processing unit.

[0020] The sensor part includes an RTK-GNSS receiver, a laser range finder, and a high-definition camera and a high-precision IMU integrated in an optoelectronic pod, responsible for collecting raw data.

[0021] The central data processing unit is responsible for executing all algorithms. The data acquisition and time synchronization module is responsible for receiving and aligning the sensor data; the multi-level coordinate system registration and transformation module uses IMU data and pre-set calibration parameters to unify all observation data to the world coordinate system; the AI visual target detection module processes video streams to identify targets; the adaptive federated Kalman filter fusion module is the core of data fusion, which receives processed data from each channel and outputs the optimal target state estimation; the dynamic task and resource scheduling module optimally allocates system resources according to the current tracking state, forming a closed-loop control.

[0022] The hardware carrier of the method can be an unmanned aerial vehicle (UAV) or an unmanned surface vehicle (USV), which is equipped with the following:

[0023] High-precision RTK-GNSS receiver: provides three-dimensional position and velocity information of the carrier in the WGS-84 coordinate system.

[0024] Laser range finder: installed on the optoelectronic pod, parallel to the camera optical axis, providing accurate straight-line distance to the target.

[0025] High-performance optical pod: built-in high-definition visible light / infrared camera and high-precision inertial measurement unit (IMU). The IMU outputs the three-axis angular velocity and acceleration of the pod (i.e., the camera and laser) in real time for accurate attitude calculation.

[0026] Central data processing unit: a high-performance embedded computing platform responsible for running all algorithms described in the present invention.

[0027] The present invention provides a multi-source data fusion offshore target real-time positioning and tracking method, which includes six core steps of data acquisition and preprocessing, multi-level coordinate system registration and transformation, AI visual target enhanced detection, target spatial position calculation, AFKF state fusion estimation and dynamic task scheduling optimization.

[0028] Specifically:

[0029] Step 1: Multi-source data acquisition and time synchronization preprocessing;

[0030] The central data processing unit collects the position / speed data of the RTK receiver, the distance data of the laser range finder, the attitude data of the IMU, and the video stream of the camera at a high frequency (e.g., 100Hz).

[0031] All data are attached with high-precision timestamps (using PTP protocol or PPS second pulse signal) to ensure time alignment in subsequent processing, with a time synchronization accuracy better than 1ms.

[0032] Step 2: Multi-level coordinate system registration and transformation;

[0033] To eliminate the influence of carrier motion, the following coordinate systems need to be established and converted:

[0034] World coordinate system (W): using WGS-84 geodetic coordinate system.

[0035] Carrier body coordinate system (B): fixed to the geometric center of the UAV or unmanned ship.

[0036] IMU coordinate system (I): fixed to the IMU inside the optical pod.

[0037] Camera coordinate system (C): fixed to the optical center of the camera.

[0038] Laser coordinate system (L): fixed to the laser range finder light outlet.

[0039] Through pre-calibration, the rigid transformation matrix between each sensor is obtained The IMU calculates its own attitude matrix in the world coordinate system in real time RTK provides the position of the carrier in the world coordinate system

[0040] Any point P in the camera coordinate system C Transform to the world coordinate system P W :

[0041]

[0042] This transformation is the basis for all subsequent calculations, and its mathematical derivation involves the operation of rotation matrix and translation vector, effectively eliminating the influence of the carrier's own motion on target observation.

[0043] Step three: AI vision target detection and enhanced positioning;

[0044] Apply an advanced deep learning target detection model (such as the YOLO series) to the camera real-time video stream. This model has been trained on a large number of marine target samples and can accurately identify targets such as ships and buoys, and output their two-dimensional bounding boxes (u, v, width, height) and detection confidence conf in the image.

[0045] Using the bounding box center (u, v) and the camera intrinsic matrix K, the direction vector of the target in the camera coordinate system can be calculated

[0046] Step four: preliminary solution of target spatial position based on laser and vision fusion;

[0047] The laser range finder measures the target distance D laser .

[0048] Combined with the direction vector obtained in step three The three-dimensional position of the target in the camera coordinate system can be preliminarily calculated

[0049] Using the coordinate transformation formula in step two, convert P C to the world coordinate system to get an observation position fused by vision and laser

[0050] Step five: adaptive federated Kalman filter (AFKF) fusion;

[0051] The core of the invention is the AFKF fusion framework, which consists of a master filter and multiple sub-filters.

[0052] State vector definition: both the master filter and the sub-filter maintain a target state vector X = [x, y, z, v x , v y , v z , a x , a y , a z ]T , including the three-dimensional position, velocity and acceleration of the target in the world coordinate system.

[0053] Sub-filter 1 (vision-laser subsystem):

[0054] Measurement: observed position calculated using step four

[0055] Measurement noise covariance matrix (R VL ): this matrix is adaptive, its size is inversely proportional to the AI detection confidence conf, the signal-to-noise ratio returned by the laser rangefinder, and the pixel size of the target in the image. When the AI confidence is low or the laser signal is weak, R VL is increased, reducing the weight of this observation.

[0056] Sub-filter 2 (RTK-carrier subsystem):

[0057] Measurement: in theory, the world coordinates of the target are equal to the world coordinates of the carrier plus the vector of the target relative to the carrier. Therefore, the carrier position provided by RTK can be used as another indirect source of observation information.

[0058] Measurement noise covariance matrix (R RTK ): this matrix is also adaptive. Its size is dynamically adjusted according to the RTK solution state (fixed solution, floating solution, single-point solution) and positioning accuracy factor (such as PDOP). When RTK is in a high-precision fixed solution state, R RTK value is very small, its weight is high.

[0059] Information fusion:

[0060] (1) Each sub-filter independently performs state prediction and update, obtaining its own state estimate X i and covariance matrix P i .

[0061] (2) The master filter collects information from each sub-filter according to the information distribution principle.

[0062] (3) The master filter calculates the globally optimal state estimate X g and covariance P g according to the following fusion formula:

[0063]

[0064] This federated structure ensures that even if a subsystem (such as vision-laser) temporarily fails due to target occlusion, the system can still rely on other subsystems (such as RTK and motion model) to continue smooth tracking, greatly enhancing robustness.

[0065] Trajectory generation: the output of AFKF is the smooth and high-precision 3D trajectory of the target.

[0066] Step six: dynamic task and computing resource scheduling based on priority;

[0067] To achieve multi-target real-time tracking, the application introduces a dynamic resource scheduling mechanism.

[0068] Target priority evaluation: the system assigns a dynamic priority to each target based on factors such as speed, distance, and whether it is a new target.

[0069] Dynamic resource allocation: the scheduling algorithm dynamically adjusts the allocation of computing resources based on target priority. For example, for high-priority targets, increase the AI detection frame rate and laser ranging frequency; for low-priority (such as slow and distant) targets, appropriately reduce the update rate.

[0070] Optimization algorithm: model the scheduling problem as a constrained optimization problem and use a heuristic algorithm (such as an improved particle swarm algorithm) to solve it, maximizing overall tracking performance under limited computing resources.

[0071] The application has the following significant technical effects and advantages compared to the prior art through the above technical solutions:

[0072] (1) Significant improvement in positioning accuracy: fusion of RTK absolute position reference, laser direct distance measurement, and high-resolution visual direction pointing, achieving multi-source complementarity. The adaptive federated Kalman filter can dynamically balance the real-time credibility of different sensors, effectively suppressing non-Gaussian noise introduced by sea waves, carrier sway, etc.

[0073] (2) Robustness is greatly enhanced: the federated filtering architecture gives the system the ability to "gracefully degrade". When AI vision temporarily fails due to bad weather or obstruction, the visual-laser sub-filter weight automatically decreases, and the system mainly relies on RTK and motion model for track prediction, maintaining uninterrupted tracking; when the RTK signal is lost, the system mainly relies on high-frequency visual-laser relative positioning information and IMU attitude and heading calculation, also maintaining stable tracking.

[0074] (3) Real-time performance and multi-target processing capacity optimization: the innovative dynamic task and computing resource scheduling algorithm intelligently focuses limited computing power on the most important targets, avoiding the waste of computing resources caused by uniform high-frequency processing of all targets.

[0075] (4) Strong environmental adaptability: fusion of active (laser) and passive (visual) detection methods, combined with RTK positioning, enables the application to work in different lighting conditions (day / night), weather conditions (sunny / rainy / thin fog), and sea conditions. The application of AI models also enables the system to have good recognition and tracking ability for different types and sizes of marine targets.

[0076] Practical application value: The method still has high precision and robustness under dynamic sea conditions, and is suitable for target monitoring, locking and navigation application of the optical pod in complex environment. BRIEF DESCRIPTION OF DRAWINGS

[0077] Figure 1 The system overall architecture of the present application is shown in the figure.

[0078] Figure 2 The data processing flowchart of the present application is shown in the figure.

[0079] Figure 3 The computing resource allocation diagram of the embodiment is shown in the figure. DETAILED DESCRIPTION

[0080] The specific technical solutions of the present application are described in combination with the drawings and embodiments.

[0081] Figure 1 The system overall architecture of the present application is shown in the figure. The system is mainly composed of a sensor part and a central data processing unit. The sensor part includes an RTK-GNSS receiver, a laser range finder, and a high-definition camera and a high-precision IMU integrated in the optical pod, which is responsible for collecting raw data. The central data processing unit is the core of the present application, which is responsible for executing all algorithms. The data acquisition and time synchronization module is responsible for receiving and aligning the sensor data; the multi-level coordinate system registration and transformation module uses the IMU data and the preset calibration parameters to unify all observation data to the world coordinate system; the AI visual target detection module processes the video stream to identify the target; the adaptive federated Kalman filter (AFKF) fusion module is the core of data fusion, which receives the processed data of each channel and outputs the optimal target state estimation; the dynamic task and resource scheduling module optimally allocates the system resources according to the current tracking state, forms a closed loop control, and ensures the real-time and high efficiency of the system in the multi-target scene.

[0082] Figure 2The data processing flow of the method of the application is described in detail. The flow starts with parallel collection of multi-source data. After high-precision time synchronization, the data is divided into two paths: the video stream enters the AI vision processing module to obtain the pixel coordinates and confidence of the target; the IMU data is used for attitude solution. Subsequently, in the coordinate system transformation and preliminary solution module, the visual direction, laser distance, carrier attitude and position are fused to calculate the target world coordinate observation values from different sources. These observation values together with their credibility information (AI confidence, RTK state, etc.) are sent to the core AFKF fusion module. Inside the AFKF module, first, the measurement noise covariance matrix is adaptively adjusted according to the credibility information, then each sub-filter runs independently, and finally the information is fused by the master filter to obtain the globally optimal state estimation. For multi-target scenes, the system will start the dynamic task and resource scheduling module to optimize the data acquisition frequency, forming a feedback closed loop. The whole process embodies the core idea of data from coarse to fine, step-by-step fusion and purification, and closed-loop optimization control.

[0083] Embodiment 1: High-precision real-time search and rescue of a person falling into the sea

[0084] (I) Technical scene pain points

[0085] Under the conditions of Puffe's 6th sea state, the traditional search and rescue means have:

[0086] 1. Radar is disturbed by sea clutter, and the detection rate of small targets (such as fallen people) is <30%

[0087] 2. Single visual tracking causes positioning error >5 meters due to carrier sway

[0088] 3. Response delay >2 minutes, delaying the golden rescue time

[0089] (II) Technical means and implementation steps

[0090] Step 1: Multi-sensor cooperative collection

[0091] 1. Laser-vision fusion positioning

[0092] The photoelectric pod carries an infrared camera (night mode) to lock the human heat source, and a laser range finder to obtain the target distance 2. RTK anti-drift compensation

[0093] The carrier RTK outputs the position (fixed solution state) in real time, and the influence of ship heave motion is removed through the coordinate transformation matrix Step 2: AFKF adaptive weight distribution

[0094] Table 1 Weight distribution table

[0095]

[0096] Note: When the visual confidence drops to 0.5 due to wave blocking, automatically increase the RTK weight to 70%

[0097] Step 3: Dynamic Resource Scheduling

[0098] Priority function:

[0099]

[0100] Where α, β, γ: weight coefficients (default 0.6, 0.3, 0.1)

[0101] Resource allocation: as shown in Figure 3 .

[0102] Example 2: Busy port multi-target ship collision avoidance monitoring

[0103] (I) Technical scene pain points

[0104] Port environment exists:

[0105] 1. Ship density > 50 vessels per square kilometer, traditional AIS update frequency is low (2-10 seconds / time)

[0106] 2. Positioning deviation caused by radar multipath effect > 10m

[0107] 3. Calculation resource competition causes target trajectory cross misjudgment

[0108] (II) Technical means and implementation steps

[0109] Step 1: Heterogeneous data space-time registration

[0110] 1. AIS-visual data association

[0111] Use YOLOv7 to identify ship types (cargo ships / fishing boats) and automatically match with AIS ship IDs

[0112] 2. Laser-assisted ranging

[0113] For small fishing boats without AIS, generate a 3D bounding box through laser point cloud

[0114] Step 2: Federated filtering multi-target tracking

[0115] 1. Sub-filter design

[0116] Filter 1: Fusion of AIS position + visual size features (ship length matching)

[0117] Filter 2: Laser point cloud clustering + RTK relative displacement

[0118] 2. Conflict resolution mechanism

[0119] When the predicted trajectories of the two ships intersect, the audible and visual warning is activated and the anti-collision constraint equation is injected:

[0120]

[0121] where D safe : safety distance (default 50 meters); T: prediction time window (default 10 seconds)

[0122] Step 3: Dynamic partitioning of resources

[0123] Table 2 Dynamic partitioning of resources

[0124] Port area Target threat level Resource allocation strategy Main channel (Zone A) High 100% CPU core, AI detection frame rate 25Hz Anchorage area (Zone C) Low 20% CPU core, detection frame rate 5Hz

[0125] Note: Dynamic adjustment of partition weights based on ship speed, cargo capacity (AIS data)

[0126] The two embodiments verify the universality of the patent technology in extreme dynamic environments (high sea state search and rescue) and high concurrency scenarios (port monitoring). Through deep fusion of multiple data sources, adaptive resource scheduling, and federated filtering architecture, the positioning accuracy breaks through the sub-meter level, and the number of multi-target tracking is improved by orders of magnitude, providing a technical foundation for smart maritime affairs.

Claims

1. A multi-source data fusion offshore target real-time positioning and tracking system, characterized in that, The system comprises a sensor part and a central data processing unit; The sensor part comprises an RTK-GNSS receiver, a laser range finder, and a high-definition camera and a high-precision IMU integrated in an optical pod, responsible for collecting raw data; The central data processing unit is responsible for executing all algorithms; The data acquisition and time synchronization module is responsible for receiving and aligning sensor data; The multi-level coordinate system registration and transformation module uses IMU data and pre-set calibration parameters to unify all observation data into the world coordinate system; the AI visual target detection module processes video streams to identify targets; The adaptive federated Kalman filter fusion module receives processed data from each channel and outputs the optimal target state estimation; the dynamic task and resource scheduling module optimally allocates system resources according to the current tracking state, forming a closed-loop control.

2. A multi-source data fusion real-time positioning and tracking method for marine targets, characterized in that, The system comprises a sensor part and a central data processing unit; Step one: multi-source data acquisition and time synchronization preprocessing; The central data processing unit collects position / speed data from the RTK receiver, distance data from the laser range finder, attitude data from the IMU, and video streams from the camera at a high frequency; All data are attached with high-precision time stamps to ensure time alignment in subsequent processing, with a time synchronization accuracy better than 1ms; Step two: multi-level coordinate system registration and transformation; The following coordinate systems are established and converted: World coordinate system W: WGS-84 geodetic coordinate system; Carrier body coordinate system B: geometric center fixed to the UAV or unmanned ship; IMU coordinate system I: IMU fixed in the optical pod; Camera coordinate system C: camera optical center; Laser coordinate system L: laser range finder light outlet; The rigid transformation matrix between each sensor is obtained by pre-calibration The IMU calculates the attitude matrix of itself in the world coordinate system in real time The RTK provides the position of the carrier in the world coordinate system Any point P in the camera coordinate system C Transformed to the world coordinate system P by the following equation W : Step three: AI visual target detection and enhanced positioning; Using the center of the bounding box (u, v) and the camera intrinsic matrix K, the direction vector of the target in the camera coordinate system is calculated Step four: target spatial position preliminary solution based on laser and vision fusion; Laser range finder measures target distance D laser ; combining the direction vectors obtained in step three preliminarily calculating the three-dimensional position of the target in the camera coordinate system Using the coordinate transformation formula of step two, convert P C to the world coordinate system to get a fused vision and laser observation position Step five: adaptive federated Kalman filter (AFKF) fusion; The AFKF fusion framework consists of a master filter and multiple sub-filters; State vector definition: both the master filter and the child filter maintain a state vector X = [x, y, z, v x ,v y ,v z ,a x ,a y ,a z ] T for the target, containing the target's three-dimensional position, velocity, and acceleration in the world coordinate system; Sub-filter 1: vision-laser subsystem: Measurement: Observed position calculated using Step Four Measuring noise covariance matrix R VL : This matrix is adaptive, and its size is inversely proportional to the AI detection confidence conf, the signal-to-noise ratio returned by the laser rangefinder, and the pixel size of the target in the image; when the AI confidence is low or the laser signal is weak, increase R VL , reduce the weight of this observation value; Sub-filter 2: RTK-carrier subsystem: Measurement: In theory, the world coordinates of the target are equal to the world coordinates of the carrier plus the vector of the target relative to the carrier; therefore, the carrier position provided by the RTK as another indirect source of observation information; Measuring noise covariance matrix R RTK This matrix is also adaptive; its size is dynamically adjusted according to the resolution state of RTK and the positioning accuracy factor; when RTK is in a high-precision fixed solution state, R RTK The value is very small, and its weight is high; Information fusion: (1) Each sub-filter independently performs state prediction and update, and obtains its own state estimate X i and covariance matrix P i ; (2) The master filter collects information from each sub-filter according to the information distribution principle; (3) The master filter computes the globally optimal state estimate X according to the following fusion formula g and covariance P g : Motion trajectory generation: the output of AFKF is a smooth and high-precision target three-dimensional motion trajectory; Step six: dynamic task and computing resource scheduling based on priority; Target priority evaluation: the system assigns a dynamic priority based on the speed, distance, and whether it is a new target of each target; Resource dynamic allocation: the scheduling algorithm dynamically adjusts the allocation of computing resources based on target priority; Optimization algorithm: model the scheduling problem as a constrained optimization problem and solve it using a heuristic algorithm to maximize the overall tracking performance under limited computing resources. The system comprises a sensor part and a central data processing unit; Step one: multi-source data acquisition and time synchronization preprocessing; The central data processing unit collects position / speed data from the RTK receiver, distance data from the laser range finder, attitude data from the IMU, and video streams from the camera at a high frequency; All data are attached with high-precision time stamps to ensure time alignment in subsequent processing, with a time synchronization accuracy better than 1ms; Step two: multi-level coordinate system registration and transformation; The following coordinate systems are established and converted: World coordinate system W: WGS-84 geodetic coordinate system; Carrier body coordinate system B: geometric center fixed to the UAV or unmanned ship; IMU coordinate system I: IMU fixed in the optical pod; Camera coordinate system C: camera optical center; Laser coordinate system L: laser range finder light outlet; Step three: AI visual target detection and enhanced positioning; Step four: target spatial position preliminary solution based on laser and vision fusion; Step five: adaptive federated Kalman filter (AFKF) fusion; The AFKF fusion framework consists of a master filter and multiple sub-filters; Sub-filter 1: vision-laser subsystem: Sub-filter 2: RTK-carrier subsystem: Information fusion: (2) The master filter collects information from each sub-filter according to the information distribution principle; Motion trajectory generation: the output of AFKF is a smooth and high-precision target three-dimensional motion trajectory; Step six: dynamic task and computing resource scheduling based on priority; Target priority evaluation: the system assigns a dynamic priority based on the speed, distance, and whether it is a new target of each target; Resource dynamic allocation: the scheduling algorithm dynamically adjusts the allocation of computing resources based on target priority; Optimization algorithm: model the scheduling problem as a constrained optimization problem and solve it using a heuristic algorithm to maximize the overall tracking performance under limited computing resources.

Images