Adaptive data fusion method for multi-source heterogeneous sensors facing dynamic targets

Abstract

The application discloses a kind of multi-source heterogeneous sensor adaptive data fusion methods for dynamic target.First, the azimuth angle, elevation angle and distance data collected by multi-source heterogeneous sensor are preprocessed, forming uniform time sequence three-dimensional observation vector;Then extract angular velocity, angular acceleration and other motion characteristics and compare with preset threshold, distinguish target is in stable or strong maneuvering motion state;According to the discrimination result, the fusion scheme is switched using the asymmetric hysteresis comparison strategy, the Kalman filter based on the second-order exponential moving average is enabled in stable state, and the long short-term memory network with double attention mechanism is enabled in strong maneuvering state, and the optimal state estimation of target is output.The application realizes double-mode adaptive switching, noise covariance online estimation and hardware acceleration, and realizes real-time and high precision on embedded platform, improves the adaptability of dynamic target tracking in complex scene.

Description

Technical Field

[0001] This invention relates to the field of multi-sensor data fusion and embedded real-time processing technology, specifically to an adaptive data fusion method for multi-source heterogeneous sensors oriented towards dynamic targets. Background Technology

[0002] With the rapid development of intelligent unmanned systems and autonomous driving technologies, accurate and reliable target state perception has become the core foundation for system safety and autonomous decision-making. In practical engineering applications, systems typically rely on multiple heterogeneous sensors working together to acquire multi-dimensional information such as the target's azimuth, pitch, and distance. However, these sensors differ significantly in sampling rate, data format, noise characteristics, and physical principles, resulting in multi-source heterogeneous data streams that pose a severe challenge to real-time fusion processing.

[0003] The current mainstream multi-source sensor fusion methods are mainly divided into two categories: one is the classic fusion algorithm based on Kalman filtering and its variants, and the other is the intelligent fusion method based on deep learning models.

[0004] Kalman filtering-based fusion methods (such as extended Kalman filtering and unscented Kalman filtering) can achieve efficient and low-latency state estimation under the assumptions of a relatively stable target motion model and a Gaussian noise distribution. They have low computational resource requirements and are easy to deploy on embedded platforms. However, when the target is in a highly maneuvering, dynamic, or nonlinear trajectory (such as sharp turns or evasive maneuvers), the fixed noise covariance assumption is difficult to match the actual dynamic changes, leading to a significant decrease in filtering accuracy or even divergence.

[0005] Deep learning-based intelligent fusion methods (such as Long Short-Term Memory networks and graph neural networks) demonstrate higher fusion accuracy in complex trajectory scenarios due to their powerful nonlinear modeling capabilities. However, these models typically have large parameters and are computationally intensive. Direct deployment on resource-constrained embedded platforms (such as ARM processors) faces severe real-time bottlenecks, making it difficult to meet millisecond-level response requirements. Existing research largely relies on high-performance GPUs or cloud computing, leading to problems such as high power consumption, high latency, and decreased reliability.

[0006] Furthermore, most existing fusion methods employ a single algorithm architecture, lacking the ability to dynamically and adaptively switch based on the target's motion trajectory. When the target transitions from a stable motion to a highly maneuverable state (high-speed movement), the filter cannot adjust in time, resulting in a lag in state estimation; conversely, continuously using complex deep learning models during the stable phase leads to a waste of computational resources.

[0007] Therefore, there is an urgent need for a multi-source heterogeneous sensor data fusion method that can adaptively switch fusion strategies according to the target motion trajectory, balance real-time performance and high precision, and can be efficiently implemented on embedded edge platforms, so as to break through the performance bottleneck of current technology in dynamic and complex scenarios. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention provides an adaptive data fusion method for multi-source heterogeneous sensors oriented towards dynamic targets, aiming to solve the problems in the background technology.

[0009] To achieve the above objectives, the present invention provides the following technical solution: an adaptive data fusion method for multi-source heterogeneous sensors oriented towards dynamic targets, comprising the following steps: Step S1: Collect multi-source heterogeneous observation data of moving targets in real time using various heterogeneous sensors, preprocess the multi-source heterogeneous observation data, and form a unified time-series three-dimensional observation vector. Step S2: Extract motion features from the three-dimensional observation vector; compare the motion features with the preset discrimination threshold to determine the current motion state of the moving target; the motion state includes steady motion state and strong maneuvering motion state; Step S3: Based on the determined motion state of the moving target, an asymmetric hysteresis comparison strategy is used to adaptively switch the fusion scheme. In the stable motion state, an improved Kalman filter fusion scheme is activated. At non-initial moments, the optimal state estimate from the previous moment is fused with the current 3D observation vector to update the state and output the optimal state estimate of the moving target at the current moment. At the initial moment, the current observation vector is directly used to initialize the state estimate to start the filtering recursion. In the strong maneuvering motion state, a dual attention mechanism long short-term memory network fusion scheme is activated to fuse multiple consecutive frames of temporal 3D observation vectors and output the optimal state estimate of the moving target at the current moment.

[0010] Furthermore, the improved Kalman filter fusion scheme takes the target's optimal state estimate and corresponding error covariance from the previous moment as input, and combines it with the current three-dimensional observation vector. The optimal state estimate of the moving target at the current moment is solved iteratively through the prediction and update equations of the Kalman filter. In this process, the process noise covariance and measurement noise covariance parameters are dynamically optimized using a second-order exponential moving average, and the Kalman gain is adjusted in real time to make the filter adapt to noise changes. The dual-attention mechanism long short-term memory network fusion scheme takes the temporal three-dimensional observation vectors from the current moment and previous consecutive frames as input, extracts the temporal hidden state through a two-layer bidirectional long short-term memory network, and then adaptively weights and aggregates the outputs of each time step through a temporal attention module. Subsequently, the feature attention module performs channel weighting on each feature dimension, and finally maps and outputs the optimal state estimate of the moving target at the current moment through a fully connected layer.

[0011] Furthermore, the multi-source heterogeneous observation data of the moving target includes the azimuth, elevation, and range information of the moving target; The three-dimensional observation vector at time t is represented as , express The azimuth angle value of the moving target after preprocessing at each time step. express The pitch angle value of the moving target after preprocessing at each moment. express The distance of the moving target after time-mapping is completed; This indicates transpose; the three-dimensional observation vectors at all times constitute a time-series three-dimensional observation vector dataset.

[0012] Furthermore, the motion features extracted from the three-dimensional observation vector in step S2 include angular velocity, angular acceleration, rate of change of linear velocity, and trajectory curvature; the preset discrimination thresholds include a low-maneuver threshold and a high-maneuver threshold. If all motion features are below the corresponding low maneuver threshold, the motion is classified as a steady motion state; if any motion feature is above the corresponding high maneuver threshold, the motion is classified as a strong maneuver motion state; if it is in between, it is also classified as a strong maneuver motion state.

[0013] Furthermore, step S2 also includes online adaptive estimation of process noise covariance based on the time-series three-dimensional observation vector dataset generated in step S1. and measurement noise covariance : Measuring noise covariance The estimate is based on the observation residuals within the sliding window: set the sliding window size. Calculate the first digit within the sliding window. Three-dimensional observation vectors at each fusion moment With the Predicted observations at each fusion time The deviation is calculated, and then its covariance matrix is ​​plotted to obtain the measurement noise covariance; where the predicted observation value is... Predicted value from the state at the previous time step Through the observation matrix The mapping yields the state prediction values ​​at non-initial times, derived from the optimal state estimate output in the previous fusion cycle via the state transition matrix. The state prediction value at the initial moment is obtained by recursively deriving from the target motion model; the state prediction value at the initial moment is obtained by converting the three-dimensional observation vector of the first frame into Cartesian coordinates and setting the velocity to zero, and is used as the starting point for subsequent recursion. Process noise covariance The estimation is based on the innovation vector of the Kalman filter. It uses a recursive formula for online updates; express The state prediction value at time 1.

[0014] Furthermore, in step S3, based on the determined motion state of the moving target, an asymmetric hysteresis comparison strategy is used to adaptively switch the fusion scheme, including: Set global toggle flag Used to mark the currently enabled fusion scheme: When the output judgment result of step S2 is a steady motion state, the global switching flag will be set. Set to 0 to enable the improved Kalman filter fusion scheme; when the output judgment result of step S2 is a strong maneuvering motion state, Set to 1 to enable the dual attention mechanism long short-term memory network fusion scheme to fuse consecutive multi-frame temporal three-dimensional observation vectors; Set an asymmetric hysteresis comparison strategy: when the S2 discrimination results for three consecutive fusion times are all in a strong maneuvering motion state, the global switching flag will be activated. Switch from 0 to 1; conversely, for five consecutive fusion time points, the global switching flag must be changed to indicate a stable motion state. Switch back from 1 to 0; The asymmetric hysteresis threshold value is dynamically adjusted based on the measurement noise covariance obtained in step S2: if the currently calculated measurement noise covariance... If the value exceeds the preset measurement noise covariance threshold, the asymmetric hysteresis threshold will be raised, increasing the requirement from 3 consecutive fusion times to 5 consecutive fusion times.

[0015] Further improvements to the Kalman filter fusion scheme include: When in a stable motion state, an improved Kalman filter algorithm based on exponential moving average is executed. The inputs to the improved Kalman filter algorithm based on exponential moving average are: the optimal state estimate and its error covariance matrix at the previous time step, and the three-dimensional observation vector at the current time step; the output is the optimal state estimate at the current time step. and its error covariance matrix; In the first fusion cycle, the three-dimensional observation vectors obtained for the first time are used. Initialization: Convert the observed values ​​to initial positions in Cartesian coordinates, set the velocity to zero, and construct the initial state vector. Simultaneously, based on the sensor's nominal ranging and angular measurement accuracy, the error covariance matrix corresponding to the dimension of the initial state vector is initialized. Process noise covariance matrix and the measurement noise covariance matrix ; The iterative fusion process begins from the second fusion cycle: Process noise covariance in step S2 and measurement noise covariance Based on this, a second-order exponential moving average model is further used to dynamically smooth and optimize its diagonal elements, and adaptively adjust the Kalman gain. First, calculate the three-dimensional observation vector at the current moment. The residual vector between the observed and predicted values ​​is the innovation vector of the Kalman filter; the square norm of the residual vector is defined as the residual sequence; then, the residual sequence is smoothed using a second-order exponential moving average model based on the level equation and trend equation. Let the process noise covariance matrix at the current time be... Measurement noise covariance matrix ,in, This represents the process noise covariance at the previous time step. express The correction increment for the time-matter noise covariance. This represents the measurement noise covariance matrix at the previous time step. This represents the correction increment for the measurement noise covariance at the current moment; After completing the dynamic estimation of the process noise covariance matrix and the measurement noise covariance matrix, the Kalman filter prediction and update steps are performed: First, use the state transition matrix The optimal state estimate at the previous time step By performing recursion, we can obtain the state estimate at the current time. Simultaneously, based on the process noise covariance matrix obtained in step S2 at the current time... and the error covariance matrix of the previous time step Calculate the error covariance matrix at the current time. ; After completing the prediction, proceed to the update step; based on the measurement noise covariance matrix obtained in step S2 at the current time... and the error covariance matrix at the current time. Combined with the observation matrix Calculate Kalman gain ; Using the current three-dimensional observation vector The residual vector between the observed and predicted values, i.e., the innovation vector of the Kalman filter, represents the state estimate at the current time. Make corrections to obtain the optimal state estimate for the current time step. Simultaneously, update the error covariance matrix; The final output is the optimal state estimate at the current moment. .

[0016] Furthermore, the dual-attention mechanism long short-term memory network fusion scheme is implemented through a dual-attention mechanism long short-term memory network model, which is trained offline in advance; the trained dual-attention mechanism long short-term memory network model is used to fuse the continuous temporal three-dimensional observation vector sequence output by S1. The input layer receives the historical 3D observation vector sequence, with the input data for a single time step being a single frame of 3D observation vector. The historical 3D observation vector sequence is then fed into a two-layer bidirectional long short-term memory network. The specific structure of the two-layer bidirectional long short-term memory network is as follows: both the first and second layers use bidirectional long short-term memory networks. The two-layer bidirectional long short-term memory network processes the input historical 3D observation vector sequence in the order of time steps and extracts the hidden state sequence corresponding to all time steps. The hidden state sequence is weighted and aggregated using a time attention module: First, each hidden state sequence is mapped to a scalar energy value through a multilayer perceptron. Then, the attention weights are obtained by applying the Softmax function to the scalar energy values ​​at all time steps. Finally, the attention weights are multiplied by their corresponding hidden states and summed to obtain the temporal attention output vector. ; The temporal attention output vector is further adaptively weighted along the feature dimension through a feature attention module. First, global average pooling and global max pooling are performed on the temporal attention output vector to obtain global average pooling feature descriptors and global max pooling feature descriptors, respectively. Then, the global average pooling feature descriptors and global max pooling feature descriptors are fed into a shared two-layer MLP. The two branches of the two-layer MLP process the global average pooling feature descriptors and global max pooling feature descriptors, respectively. The outputs of the two branches of the two-layer MLP are added together and then passed through the Sigmoid activation function to obtain the feature weight vector. Finally, the temporal attention output vector is multiplied element-wise with the feature weight vector to obtain the final enhanced feature. The fully connected output layer maps the final enhanced feature to the optimal state estimate.

[0017] An electronic device includes a processor, a memory, and a bus, wherein the processor and the memory are connected via the bus, wherein the memory is used to store a set of program code, and the processor is used to call the program code stored in the memory to execute a multi-source heterogeneous sensor adaptive data fusion method for dynamic targets.

[0018] A non-volatile computer storage medium storing computer-executable instructions that execute an adaptive data fusion method for multi-source heterogeneous sensors oriented towards dynamic targets.

[0019] Compared with existing technologies, the present invention has the following advantages: (1) Adaptive Dual-Mode Fusion: Based on the target motion state, the improved Kalman filter fusion scheme and the dual-attention mechanism long short-term memory network fusion scheme are adaptively switched, which takes into account both the low latency real-time performance in stable scenarios and the high-precision fitting capability in complex maneuver scenarios. Existing technologies mostly adopt a single fusion architecture—the traditional Kalman filter and its variants perform well in stable linear scenarios, but when the target undergoes nonlinear motions such as sharp turns and S-shaped maneuvers, the linearization assumption leads to a sharp drop in estimation accuracy or even divergence; while the pure data-driven deep learning fusion model has nonlinear modeling capabilities, continuous operation in the stable tracking stage will cause unnecessary waste of computing power and increased power consumption, and it is difficult to meet the real-time constraints of embedded platforms. This invention solves the problem that a single algorithm cannot simultaneously meet the requirements of real-time performance and accuracy by using online trajectory type discrimination and adaptive strategy switching to select the most suitable fusion method in the two scenarios.

[0020] (2) Improved EMA adaptation of Kalman filtering: The second-order exponential moving average is used to dynamically estimate the process noise covariance and measurement noise covariance, avoiding the dependence of traditional Kalman filtering on a fixed noise model. Compared with the problems of existing adaptive filtering being prone to divergence and relying on manual parameter tuning, this invention achieves online estimation with low computational cost and stable convergence through EMA smoothing.

[0021] (3) Dual Attention Mechanism LSTM: Based on LSTM, temporal attention and feature attention are introduced to weight the temporal dependency and the dimension of multi-source heterogeneous features respectively, which effectively improves the fusion accuracy and model interpretability under nonlinear trajectory. Existing LSTM treats each time step and sensor channel equally and lacks the ability to filter key information. The dual attention mechanism of this invention automatically focuses on the maneuvering turning point and key mode.

[0022] (4) PS-PL Collaborative Hardware Acceleration: Based on the Zynq platform, deterministic tasks such as data acquisition, preprocessing, and caching are offloaded to the PL side for parallel hardware processing, while the PS side focuses on fusion algorithm scheduling and achieves low-latency data interaction through the AXI bus. Existing pure ARM solutions are prone to frame loss when processing high-throughput multi-channel data, while pure GPU solutions have high power consumption. This invention achieves real-time performance, low power consumption, and high throughput on an embedded platform. Attached Figure Description

[0023] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0024] Example 1

[0025] like Figure 1As shown, this embodiment provides an adaptive data fusion method for multi-source heterogeneous sensors oriented towards dynamic targets, including the following steps:

[0026] Step S1: Collect multi-source heterogeneous observation data of moving targets in real time using various heterogeneous sensors, preprocess the multi-source heterogeneous observation data, and form a three-dimensional observation vector with unified time sequence.

[0027] In this embodiment, multi-source heterogeneous observation data of aerial moving targets are first acquired in real time using various heterogeneous sensors. Sensor types include, but are not limited to, millimeter-wave radar, lidar, electro-optical trackers, or inertial measurement units. Each sensor outputs the azimuth, elevation, and range information of the moving target. In practical engineering, the sampling frequencies of different sensors are often inconsistent; for example, the output frequency of a radar is 20 Hz, while the output frequency of an electro-optical tracker can reach 50 Hz, and there are also random offsets in the sampling time. To obtain a time-aligned multi-source data stream in subsequent fusion, the acquired multi-source heterogeneous observation data must undergo unified preprocessing.

[0028] The preprocessing flow includes frame verification, format conversion, outlier removal, timestamp alignment, and range normalization in sequence. Frame verification checks the integrity of data packets, such as verifying the CRC field in UDP packets or custom frame tail flags, discarding packets that fail verification. Format conversion unifies the binary or text formats defined by different sensor manufacturers into the internally agreed data structure, such as unifying all data into 32-bit floating-point triplets (azimuth, elevation, and range). Outlier removal uses a dynamic method based on the 3σ criterion: for each sensor channel, a historical observation window of length 20 is maintained, and the mean of the data within the window is calculated. and standard deviation If the current observed value deviates from the mean by more than If a value is found to be outlier, it is either replaced by a one-step prediction based on the state of the previous time step, or the frame is discarded and the process waits for the next frame. Timestamp alignment is a challenge in preprocessing. This embodiment employs a linear interpolation method based on the sampling rate to unify all sensor data to a fixed fusion time. Assuming a fusion period of 50 milliseconds (i.e., 20 Hz), for each fusion moment... Find the two closest original sampling points before and after this moment, and calculate the result using linear interpolation. The equivalent observation at time. If a sensor is at... If no valid data is available near the current time, the most recent valid observation is used with an increase in noise variance. Range normalization employs max-min standardization, which normalizes the azimuth angle (typically ranging from -). arrive or 0 to 2 ), pitch angle (typically ranging from - / 2 to / 2) and distance (e.g., 0 to 1000 meters) are linearly mapped to the interval [0,1], and the calculation formula is as follows: ; In the formula, This represents the data after normalization. This represents data to be normalized; Indicates the lower limit of the sensor's physical quantity measurement range; This indicates the upper limit of the sensor's physical quantity measurement range.

[0029] After the above preprocessing, each fusion time A three-dimensional observation vector can be obtained. , express The azimuth angle value of the moving target after preprocessing at each time step. express The pitch angle value of the moving target after preprocessing at each moment. express The distance of the moving target after time-mapping is completed; This indicates transpose; the three-dimensional observation vectors at all times constitute a time-seriesd three-dimensional observation vector dataset, eliminating the time-series misalignment, data conflict, and abnormal interference of the original data.

[0030] Step S2: Extract motion features from the three-dimensional observation vector; compare the motion features with the preset discrimination threshold to determine the current motion state of the moving target; the motion state includes steady motion state and strong maneuvering motion state.

[0031] In this embodiment, relying on the time-series three-dimensional observation vector dataset output in step S1, a historical observation queue is dynamically maintained from the three-dimensional observation vectors continuously output from the dataset at each time point. Based on the continuous multi-frame observation sequence in the historical observation queue, the target motion features are extracted, and the target motion state is divided in combination with a preset discrimination threshold. At the same time, the covariance parameter is calculated online to provide data support for subsequent fusion strategy switching.

[0032] Specifically, the generation and update rule for the historical observation queue is as follows: every time a fusion time has elapsed... Step S1 outputs a brand new 3D observation vector and merges it into the time-series 3D observation vector dataset. The latest generated 3D observation vector is retrieved from this dataset, and the oldest 3D observation vector in the historical observation queue is removed. The latest generated 3D observation vector is then added to the end of the historical observation queue, and the total length of the historical observation queue is continuously maintained at a fixed value. This embodiment A value of 10 corresponds to a 0.5-second time window (because the fusion period is 50 milliseconds); based on the three-dimensional observation vectors in this historical observation queue, four types of motion features are calculated: angular velocity, angular acceleration, rate of change of linear velocity, and trajectory curvature. Angular velocity is estimated from the rate of change of azimuth and pitch angles: the azimuth angles at two adjacent fusion moments in the historical observation queue are used. , With pitch angle , Calculate the rate of change of azimuth angle respectively. Pitch angle change rate Angular velocity is the Euclidean modulus of the rate of change of azimuth and the rate of change of pitch, i.e. ; express The azimuth angle value of the moving target after time-mapping is completed; express The pitch angle value of the moving target after preprocessing at each moment; This represents the difference in azimuth angle between two adjacent frames; Indicates a time interval; This represents the difference in pitch angle between two adjacent frames; express The angular velocity of the moving target at any given moment.

[0033] Angular acceleration is the first-order difference of angular velocity: extracting angular velocities between adjacent time moments. , Calculate angular acceleration ; express The angular velocity of the moving target at any given moment; express The angular acceleration of the moving target at any given time.

[0034] The rate of change of linear velocity is obtained by the first difference of the rate of change of distance: firstly, using the distance between adjacent time points... , Solving for radial linear velocity Combined with the radial velocity at the previous moment The rate of change of linear velocity was calculated. ; express The distance of the moving target after time-mapping is completed; express The rate of change of the linear velocity of the moving target at time t.

[0035] trajectory curvature The reciprocal of the radius of curvature is obtained by calculating the azimuth changes at three consecutive points: the azimuths corresponding to three consecutive fusion moments selected from the historical observation queue. , , The radius of curvature of the trajectory is calculated based on the angular offset of the three azimuth angles, and the reciprocal of the radius of curvature is taken as the trajectory curvature.

[0036] To quantify the discrimination, multiple threshold levels are set: The first threshold (low maneuverability threshold) is set as follows: angular velocity 0.2 radians / second, angular acceleration 0.5 radians / second², and linear velocity change rate 10 meters / second².

[0037] The second threshold (high maneuverability threshold) is set as follows: angular velocity 0.8 radians / second, angular acceleration 2.0 radians / second², and linear velocity change rate 30 meters / second².

[0038] If all parameters of the current motion feature are below the corresponding first threshold, it is identified as a steady motion state (steady linear / uniform motion); if any parameter of the motion feature is above the second threshold, it is identified as a strong maneuvering motion state (strong maneuvering, high dynamics, nonlinear complex trajectory motion); if it is between the two, it can be classified as a medium maneuvering state. In this embodiment, in order to simplify the decision, it is classified as a strong maneuvering motion state to prioritize accuracy.

[0039] Meanwhile, based on the time-series three-dimensional observation vector dataset generated in step S1, the process noise covariance is estimated online adaptively. and measurement noise covariance .

[0040] Measuring noise covariance The estimate is based on the observation residuals within the sliding window: set the sliding window size. Calculate the first digit within the sliding window. Three-dimensional observation vectors at each fusion moment With the Predicted observations at each fusion time The deviation is then calculated, and its covariance matrix is ​​plotted to obtain the measurement noise covariance: ; Among them, the predicted observation value Predicted value from the state at the previous time step Through the observation matrix The mapping yields: Observation matrix Defined as observation function Jacobian matrix at the predicted state value; This represents the target state vector, which typically includes motion state variables such as the target's position and velocity. Indicates azimuth; Indicates the pitch angle; Indicates distance.

[0041] The state prediction values ​​at non-initial times are derived from the optimal state estimate output by the Kalman filter algorithm based on exponential moving average from the previous fusion cycle, through the state transition matrix. Based on the target motion model, the following is derived: ; express The one-step state prediction value at time t. express The optimal state estimate at time t, the state transition matrix It is the core matrix of the Kalman filter prediction step.

[0042] The initial state prediction value is obtained by converting the first frame's three-dimensional observation vector into Cartesian coordinates and setting the velocity to zero, and serves as the starting point for subsequent recursion.

[0043] Process noise covariance The estimation is based on the innovation vector of the Kalman filter. Using a recursive formula Perform online updates, including for The fading factor of time, Indicates the forgetting factor; express Kalman gain at time step; express Transpose of; , They represent time, The error covariance at time; , They represent time, The process noise covariance matrix at time step; express Transpose of; express Transpose of; express The three-dimensional observation vector at time; express The state prediction value at time 1.

[0044] In the initial stage (before) (per cycle), since the sliding window is not full, a preset initial value for the measurement noise covariance is used. Initial values ​​of process noise covariance (Based on the sensor's nominal accuracy and system model uncertainty) Filtering is performed using a Kalman filter algorithm improved by exponential moving average. After the sliding window is full, online adaptive estimation of process noise covariance is initiated. and measurement noise covariance process.

[0045] These two covariance parameters will be used in subsequent steps: Measuring noise covariance On the one hand, it is used to dynamically adjust the hysteresis comparison threshold in step S3, that is, in When the threshold is large, the switching threshold of the smooth transition is appropriately raised to suppress false switching in high-noise environments; on the other hand, it is used for gain calculation and noise adaptive update of the improved Kalman filter in step S4.

[0046] Process noise covariance It is then used for error covariance calculation in the Kalman filter prediction stage in step S4, and dynamically updated with the residual smoothing result under the EMA adaptive framework.

[0047] The real-time estimation of the two covariances mentioned above provides key parameter basis for the adaptive fusion mechanism of the present invention.

[0048] Step S3: Based on the determined motion state of the moving target, an asymmetric hysteresis comparison strategy is used to adaptively switch the fusion scheme. In the stable motion state, an improved Kalman filter fusion scheme is activated. At non-initial moments, the optimal state estimate from the previous moment is fused and updated with the three-dimensional observation vector output in step S1 at the current moment to output the optimal state estimate of the moving target at the current moment. At the initial moment (the first fusion cycle), since there is no optimal state estimate from the previous moment, the current observation vector is directly used to initialize the state estimate (the optimal state estimate of the first fusion cycle) to start the filtering recursion. In the strong maneuvering motion state, a dual attention mechanism long short-term memory network fusion scheme is activated to fuse the three-dimensional observation vectors of multiple consecutive frames to output the optimal state estimate of the moving target at the current moment, suppressing erroneous switching caused by single-frame noise.

[0049] In this embodiment, the target motion state determination result output in step S2 is received, along with the measurement noise covariance estimated in real time. By relying on motion state information and combining an asymmetric hysteresis comparison strategy, the adaptive selection and switching of the fusion scheme is completed, while also incorporating noise covariance. Dynamically adjust the switching threshold to avoid incorrect switching caused by sensor noise disturbances.

[0050] Specifically, setting a global toggle flag Used to mark the currently enabled fusion scheme: When the output judgment result of step S2 is a steady motion state, the global switching flag will be set. Set to 0, and enable the improved Kalman filter fusion scheme. In the first fusion cycle, since there is no optimal state estimate from the previous time step, the first acquired three-dimensional observation vector is used. The target state estimate is directly initialized (e.g., the 3D observation vector is converted to position in Cartesian coordinates, and the velocity is initialized to zero), and this initial state is used as a replacement for the "optimal state information of the previous moment." Subsequent cycles use the optimal state estimate value fused from the previous moment. This scheme has low latency characteristics and is suitable for uniform and stable target tracking; when the output judgment result of step S2 is a strong maneuvering motion state, Set to 1 to enable the dual attention mechanism long short-term memory network fusion scheme to fuse consecutive multi-frame temporal three-dimensional observation vectors, adapting to high-maneuverability and nonlinear motion scenarios of targets.

[0051] To avoid frequent switching of the fusion scheme and deterioration of tracking stability due to single-frame observation noise when the target trajectory is in the critical range, this embodiment configures an asymmetric hysteresis comparison strategy: the global switching flag is only activated when the S2 discrimination results for three consecutive fusion times (three consecutive frames, totaling 150 milliseconds) are all in a strong maneuvering motion state. The global switching flag can be changed from 0 to 1; conversely, the global switching flag can only be changed if the discrimination results of 5 consecutive fusion times (5 consecutive frames) are all in a stable motion state. Switching back from 1 to 0. This asymmetric hysteresis threshold effectively filters out instantaneous single-frame noise interference, ensuring a smooth transition of the fusion strategy.

[0052] The specific values ​​in the asymmetric hysteresis threshold (3 consecutive frames from steady motion to strong maneuvering, 5 consecutive frames from strong maneuvering to steady motion) are not fixed, but can be dynamically adjusted in conjunction with the measurement noise covariance obtained in real time in step S2: if the currently calculated measurement noise covariance... A larger value indicates a higher level of sensor observation noise. Therefore, the asymmetric hysteresis threshold for transitioning from a stable target to a strong maneuver should be appropriately increased (for example, increasing the requirement from 3 consecutive frames to 5 consecutive frames) to reduce the probability of misjudging a stable target as a maneuver or incorrectly switching to a neural network fusion scheme in a high-noise environment.

[0053] The corresponding fusion scheme outputs the optimal state estimate of the moving target. An improved Kalman filter fusion scheme uses the target's optimal state estimate and its error covariance from the previous time step as input, combined with the current three-dimensional observation vector, to iteratively solve for the optimal state estimate of the moving target at the current time step through the Kalman filter's prediction and update equations. During this process, a second-order exponential moving average is used to dynamically optimize the process noise covariance and measurement noise covariance parameters, adjusting the Kalman gain in real time to adapt the filter to noise changes and output a low-latency, highly stable state estimate. The dual-attention mechanism long short-term memory network fusion scheme uses the temporal three-dimensional observation vectors from the current time step and previous consecutive frames as input. A two-layer bidirectional long short-term memory network extracts the temporal hidden state, and a temporal attention module adaptively weights and aggregates the outputs of each time step. Subsequently, a feature attention module performs channel weighting on each feature dimension, and finally, a fully connected layer maps and outputs the optimal state estimate of the moving target at the current time step. High-precision fitting of nonlinear complex trajectories is achieved through temporal and feature dual-attention weighting.

[0054] according to The values ​​of are used to implement two fusion schemes, which are described in detail below.

[0055] Improved Kalman filter fusion scheme ( )

[0056] When in a stable trajectory mode (stable motion state), an improved Kalman filter algorithm based on exponential moving average is executed. The input of this algorithm is: the previous time step ( The optimal state estimate at time ( ) and its error covariance matrix and the current moment ( The three-dimensional observation vector output by step S1 at time (time) The output is the optimal state estimate at the current moment. and its error covariance .

[0057] In the first integration cycle ( Since there is no optimal state estimate from the previous time step, the three-dimensional observation vector obtained initially is used. Initialization: Convert the observed values ​​to initial positions in Cartesian coordinates, set the velocity to zero, and construct the initial state vector. Simultaneously, based on the sensor's nominal ranging and angular measurement accuracy, the error covariance matrix corresponding to the dimension of the initial state vector is initialized. Process noise covariance matrix and the measurement noise covariance matrix .

[0058] Starting from the second cycle, the following iterative fusion process will begin.

[0059] Traditional Kalman filtering relies on a pre-defined fixed process noise covariance. and measurement noise covariance However, in real-world environments, noise characteristics vary depending on the target's motion state and the sensor's environment. To address this issue, this embodiment uses the online estimation obtained in step S2... and Based on this, a second-order exponential moving average (EMA) model is further used to dynamically smooth and optimize its diagonal elements, thereby adaptively adjusting the Kalman gain.

[0060] First, calculate the current time ( Three-dimensional observation vector at time (time) The residual vector between the observed and predicted values ​​(i.e., the innovation vector of the Kalman filter). The square norm of the residual vector is defined as the residual sequence. Subsequently, the residual series was smoothed using a second-order exponential moving average (EMA) model based on the level equation and trend equation.

[0061] The horizontal equation is: ; In the formula, This represents the residual sequence at the current time step; This represents the smoothed level estimate at the current time. This represents the smoothed level estimate from the previous time step; This indicates the trend term from the previous moment; This represents the horizontal smoothing coefficient.

[0062] The trend equation is: ; In the formula, This indicates the trend at the current moment; This represents the trend smoothing coefficient.

[0063] It describes the monotonic trend of the estimated value. Smoothing coefficient and The value range of is (0,1). In this embodiment, Take 0.3, Setting it to 0.1 gives the smoothing result a higher weight to recent data while preserving the long-term trend; Let the process noise covariance matrix at the current time be... Measurement noise covariance matrix ,in, This represents the process noise covariance at the previous time step. express The correction increment for the time-matter noise covariance. This represents the measurement noise covariance matrix at the previous time step. This represents the correction increment for the measurement noise covariance at the current moment; and All are diagonal matrices and the size of their diagonal elements is equal to the smoothed residual term (i.e. Compared to The change or absolute amount is directly proportional; the calculated increment ( and The noise is strictly limited to a preset reasonable range of variation to prevent excessive oscillation of the noise matrix, and finally completes the adaptive noise correction based on the current residual characteristics.

[0064] Finish and After dynamic estimation, the Kalman filter prediction and update steps are performed. First, the state transition matrix is ​​used... The optimal state estimate at the previous time step By performing recursion, we can obtain the state estimate at the current time. Simultaneously, based on the process noise covariance matrix obtained online in step S2 at the current time... and the error covariance matrix of the previous time step Calculate the error covariance matrix at the current time. , is represented as: ; ; After prediction is completed, the update step begins. The measurement noise covariance matrix at the current time is estimated online based on step S2. and the error covariance matrix at the current time. Combined with the observation matrix Calculate Kalman gain , is represented as: ; In the formula, Represents the observation matrix Transpose of; Using the current three-dimensional observation vector The residual vector between the observed and predicted values ​​(i.e., the innovation vector of the Kalman filter) provides an estimate of the state at the current time. Make corrections to obtain the optimal state estimate for the current time step. : ; At the same time, update the error covariance matrix: ; In the formula, express The posterior error covariance matrix corresponding to the optimal state estimate at time step; Represents the identity matrix.

[0065] The final output is the optimal state estimate at the current moment. The state vector It can include the Cartesian coordinates of the target. ,speed And attitude angles, etc. The specific dimensions are set according to the actual tracking requirements.

[0066] Through adaptive EMA adjustment, the Kalman filter can maintain low delay and high stability under steady trajectories, while dynamically adjusting the filter gain to adapt to noise variations, avoiding performance degradation caused by mismatch in a fixed noise model. Finally, the current time... and Save the data as input for prediction calculations in the next fusion cycle, forming a closed-loop iteration.

[0067] Dual attention mechanism long short-term memory network fusion scheme ( )

[0068] The dual-attention mechanism long short-term memory network fusion scheme is implemented through a dual-attention mechanism long short-term memory network model. This model is pre-trained offline, and the training dataset includes trajectories of various typical complex maneuvers such as S-shaped maneuvers, sharp turns, spiral dives, and evasive maneuvers, along with corresponding target state labels. The acquisition and labeling process of the training data is as follows: raw sensor data (azimuth, pitch, and range) under the above-mentioned maneuver trajectories are collected using multi-source heterogeneous sensors (such as millimeter-wave radar and electro-optical trackers) mounted on a mobile platform. Simultaneously, a high-precision integrated navigation system (e.g., a tightly coupled differential GPS and inertial navigation system) is used to synchronously record the true position, velocity, and attitude of the moving target as state labels. After preprocessing the raw sensor data as described in step S1 (time alignment, outlier removal, and range normalization), it is paired with the state labels at corresponding times to form the training dataset. The trained dual-attention mechanism long short-term memory network model is then used to fuse the 20-bit continuous temporal three-dimensional observation vector sequence output from S1.

[0069] The first step involves the input layer receiving the 20-step historical 3D observation vector sequence. Each time step input is a single-frame 3D observation vector. The second step feeds the historical 3D observation vector sequence sequentially into a two-layer bidirectional long short-term memory (Bi-LSTM) network. The specific structure of the two-layer Bi-LSTM network is as follows: The first layer uses a Bi-LSTM network with an input dimension of 3, 64 hidden units in both the forward and backward directions, and outputs a concatenation of the forward and backward hidden states at each time step, resulting in an output dimension of 128. The second layer also uses a Bi-LSTM network with an input dimension of 128, 64 hidden units in both the forward and backward directions, and an output dimension of 128. The two-layer Bi-LSTM network processes the input historical 3D observation vector sequence in time-step order, extracting the hidden state sequence corresponding to all time steps. , Indicates the first The hidden state vector corresponds to each time step; the dimension of the hidden state vector is 128.

[0070] The hidden state sequences are weighted and aggregated using a time-attention module. Specifically, each hidden state sequence is first mapped to a scalar energy value using a multilayer perceptron (MLP). ; In the formula, Indicates the first scalar energy value at each time step; This represents the weight matrix in the temporal attention module; This represents the weight vector in the temporal attention module; This represents the learnable bias parameters in the temporal attention module; This represents the hyperbolic tangent activation function.

[0071] Then, the attention weights are obtained by applying the Softmax function to the scalar energy values ​​at all time steps: ; In the formula, Indicates the first Attention weights for each time step; Represents an exponential function; Indicates the first scalar energy value at each time step; Finally, the weights are multiplied by their corresponding hidden states and summed to obtain the temporal attention output vector. : ; In the formula, This represents the temporal attention output vector, which aggregates the most discriminative temporal information throughout the entire historical window.

[0072] The temporal attention output vector is further processed by the feature attention module. Adaptive weighting is applied along the feature dimension. The feature attention module borrows design ideas from the Convolutional Block Attention Module (CBAM); firstly, the temporal attention output vector is... Perform global average pooling and global max pooling respectively to obtain the global average pooling feature descriptor. and global max pooling feature descriptor Both descriptors have dimensions equal to the temporal attention output vector. Same (128 dimensions). Then, the global average pooling feature descriptor is used. and global max pooling feature descriptor The input is fed into a shared two-layer MLP (the first layer reduces the dimensionality to 8, and the second layer increases it back to 128). The two branches of the two-layer MLP respectively process the global average pooling feature descriptors. and global max pooling feature descriptor The process involves summing the outputs of the two branches of the two-layer MLP and then applying the result to the Sigmoid activation function to obtain the feature weight vector. Finally, the temporal attention output vector is... With feature weight vector Element-wise multiplication yields the final enhanced feature. : ; By using feature attention, the model can automatically increase the response to key sensor feature dimensions (such as the weight of angular velocity features during turning) in the current maneuver state, while suppressing interference from irrelevant or noisy dimensions.

[0073] The fully connected output layer will ultimately enhance the features. Mapped to optimal state estimate The output dimension is consistent with the state vector dimension of the Kalman filter (e.g., 9-dimensional: three-dimensional position, three-dimensional velocity, and three-dimensional attitude), which facilitates subsequent unified processing and evaluation.

[0074] During training of a dual-attention mechanism long short-term memory network model, the loss function Mean squared error (MSE): ; In the formula, This represents the total number of samples in the training batch; Indicates the first The network predicted state value corresponding to each training sample; Indicates the first The true state label corresponding to each training sample; This represents the square of the Euclidean norm.

[0075] The optimizer was Adam, with an initial learning rate of 0.001, a batch size of 64, and 100 training epochs. To prevent overfitting, a Dropout layer was added after the LSTM layers, with a dropout rate of 0.2. After training, the model parameters were fixed and used directly in the online inference phase. Due to the limited computing power of the embedded platform's ARM architecture, this embodiment lightweighted the LSTM inference: ARM NEON inline functions were used to accelerate matrix multiplication, and the weights of the fully connected layers were pre-quantized with 8-bit integers, significantly reducing computational latency without noticeable loss of accuracy.

[0076] This invention is based on the PS-PL collaborative hardware implementation of the Zynq platform.

[0077] At the programmable logic (PL) level, the following key modules were designed to achieve hardware-level parallel reception and preprocessing of multi-source data. First, the Ethernet MAC hard core (GEM3) module is configured in gigabit mode to receive UDP packets from multiple sensor simulators or network switches in real time. This module can capture data at a line rate of 1000 Mbps and extract the payload, storing it in its internal receive FIFO. Second, the data preprocessing module is implemented using pipelined Verilog code, sequentially performing frame verification (checking a custom frame tail CRC), format conversion (parsing the byte stream into 32-bit floating-point numbers), outlier removal (3σ detection via a hardware comparator, with window mean and standard deviation calculated by a sliding accumulator), timestamp alignment (using a high-precision counter within the FPGA to timestamp each arriving packet, then interpolating based on the timing signal of the fusion cycle), and range normalization (using a floating-point multiplier for linear mapping). All preprocessing operations are pipelined within the same clock domain (200 MHz), with a total delay of less than 1 microsecond from packet entry to normalization output. Then, the hardware circular buffer is implemented based on Block RAM, with a depth of 4096 and a bit width of 64 bits. It adopts a dual-port design, with one port for writing by the preprocessing module and the other port for reading by AXI DMA. When the PS end is not processing in time, the data is temporarily stored in the circular buffer, and an interrupt signal is generated when it is half full or nearly full, effectively preventing data loss. Finally, the AXI4-Stream interface packages the data in the buffer according to a custom frame format and transmits the data directly to the designated circular queue in the DDR memory of the PS end through the AXI DMA controller. After the transmission is completed, a PL-PS interrupt is triggered to notify the PS end that new data has arrived.

[0078] On the processor system (PS) side, the Petalinux embedded operating system runs. A dedicated device driver was developed, which maps the data buffer filled with PL in DDR to user space through a memory mapping mechanism, allowing the application to directly read the latest sensor data frames. The main control program executes the discrimination and fusion algorithms in steps S2 to S3 with a timer interrupt cycle of 50 milliseconds. The Kalman filter algorithm based on the exponential moving average is implemented entirely in C language with floating-point operations, and the time taken for a single iteration is about 0.02 milliseconds. The inference process of the dual attention mechanism long short-term memory network model utilizes the ARM NEON instruction set to accelerate matrix multiplication, and combines symmetric quantization technology to convert the model weights from 32-bit floating-point to 8-bit integers, reducing the inference latency from the original 3.5 milliseconds to 0.8 milliseconds, meeting the real-time requirements within a 50-millisecond cycle. The target state estimate obtained by fusion is sent back to the host computer or flight control system through another UDP socket, and is also stored locally for closed-loop feedback.

[0079] Data interaction between PL and PS is completed entirely through the AXI bus. The measured interrupt latency from PL capturing UDP packets to data readiness in PS memory is about 2 microseconds. The end-to-end latency of the entire pipeline (from sensor data arriving at the network port to the output of the fusion result) is stable within 1.2 milliseconds, which is much less than the fusion cycle of 50 milliseconds, leaving sufficient computational margin.

[0080] Closed-loop adaptive iteration and feedback

[0081] To continuously improve tracking stability and fusion accuracy in complex scenarios, a closed-loop adaptive iterative mechanism is introduced. After each fusion, the optimal state estimate of the moving target is output. Simultaneously calculate the fusion error: ; In the formula, This represents the fusion error vector at the current moment; The magnitude of the fusion error The changes in the trajectory characteristic parameters (angular velocity, angular acceleration, curvature, etc.) and noise estimation at the current moment. and The data is packaged into a feedback vector and sent to the discrimination module.

[0082] Upon receiving feedback, the discrimination module dynamically adjusts its internal parameters. For example, if the fusion error is large for several consecutive cycles but the trajectory feature parameters remain below the first threshold, it indicates that the threshold may be set too leniently. In this case, the discrimination module automatically lowers the value of the first threshold (by 5% each time, but not exceeding the lower limit). Conversely, if frequent misjudgments occur (i.e., the actual trajectory is stable but misjudged as maneuvering), the second threshold is appropriately increased. Furthermore, the noise estimation window length also adaptively adjusts according to the fluctuation of the feedback error: when the error fluctuates drastically, the window length is reduced for a faster response; when the error is stable, the window length is increased to obtain a more robust estimate.

[0083] During the intelligent fusion model's activation (mode_t=1), the LSTM model is periodically incrementally fine-tuned using newly arrived observation data. Specifically, an online buffer with a capacity of 1000 samples is maintained. Whenever fusion is complete and deemed safe (i.e., the fusion error is less than a threshold), the current input sequence and its corresponding true state (approximated by the fusion result) are stored in the buffer. When the buffer is full, a background thread is triggered to perform three rounds of fine-tuning on the LSTM model using mini-batch gradient descent (batch size 32), with the learning rate set to one-tenth of the initial training rate (0.0001). The fine-tuning process is executed on an idle core of the ARM processor, thus not affecting the real-time performance of the main fusion task.

[0084] Guarantee Mechanism

[0085] To enhance robustness, this embodiment also incorporates multiple safeguard mechanisms. First, under a stable trajectory (mode_t=0), if the fusion residuals for 20 consecutive frames are all less than 0.01 (normalized units), it indicates that the Kalman filter has reached a steady state and the noise is low. At this point, the noise covariance during the automatic freezing process is... and measurement noise covariance The update, i.e., stopping EMA adaptation and fixing the current optimal value, is to stop EMA adaptation and fix the current optimal value. and This reduces the computational load. Secondly, under complex, highly maneuverable trajectories (…), the computational load is further reduced. If the confidence score of the dual-attention mechanism long short-term memory network model output is lower than 0.6 (confidence score is defined as the maximum probability of the Softmax output, or the reciprocal of the prediction variance for regression tasks), the model is considered unreliable. In this case, a temporary fallback to the improved Kalman filter fusion scheme is initiated, and model retraining is started asynchronously in the background: using recently accumulated online buffer data, a lightweight LSTM model is retrained or the original model is fine-tuned. The model is switched back after the accuracy of the new model is verified. Third, when the trajectory type changes from complex, high-maneuverability to a stationary trajectory, the Kalman filter needs to be reset because the state variable definitions of the two fusion modes may differ: the state covariance matrix... Setting it to 10 times the initial value allows the filter to converge quickly to the current true state, avoiding estimation bias caused by mismatch of historical information.

[0086] Example 2

[0087] This embodiment provides an electronic device, including a processor, a memory, and a bus. The processor and the memory are connected via the bus. The memory is used to store a set of program code, and the processor is used to call the program code stored in the memory to execute the multi-source heterogeneous sensor adaptive data fusion method for dynamic targets according to Embodiment 1.

[0088] Example 3

[0089] This embodiment provides a non-volatile computer storage medium storing computer-executable instructions that execute a multi-source heterogeneous sensor adaptive data fusion method for dynamic targets according to Embodiment 1.

[0090] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multi-source heterogeneous sensor adaptive data fusion method for dynamic targets, characterized in that, Includes the following steps: Step S1: Collect multi-source heterogeneous observation data of moving targets in real time using various heterogeneous sensors, preprocess the multi-source heterogeneous observation data, and form a unified time-series three-dimensional observation vector. Step S2: Extract motion features from the three-dimensional observation vector; compare the motion features with the preset discrimination threshold to determine the current motion state of the moving target; the motion state includes steady motion state and strong maneuvering motion state; Step S3: Based on the determined motion state of the moving target, an asymmetric hysteresis comparison strategy is used to adaptively switch the fusion scheme. In the stable motion state, an improved Kalman filter fusion scheme is activated. At non-initial moments, the optimal state estimate from the previous moment is fused with the current 3D observation vector to update the state and output the optimal state estimate of the moving target at the current moment. At the initial moment, the current observation vector is directly used to initialize the state estimate to start the filtering recursion. In the strong maneuvering motion state, a dual attention mechanism long short-term memory network fusion scheme is activated to fuse multiple consecutive frames of temporal 3D observation vectors and output the optimal state estimate of the moving target at the current moment.

2. The adaptive data fusion method for multi-source heterogeneous sensors oriented towards dynamic targets according to claim 1, characterized in that: An improved Kalman filter fusion scheme uses the target's optimal state estimate and corresponding error covariance from the previous time step as input, combined with the current three-dimensional observation vector, to iteratively solve for the optimal state estimate of the moving target at the current time step through the prediction and update equations of the Kalman filter. During this process, a second-order exponential moving average is used to dynamically optimize the process noise covariance and measurement noise covariance parameters, adjusting the Kalman gain in real time to adapt the filter to noise changes. A dual-attention mechanism long short-term memory network fusion scheme uses the temporal three-dimensional observation vectors from the current time step and previous consecutive frames as input. The temporal hidden state is extracted through a two-layer bidirectional long short-term memory network, and then the output of each time step is adaptively weighted and aggregated through a temporal attention module. Subsequently, the feature attention module performs channel weighting on each feature dimension, and finally, the optimal state estimate of the moving target at the current time step is mapped and output through a fully connected layer.

3. The adaptive data fusion method for multi-source heterogeneous sensors oriented towards dynamic targets according to claim 2, characterized in that: Multi-source heterogeneous observation data of moving targets include the azimuth, elevation, and range information of the moving targets; The three-dimensional observation vector at time t is represented as , express The azimuth angle value of the moving target after preprocessing at each time step. express The pitch angle value of the moving target after preprocessing at each moment. express The distance of the moving target after time-mapping is completed; This indicates transpose; the three-dimensional observation vectors at all times constitute a time-series three-dimensional observation vector dataset.

4. The adaptive data fusion method for multi-source heterogeneous sensors oriented towards dynamic targets according to claim 3, characterized in that: The motion features extracted from the three-dimensional observation vector in step S2 include angular velocity, angular acceleration, rate of change of linear velocity, and trajectory curvature; the preset discrimination thresholds include a low-maneuvering threshold and a high-maneuvering threshold. If all motion features are below the corresponding low maneuver threshold, the motion is classified as a steady motion state; if any motion feature is above the corresponding high maneuver threshold, the motion is classified as a strong maneuver motion state; if it is in between, it is also classified as a strong maneuver motion state.

5. The adaptive data fusion method for multi-source heterogeneous sensors oriented towards dynamic targets according to claim 4, characterized in that: Step S2 also includes online adaptive estimation of process noise covariance based on the time-seriesd three-dimensional observation vector dataset generated in step S1. and measurement noise covariance : Measuring noise covariance The estimate is based on the observation residuals within the sliding window: set the sliding window size. Calculate the first digit within the sliding window. Three-dimensional observation vectors at each fusion moment With the Predicted observations at each fusion time The deviation is calculated, and then its covariance matrix is ​​plotted to obtain the measurement noise covariance; where the predicted observation value is... Predicted value from the state at the previous time step Through the observation matrix The mapping yields the state prediction values ​​at non-initial times, derived from the optimal state estimate output in the previous fusion cycle via the state transition matrix. The state prediction value at the initial moment is obtained by recursively deriving from the target motion model; the state prediction value at the initial moment is obtained by converting the three-dimensional observation vector of the first frame into Cartesian coordinates and setting the velocity to zero, and is used as the starting point for subsequent recursion. Process noise covariance The estimation is based on the innovation vector of the Kalman filter. It uses a recursive formula for online updates; express The state prediction value at time 1.

6. The adaptive data fusion method for multi-source heterogeneous sensors oriented towards dynamic targets according to claim 5, characterized in that: In step S3, based on the determined motion state of the moving target, an asymmetric hysteresis comparison strategy is used to adaptively switch the fusion scheme, including: Set global toggle flag Used to mark the currently enabled fusion scheme: When the output judgment result of step S2 is a steady motion state, the global switching flag will be set. Set to 0 to enable the improved Kalman filter fusion scheme; when the output judgment result of step S2 is a strong maneuvering motion state, Set to 1 to enable the dual attention mechanism long short-term memory network fusion scheme to fuse consecutive multi-frame temporal three-dimensional observation vectors; Set an asymmetric hysteresis comparison strategy: when the S2 discrimination results for three consecutive fusion times are all in a strong maneuvering motion state, the global switching flag will be activated. Switch from 0 to 1; conversely, for five consecutive fusion time points, the global switching flag must be changed to indicate a stable motion state. Switch back from 1 to 0; The asymmetric hysteresis threshold value is dynamically adjusted based on the measurement noise covariance obtained in step S2: if the currently calculated measurement noise covariance... If the value exceeds the preset measurement noise covariance threshold, the asymmetric hysteresis threshold will be raised, increasing the requirement from 3 consecutive fusion times to 5 consecutive fusion times.

7. The adaptive data fusion method for multi-source heterogeneous sensors oriented towards dynamic targets according to claim 6, characterized in that: The improved Kalman filter fusion scheme includes: When in a stable motion state, an improved Kalman filter algorithm based on exponential moving average is executed. The inputs to the improved Kalman filter algorithm based on exponential moving average are: the optimal state estimate and its error covariance matrix at the previous time step, and the three-dimensional observation vector at the current time step; the output is the optimal state estimate at the current time step. and its error covariance matrix; In the first fusion cycle, the three-dimensional observation vectors obtained for the first time are used. Initialization: Convert the observed values ​​to initial positions in Cartesian coordinates, set the velocity to zero, and construct the initial state vector. Simultaneously, based on the sensor's nominal ranging and angular measurement accuracy, the error covariance matrix corresponding to the dimension of the initial state vector is initialized. Process noise covariance matrix and the measurement noise covariance matrix ; The iterative fusion process begins from the second fusion cycle: Process noise covariance in step S2 and measurement noise covariance Based on this, a second-order exponential moving average model is further used to dynamically smooth and optimize its diagonal elements, and adaptively adjust the Kalman gain. First, calculate the three-dimensional observation vector at the current moment. The residual vector between the observed and predicted values ​​is the innovation vector of the Kalman filter; the square norm of the residual vector is defined as the residual sequence; then, the residual sequence is smoothed using a second-order exponential moving average model based on the level equation and trend equation. Let the process noise covariance matrix at the current time be... Measurement noise covariance matrix ,in, This represents the process noise covariance at the previous time step. express The correction increment for the time-matter noise covariance. This represents the measurement noise covariance matrix at the previous time step. This represents the correction increment for the measurement noise covariance at the current moment; After completing the dynamic estimation of the process noise covariance matrix and the measurement noise covariance matrix, the Kalman filter prediction and update steps are performed: First, use the state transition matrix The optimal state estimate at the previous time step By performing recursion, we can obtain the state estimate at the current time. Simultaneously, based on the process noise covariance matrix obtained in step S2 at the current time... and the error covariance matrix of the previous time step Calculate the error covariance matrix at the current time. ; After completing the prediction, proceed to the update step; based on the measurement noise covariance matrix obtained in step S2 at the current time... and the error covariance matrix at the current time. Combined with the observation matrix Calculate Kalman gain ; Using the current three-dimensional observation vector The residual vector between the observed and predicted values, i.e., the innovation vector of the Kalman filter, represents the state estimate at the current time. Make corrections to obtain the optimal state estimate for the current time step. Simultaneously, update the error covariance matrix; The final output is the optimal state estimate at the current moment. .

8. The adaptive data fusion method for multi-source heterogeneous sensors oriented towards dynamic targets according to claim 7, characterized in that: The dual-attention mechanism long short-term memory network fusion scheme is implemented through a dual-attention mechanism long short-term memory network model, which is trained offline in advance. The trained dual-attention mechanism long short-term memory network model is used to fuse the continuous temporal three-dimensional observation vector sequence output by S1. The input layer receives the historical 3D observation vector sequence, with the input data for a single time step being a single frame of 3D observation vector. The historical 3D observation vector sequence is then fed into a two-layer bidirectional long short-term memory network. The specific structure of the two-layer bidirectional long short-term memory network is as follows: both the first and second layers use bidirectional long short-term memory networks. The two-layer bidirectional long short-term memory network processes the input historical 3D observation vector sequence in the order of time steps and extracts the hidden state sequence corresponding to all time steps. The hidden state sequence is weighted and aggregated using a time attention module: First, each hidden state sequence is mapped to a scalar energy value through a multilayer perceptron. Then, the attention weights are obtained by applying the Softmax function to the scalar energy values ​​at all time steps. Finally, the attention weights are multiplied by their corresponding hidden states and summed to obtain the temporal attention output vector. ; The temporal attention output vector is further adaptively weighted along the feature dimension through a feature attention module. First, global average pooling and global max pooling are performed on the temporal attention output vector to obtain global average pooling feature descriptors and global max pooling feature descriptors, respectively. Then, the global average pooling feature descriptors and global max pooling feature descriptors are fed into a shared two-layer MLP. The two branches of the two-layer MLP process the global average pooling feature descriptors and global max pooling feature descriptors, respectively. The outputs of the two branches of the two-layer MLP are added together and then passed through the Sigmoid activation function to obtain the feature weight vector. Finally, the temporal attention output vector is multiplied element-wise with the feature weight vector to obtain the final enhanced feature. The fully connected output layer maps the final enhanced features to the optimal state estimate.

9. An electronic device comprising a processor, a memory, and a bus, wherein the processor and the memory are connected via the bus, wherein, The memory is used to store a set of program codes, and the processor is used to call the program codes stored in the memory, characterized in that it executes the adaptive data fusion method for multi-source heterogeneous sensors oriented towards dynamic targets as described in any one of claims 1-8.

10. A non-volatile computer storage medium storing computer-executable instructions, characterized in that, The computer can execute instructions to perform the adaptive data fusion method for multi-source heterogeneous sensors oriented towards dynamic targets as described in any one of claims 1-8.

Images