A state parameter reconstruction method based on multi-source heterogeneous sensor time sequence fusion
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF SCI & TECH
- Filing Date
- 2026-04-22
- Publication Date
- 2026-08-07
AI Technical Summary
[0007]本发明针对现有技术在多源异构传感器数据融合中存在的精度不足、时序信息利用有限以及难以适应不同传感器物理特性差异的问题，提供一种基于多源异构传感器时序融合的状态参数重构方法
[0036]（1）与现有卡尔曼滤波、扩展卡尔曼滤波等传统状态估计方法相比，本发明提出针对不同物理特性传感器数据的分支特征建模，能够对异构传感器差异化特征进行独立提取与处理，有效避免不同类型数据间的相互干扰，提升特征表征的针对性与准确性。
Smart Images
Figure CN122528094A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of information processing technology, specifically relating to a method for reconstructing state parameters based on time-series fusion of multi-source heterogeneous sensors, and more particularly to a method for achieving high-precision reconstruction of target state parameters by performing branch modeling and feature fusion on time-series data of multi-source heterogeneous sensors with different physical characteristics. Background Technology
[0002] In engineering applications such as target detection, target tracking, and intelligent sensing, it is often necessary to acquire the target's spatial state parameters, such as pitch angle, azimuth angle, and distance, in real time through sensors. These state parameters are the key basis for target localization, trajectory analysis, and control decisions, and their measurement accuracy directly affects the overall performance and reliability of the system.
[0003] In practical engineering, high-precision sensors are difficult to deploy widely due to limitations in system cost, size, power consumption, and installation environment. Instead, low-precision sensors, which are inexpensive and simple in structure, are typically used. However, low-precision sensors are susceptible to environmental interference and inherent performance limitations during measurement, resulting in output data that generally suffers from large measurement errors, poor stability, and insufficient accuracy, making it difficult to directly meet the demands of high-precision applications.
[0004] To improve the measurement accuracy of target state parameters, existing technologies typically employ multi-sensor data fusion methods. In their paper "Approaches to Multisensor Data Fusion in Target Tracking," Smith D and Singh S systematically analyzed multi-sensor data fusion techniques for target tracking using multi-sensor Kalman filtering, weighted averaging, and particle filtering. They pointed out that existing methods struggle to fully consider sensor heterogeneity and temporal characteristics when processing low-precision sensor data, leading to insufficient accuracy and stability of the fusion results. In their paper "Data Fusion for Target Tracking in Wireless Sensor Networks Using Quantized Innovations and Kalman Filtering," Xu J, Li J, and Xu S proposed a distributed multi-sensor fusion method based on quantized innovations and Kalman filtering for target tracking in wireless sensor networks. They analyzed the impact of the fusion strategy on target state estimation and pointed out that when sensor data noise is high or sampling is inconsistent, traditional Kalman filtering methods struggle to guarantee fusion accuracy and system robustness.
[0005] However, the aforementioned traditional fusion methods still have significant limitations. On the one hand, most of these methods rely on fixed mathematical models or preset weight parameters, making it difficult to fully characterize the differences in physical measurement mechanisms, error distributions, and dynamic responses among different types of sensors. On the other hand, when sensor data has high noise, inconsistent sampling frequencies, missing data, or low measurement accuracy, traditional methods struggle to achieve stable and reliable fusion results. Furthermore, target state parameters typically exhibit significant temporal dynamic changes, while existing methods often employ static models or simple filtering strategies, making it difficult to fully exploit the temporal correlations of historical data or effectively utilize the complementary information between different types of sensor data, thus limiting fusion accuracy and system robustness.
[0006] In complex application environments, different types of sensors typically exhibit significant differences in measurement mechanisms and data characteristics. For example, angle sensors and distance sensors differ markedly in data distribution, noise characteristics, and dynamic response. A unified model can easily lead to mutual interference between different types of features, further affecting the accuracy of state parameter reconstruction. Therefore, designing a fusion processing method that can fully utilize complementary information from different sensors, effectively suppress noise, and achieve high-precision state parameter reconstruction, without significantly increasing hardware costs and system complexity, remains a crucial technical problem that urgently needs to be solved in this field. Summary of the Invention
[0007] This invention addresses the problems of insufficient accuracy, limited utilization of temporal information, and difficulty in adapting to the differences in physical characteristics of different sensors in existing technologies for multi-source heterogeneous sensor data fusion. It provides a state parameter reconstruction method based on temporal fusion of multi-source heterogeneous sensors.
[0008] The core idea of this method is as follows: First, based on the physical characteristics and data features of different sensors, the sensor data is branched and modeled to extract the feature information of each type of sensor. Then, the temporal features of each branch are modeled using a Long Short-Term Memory (LSTM) network to fully capture the dynamic dependence of the target state parameters over time. Finally, a Transformer based on a self-attention mechanism is used to correlate and fuse the features of each branch, fully explore the complementary information between multi-source heterogeneous sensors, and achieve high-precision reconstruction of the target state parameters.
[0009] The method of this invention can achieve equivalent high-precision measurement through algorithmic means without improving the accuracy of sensor hardware, significantly improving the accuracy of target state parameter estimation and system robustness, and is applicable to state monitoring and control needs in various types of sensors, complex environments and dynamic target scenarios.
[0010] The technical solution for achieving this invention is as follows: a method for reconstructing state parameters based on time-series fusion of multi-source heterogeneous sensors, comprising the following steps:
[0011] Step 1: Collect raw time-series data of the same target using multi-source heterogeneous sensors, perform standardization preprocessing on the raw time-series data, and divide it into model training samples and test samples, as follows:
[0012] S11. Obtain raw time-series data obtained by at least two heterogeneous sensors from collaborative observation of the same target. The raw time-series data includes at least timestamps, target angle measurements, and target distance measurements. The sampling frequencies and time bases of the different sensors may be from different sources.
[0013] S12. Establish a timeline based on the system's unified clock, set a target time resolution, and use linear interpolation to map the original time-series data to a unified time node to achieve time synchronization of multi-source data. Use a statistical threshold filtering method to correct abnormal measurement data, ensuring that the integrity of the compensated data is no less than 99.9%, thus obtaining time-synchronized multi-source data.
[0014] S13. Construct a multi-dimensional standardized time series from the time-synchronized multi-source data. See equation (1):
[0015] (1),
[0016] In the formula, For timestamps, This is the set of pitch angle measurements from all sensors. This is the set of azimuth angle measurements from all sensors. This is the set of distance measurements from all sensors.
[0017] S14. Set up a sliding window mechanism to fix the window length. Sliding step size 1 for multidimensional normalized time series Slice the image to generate a dimension of 1. The model input samples, where For batch size, This represents the total dimension of the standardized time series. The standardized samples are divided into training and testing samples in a 7:3 ratio.
[0018] Proceed to step 2.
[0019] Step 2: Establish an LSTM-Transformer cascaded network for state parameter reconstruction. The network consists of three parallel independent LSTM encoding branches, and a feature concatenation module, a position encoding module, a Transformer encoder, and a task decoupling dual-branch regression head, which are sequentially connected after the three LSTM encoding branches, as detailed below:
[0020] S21. Establish three parallel and independent LSTM coding branches: angle feature coding branch, distance feature coding branch, and time evolution feature coding branch, to extract the corresponding angle time-series feature sequences. Distance time series feature sequences and time series feature sequences .
[0021] S22. The feature concatenation module concatenates the temporal feature sequences output from the three LSTM encoding branches along the feature dimension to obtain a fused temporal feature matrix. .
[0022] S23, The location encoding module fuses the temporal feature matrix. Location information enhancement is performed to obtain fused features with injected location information. .
[0023] S24. The Transformer encoder models the global dependencies between different time steps and different feature dimensions through a self-attention mechanism, resulting in a globally enhanced temporal feature matrix. .
[0024] S25, Task decoupling dual-branch regression head enhances global temporal feature matrix Perform angle reconstruction and distance reconstruction separately.
[0025] Proceed to step 3.
[0026] Step 3: Input the training samples obtained in Step 1 into the LSTM-Transformer cascaded network established in Step 2, set the loss function and optimizer, and perform end-to-end iterative training on the network to obtain the converged state parameter reconstruction model, as follows:
[0027] S31. Construct a weighted mean square error loss function Loss to balance the reconstruction accuracy of angle and distance parameters, as shown in equation (15):
[0028] (15),
[0029] S32. Use the AdamW optimizer to iteratively update the network parameters, with weight decay set to... Set the batch size to B, the number of training rounds to N, and the learning rate to be dynamically adjusted using a cosine annealing strategy until the loss function converges, thus obtaining the reconstructed model of the state parameters after training.
[0030] Proceed to step 4.
[0031] Step 4: Input the test samples obtained in Step 1 into the state parameter reconstruction model trained in Step 3, output the target state parameter reconstruction result, and complete the test and verification of model performance, as follows:
[0032] S41. Input the test sample into the trained state parameter reconstruction model, and output the standardized reconstruction values of pitch angle, azimuth angle and distance corresponding to the test sample.
[0033] S42. Restore the standardized reconstructed values to their corresponding physical dimensions, including pitch and azimuth angles to degrees and distances to meters, ensuring that the timestamps of the output state parameters are completely consistent with the time base of the original observation data, and obtain the final state parameter reconstruction results.
[0034] S43. Using root mean square error (RMSE) and mean absolute error (MSE) as evaluation metrics, compare the reconstruction results with the actual labeled values to complete the test and verification of model performance.
[0035] Compared with the prior art, the advantages of this invention are:
[0036] (1) Compared with existing traditional state estimation methods such as Kalman filtering and extended Kalman filtering, this invention proposes branch feature modeling for sensor data with different physical characteristics, which can independently extract and process the differentiated features of heterogeneous sensors, effectively avoid mutual interference between different types of data, and improve the pertinence and accuracy of feature representation.
[0037] (2) Compared with existing fusion methods that use a single LSTM or simple feature splicing, this invention adopts a hybrid network structure of LSTM and Transformer. It leverages the complementary advantages of LSTM in capturing long-term temporal dependencies and Transformer in modeling global correlation features of multiple sensors to achieve deep correlation fusion of multi-branch temporal features, fully explore the complementary information of multi-source data and historical temporal information, and significantly enhance the ability to suppress noise and abnormal data and the robustness of the model.
[0038] (3) Without increasing the cost of sensor hardware, the present invention achieves high-precision reconstruction of target state parameters, and the output effect is equivalent to that of a high-precision sensor, which has high engineering application value.
[0039] (4) The present invention has strong adaptability to sensor types and quantities, and is suitable for various low-precision heterogeneous sensor combination scenarios, with good versatility and scalability. Attached Figure Description
[0040] Figure 1 This is a schematic diagram of the state parameter reconstruction method based on multi-source heterogeneous sensors according to the present invention.
[0041] Figure 2 This is a diagram of the LSTM-Transformer cascaded state parameter reconstruction network architecture of the present invention.
[0042] Figure 3 This is a schematic diagram comparing the state parameter reconstruction results of an embodiment of the present invention.
[0043] Figure 4 This is a schematic diagram comparing the state parameter residuals of the embodiment of the present invention with the original sensor data.
[0044] Figure 5 This is a comparison chart showing the accuracy improvement between the method of this invention and the traditional method. Detailed Implementation
[0045] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit its scope of protection. All equivalent modifications, substitutions, and improvements that can be obtained by those skilled in the art without creative effort based on the core concept of the present invention fall within the scope of protection of the present invention.
[0046] This embodiment addresses the scenario of high-accuracy reconstruction of three types of state parameters of a flight target: pitch angle, azimuth angle, and range. It implements the state parameter reconstruction method based on time-series fusion of multi-source heterogeneous sensors proposed in this invention to verify the technical effect of this invention.
[0047] The hardware operating environment of this embodiment includes: a multi-source heterogeneous sensor group, a synchronous data acquisition module, and a data processing terminal; wherein, the multi-source heterogeneous sensor group is used to conduct collaborative observation of the same flight target and output raw measurement data, the synchronous data acquisition module is used to complete the synchronous acquisition and transmission of multi-source data, and the data processing terminal is used to run the method flow of the present invention.
[0048] Optionally, the multi-source heterogeneous sensor group in this embodiment includes a vortex sensor (hereinafter referred to as sensor 1) and an electrostatic sensor (hereinafter referred to as sensor 2). Those skilled in the art can replace it with other types of heterogeneous sensor combinations according to the actual scenario, and all of them are within the protection scope of this invention.
[0049] Combination Figure 1A method for reconstructing state parameters based on multi-source heterogeneous sensors includes the following steps:
[0050] Step 1: Collect raw time-series data of the same target using multi-source heterogeneous sensors, perform standardization preprocessing on the raw time-series data, and divide it into model training samples and test samples, as follows:
[0051] S11. Obtain raw time-series data obtained by at least two heterogeneous sensors from collaborative observation of the same target. The raw time-series data includes at least timestamps, target angle measurements, and target distance measurements. The sampling frequencies and time bases of the different sensors may be from different sources.
[0052] S12. Establish a timeline based on the system's unified clock, set a target time resolution, and use linear interpolation to map the original time-series data to a unified time node to achieve time synchronization of multi-source data. Use a statistical threshold filtering method to correct abnormal measurement data, ensuring that the integrity of the compensated data is no less than 99.9%, thus obtaining time-synchronized multi-source data.
[0053] S13. Construct a 6-dimensional normalized time series from the preprocessed data:
[0054] ,
[0055] In the formula, t is the timestamp. These are the elevation angle, azimuth angle, and distance measurements of sensor 1, respectively. These are the elevation and azimuth measurements of sensor 2, respectively.
[0056] S14. Set up a sliding window mechanism to fix the window length. Sliding step size 1-to-6 dimensional normalized time series Slice the image to generate a dimension of 1. The model input samples, where the batch size The standardized samples were divided into training samples and test samples in a 7:3 ratio.
[0057] Proceed to step 2.
[0058] Step 2: Establish an LSTM-Transformer cascaded network for state parameter reconstruction. Network details are as follows: Figure 2 As shown, the network sequentially includes three parallel and independent LSTM encoding branches, and a feature concatenation module, a position encoding module, a Transformer encoder, and a task decoupling dual-branch regression head, which are sequentially connected after the three LSTM encoding branches, as detailed below:
[0059] S21. Establish three parallel and independent LSTM coding branches: angle feature coding branch, distance feature coding branch, and time evolution feature coding branch, to extract the corresponding angle time-series feature sequences. Distance time series feature sequences and time series feature sequences .
[0060] All three LSTM encoding branches use the same gating logic. The accurate capture of long-term time-series dependencies is achieved through the collaborative work of the forget gate, input gate, and output gate. The core calculation process is shown in equations (2)-(7):
[0061] (2),
[0062] (3),
[0063] (4),
[0064] (5),
[0065] (6),
[0066] (7),
[0067] In the formula: It is the Sigmoid activation function. It is the element-level Hadamard product. These are the outputs of the forget gate, input gate, and output gate, respectively. Input features for the current time step. This is the hidden state from the previous moment. This represents the current state of the cell. This represents the current state of the candidate cells. Learnable weights that can be trained independently for each branch. Learnable biases for independent training of each branch.
[0068] The specific configuration of the three LSTM encoding branches is as follows:
[0069] Angular feature encoding branch: Input angle-related terms The input dimension is 4. The LSTM encoder is configured with 128 hidden layers, 2 layers, and a dropout rate of 0.25. The output is an angular temporal feature sequence. It is used to extract the local temporal correlation and short-term dynamic change patterns of angular features.
[0070] Distance feature encoding branch: Input distance-related terms The input dimension is 1. The LSTM encoder is configured with 64 hidden layers, 2 layers, and a dropout rate of 0.25. The output is a distance temporal feature sequence. It is used to extract the temporal variation trend and local fluctuation information of distance features.
[0071] Temporal evolution feature encoding branch: Input time term The input dimension is 1. The LSTM encoder is configured with 32 hidden layers, 2 layers, and a dropout rate of 0.25. The output is a temporal series feature sequence. It is used to capture the periodic and trend evolution characteristics of time series.
[0072] S22. The feature concatenation module concatenates the temporal feature sequences output from the three LSTM encoding branches along the feature dimension to obtain a fused temporal feature matrix. .
[0073] The temporal features output from the three branches are concatenated along the feature dimension to obtain the fused feature matrix, as shown in Equation (8):
[0074] (8),
[0075] Total feature dimension after splicing ,Keep With the temporal dimension unchanged, the dimension of the output fused feature tensor is... .
[0076] S23, The location encoding module fuses the temporal feature matrix. Location information enhancement is performed to obtain fused features with injected location information. .
[0077] The position encoding module uses sinusoidal position encoding to enhance the position information of the fused temporal feature matrix, injecting temporal position information into the Transformer. The calculation process is shown in equations (9)-(10):
[0078] (9),
[0079] (10),
[0080] In the formula, This refers to the position index in the time series. Indexed by feature dimensions.
[0081] Encode the position With fusion features By adding elements one by one, the position information is injected, and the fused feature of the injected position information is obtained. .
[0082] S24. The Transformer encoder fuses features based on injected positional information through a self-attention mechanism. Modeling the global dependencies between different time steps and different feature dimensions yields a globally enhanced temporal feature matrix. .
[0083] Fusion features that inject location information Input a Transformer encoder and model the global dependencies between different time steps and different feature dimensions through a self-attention mechanism.
[0084] The core of the Transformer encoder is the scaling dot product attention mechanism. :
[0085] (11),
[0086] In the formula, These are query, key, and value matrices, respectively. The dimension of the key vector, divided by This is used to avoid gradient vanishing due to excessively large inner product values. This indicates transpose. It is a normalized exponential function.
[0087] Employing a multi-head self-attention mechanism The extended feature representation space is calculated as shown in equation (12):
[0088] (12),
[0089] In the formula, For the number of attention heads, This indicates a splicing operation. For single-head attention output, , All are learnable linear projection matrices.
[0090] The Transformer encoder in this unit is configured with: 3 encoding layers and a feedforward network intermediate layer dimension of [missing information]. With a dropout rate of 0.25, residual connections and layer normalization are introduced to stabilize the training process, outputting a global augmented feature tensor. .
[0091] S25, Task decoupling dual-branch regression head enhances global temporal feature matrix Perform angle reconstruction and distance reconstruction separately.
[0092] Angle reconstruction branch: Employs independent multi-head self-attention modules (4 heads) for... Angle feature enhancement is performed to obtain the angle-enhanced time series. ,Pick Features of the last time step The pitch and azimuth angles are reconstructed using a two-layer fully connected network. The calculation process is shown in equation (13):
[0093] (13),
[0094] In the formula, the pitch angle Azimuth Reconstructed value , Learnable weights for reconstructing branches based on perspective. Learnable biases for angle reconstruction branches It is a linear rectification activation function.
[0095] Distance reconstruction branch: Employs independent multi-head self-attention modules (2 heads) to globally enhance the temporal feature matrix. Distance feature enhancement is performed to obtain distance-enhanced time series sequences. ,Pick Features of the last time step , and the global enhanced temporal feature matrix characteristics of the last moment Weighted fusion is performed, as shown in equation (14):
[0096] (14),
[0097] In the formula, The preset weighting coefficient is 0.6. The fused features... Input a two-layer fully connected network, reconstruct the distance, and output the distance. Reconstructed value .
[0098] Step 3: Input the training samples obtained in Step 1 into the LSTM-Transformer cascaded network established in Step 2, set the loss function and optimizer, and perform end-to-end iterative training on the network to obtain the converged state parameter reconstruction model.
[0099] S31. Construct a weighted mean square error loss function to balance the reconstruction accuracy of angle and distance parameters, as shown in equation (15):
[0100] (15),
[0101] In the formula, The true values of the state parameters and the weighting coefficients. , , Let be the mean square error function.
[0102] S32. Use the AdamW optimizer to iteratively update the network parameters, with weight decay set to... The training epochs are set to N=100 epochs, and the learning rate is dynamically adjusted using a cosine annealing strategy until the loss function converges, thus obtaining the reconstructed state parameter model after training.
[0103] Step 4: Input the test samples obtained in Step 1 into the state parameter reconstruction model trained in Step 3, output the target state parameter reconstruction result, and complete the test and verification of model performance.
[0104] S41. Input the test sample into the trained state parameter reconstruction model. The network sequentially completes multi-branch feature extraction, global fusion modeling and task decoupling regression, and outputs the standardized reconstruction values of pitch angle, azimuth angle and distance corresponding to the test sample.
[0105] S42. Restore the standardized reconstructed values to their corresponding physical dimensions, including pitch and azimuth angles to degrees (°) and distance to meters (m), ensuring that the timestamps of the output state parameters are completely consistent with the time base of the original observation data, thus obtaining the final target state parameter reconstruction result. The output sequence format is as follows: .
[0106] S43. Using root mean square error (RMSE) and mean absolute error (MAE) as evaluation metrics, the reconstruction results of this embodiment are compared with the true labeled values, and also compared with the original sensor observation results and the traditional Kalman filter fusion method to verify the technical effect of the present invention. The test results are as follows:
[0107] Figure 3 The figures present the original heterogeneous sensor observations, the reconstruction results obtained by the method of this invention, and the corresponding reference ground truth curves. As can be seen from the figures, the state parameter curves output by the method of this invention closely match the reference ground truth, while the original heterogeneous sensor observations show significant deviations. This comparative result demonstrates that the multi-source feature branch modeling and fusion processing method described in this invention can significantly improve the accuracy of state parameter estimation, achieving an output effect equivalent to that of a high-precision sensor.
[0108] Figure 4 The comparison shows the error distribution of the original heterogeneous sensor observation results and the error distribution of the output results of the method of the present invention. It can be clearly seen that the residual amplitude of the method of the present invention is significantly smaller than that of the original sensor data residual, and the error fluctuation is more stable. This indicates that the method of the present invention can effectively suppress observation noise and measurement error and significantly improve the reconstruction accuracy of target state parameters without increasing the accuracy of sensor hardware.
[0109] Under the same test data conditions, the method of the present invention achieves a significant improvement in accuracy across all state parameters compared to the original heterogeneous sensor, as detailed below:
[0110] (1) Pitch angle reconstruction accuracy:
[0111] The root mean square error was reduced from 3.8944° (sensor 2) and 1.7744° (sensor 1) of the original sensor to 0.9424°.
[0112] The mean absolute error decreased from 3.2429° (sensor 2) and 1.4718° (sensor 1) to 0.5920°.
[0113] (2) Azimuth angle reconstruction accuracy:
[0114] The root mean square error was reduced from 3.7041° (sensor 2) and 0.9438° (sensor 1) to 0.5101°.
[0115] The mean absolute error decreased from 3.0532° (sensor 2) and 0.7832° (sensor 1) to 0.3919°.
[0116] (3) Distance reconstruction accuracy:
[0117] The root mean square error decreased from 0.4465m (sensor 1) to 0.1438m.
[0118] The mean absolute error decreased from 0.3668m (sensor 1) to 0.1082m.
[0119] In summary, the method of this invention significantly outperforms the direct observation results of any single low-precision sensor in all three key state parameters: pitch angle, azimuth angle, and range. Compared to a superior sensor benchmark, the method of this invention achieves an accuracy improvement of approximately 46.9% (pitch angle), 45.9% (azimuth angle), and 67.8% (range) at the root mean square error level, and 59.8%, 49.9%, and 70.5% at the mean absolute error level, respectively.
[0120] like Figure 5 As shown, based on the average of the relative improvement ratios of each indicator, the method of the present invention improves the overall accuracy of the root mean square error by approximately 53.5% and the overall accuracy of the average absolute error by approximately 60.1%, which fully verifies the significant technical effect of the present invention in achieving high-precision state reconstruction through multi-source data fusion without improving the accuracy of sensor hardware.
[0121] Under the same data conditions, using the traditional method (Kalman filter fusion method), the overall accuracy improvement of the root mean square error is approximately 24.5%, and the overall accuracy improvement of the mean absolute error is approximately 30.2%. The comparison of these two indicators shows that the method of this invention improves the error suppression capability by nearly 1 times compared with the traditional method, and can more effectively fuse observation data from multiple low-precision sensors into high-precision output results, thereby significantly improving the accuracy of target state estimation.
[0122] This embodiment fully demonstrates the technical advantages of the present invention in multi-source heterogeneous sensor data fusion and high-precision state reconstruction, verifying the effectiveness and practicality of the described technical solution. It should be noted that the specific network structure, parameter configuration, and algorithm details mentioned in this embodiment are merely one of many possible implementations of the present invention and are not intended to limit the invention. Those skilled in the art, guided by the concept of this invention, may employ other equivalent technical means to achieve the same function, all of which fall within the protection scope of this invention.
Claims
1. A method for reconstructing state parameters based on time-series fusion of multi-source heterogeneous sensors, characterized in that, The steps are as follows: Step 1: Collect raw time-series data of the same target using multi-source heterogeneous sensors, perform standardization preprocessing on the raw time-series data, divide it into model training samples and test samples, and proceed to Step 2. Step 2: Establish an LSTM-Transformer cascaded network for state parameter reconstruction. The network includes three parallel independent LSTM encoding branches, and a feature splicing module, a position encoding module, a Transformer encoder, and a task decoupling dual-branch regression head that are sequentially connected after the three LSTM encoding branches. Proceed to Step 3. Step 3: Input the training samples obtained in Step 1 into the LSTM-Transformer cascaded network established in Step 2, set the loss function and optimizer, perform end-to-end iterative training on the network, obtain the converged state parameter reconstruction model, and proceed to Step 4. Step 4: Input the test samples obtained in Step 1 into the state parameter reconstruction model trained in Step 3, output the target state parameter reconstruction result, and complete the test and verification of model performance.
2. The state parameter reconstruction method based on time-series fusion of multi-source heterogeneous sensors according to claim 1, characterized in that, Step 1 is detailed as follows: S11. Obtain raw time-series data obtained by at least two heterogeneous sensors from collaborative observation of the same target. The raw time-series data includes at least timestamps, target angle measurements, and target distance measurements. The sampling frequencies and time bases of the different sensors may be from different sources. S12. Establish a time axis based on the system's unified clock, set the target time resolution, and use linear interpolation to map the original time series data to a unified time node to achieve time synchronization of multi-source data; use a statistical threshold filtering method to correct abnormal measurement data so that the integrity of the compensated data is not less than 99.9%, and obtain multi-source data after time synchronization. S13. Construct a multi-dimensional standardized time series from the time-synchronized multi-source data. See equation (1): (1)， In the formula, For timestamps, This is the set of pitch angle measurements from all sensors. This is the set of azimuth angle measurements from all sensors. This is the set of distance measurements from all sensors. S14. Set up a sliding window mechanism to fix the window length. Sliding step size 1 for multidimensional normalized time series Slice the image to generate a dimension of 1. The model input samples, where For batch size, The total dimension of the standardized time series is defined; the standardized samples are divided into training samples and test samples in a 7:3 ratio.
3. The state parameter reconstruction method based on time-series fusion of multi-source heterogeneous sensors according to claim 2, characterized in that, Step 2, establish the LSTM-Transformer cascaded network for state parameter reconstruction, as follows: S21. Establish three parallel and independent LSTM coding branches: angle feature coding branch, distance feature coding branch, and time evolution feature coding branch, to extract the corresponding angle time-series feature sequences. Distance time series feature sequences and time series feature sequences ; S22. The feature concatenation module concatenates the temporal feature sequences output from the three LSTM encoding branches along the feature dimension to obtain a fused temporal feature matrix. ; S23, The location encoding module fuses the temporal feature matrix. Location information enhancement is performed to obtain fused features with injected location information. ; S24. The Transformer encoder models the global dependencies between different time steps and different feature dimensions through a self-attention mechanism, resulting in a globally enhanced temporal feature matrix. ; S25, Task decoupling dual-branch regression head enhances global temporal feature matrix Perform angle reconstruction and distance reconstruction separately.
4. The state parameter reconstruction method based on time-series fusion of multi-source heterogeneous sensors according to claim 3, characterized in that, S21 is as follows: All three LSTM encoding branches use the same gating logic. The accurate capture of long-term time-series dependencies is achieved through the collaborative work of the forget gate, input gate, and output gate. The core calculation process is shown in equations (2)-(7): (2)， (3)， (4)， (5)， (6)， (7)， In the formula: It is the Sigmoid activation function. This is the element-level Hadama product; These are the outputs of the forget gate, input gate, and output gate, respectively. Input features for the current time step. This is the hidden state from the previous moment. This represents the current state of the cell. This represents the cell state at the previous moment. This represents the current state of the candidate cells. Learnable weights that can be trained independently for each branch. Learnable biases for independent training of each branch; The specific configuration of the three LSTM encoding branches is as follows: Angular feature encoding branch: Input angle-related terms Extracting local temporal correlations and short-term dynamic changes of angular features, the output dimension is... Angle time series feature sequence ,in Let this be the hidden layer dimension of this branch of LSTM; Distance feature encoding branch: Input distance-related terms Extract the temporal variation trend and local fluctuation information of distance features, with an output dimension of Distance time series feature sequences ,in Let this be the hidden layer dimension of this branch of LSTM; Temporal evolution feature encoding branch: Input time term It captures the periodic and trend evolution characteristics of time series, with an output dimension of Time-series feature sequences ,in Let be the hidden layer dimension of this branch of LSTM.
5. The state parameter reconstruction method based on time-series fusion of multi-source heterogeneous sensors according to claim 4, characterized in that, In S22, the feature concatenation module concatenates the temporal feature sequences output from the three LSTM encoding branches along the feature dimension to obtain a fused temporal feature matrix. : (8)， Total feature dimension after splicing ,Keep With the temporal dimension unchanged, the dimension of the output fused feature tensor is... .
6. The state parameter reconstruction method based on time-series fusion of multi-source heterogeneous sensors according to claim 5, characterized in that, In S23, the position encoding module uses sinusoidal position encoding to enhance the position information of the fused temporal feature matrix, injecting temporal position information into the Transformer: (9)， (10)， In the formula, This refers to the position index in the time series. Indexed by feature dimension; Encode the position With fused temporal feature matrix By adding elements one by one, the position information is injected, and the fused feature of the injected position information is obtained. .
7. The state parameter reconstruction method based on time-series fusion of multi-source heterogeneous sensors according to claim 6, characterized in that, In S24, the Transformer encoder models the global dependencies between different time steps and different feature dimensions through a self-attention mechanism. Its core is the scaled dot product attention mechanism, as detailed below: Fusion features that inject location information Input a Transformer encoder and model the global dependencies between different time steps and different feature dimensions through a self-attention mechanism; The core of the Transformer encoder is the scaling dot product attention mechanism. : (11)， In the formula, These are query, key, and value matrices, respectively. The dimension of the key vector, divided by This is used to prevent gradient vanishing due to excessively large inner product values; Indicates transpose; It is a normalized exponential function; Employing a multi-head self-attention mechanism The extended feature representation space is calculated as shown in equation (12): (12)， In the formula, For the number of attention heads, Indicates a splicing operation; For single-head attention output, , All are learnable linear projection matrices; By introducing residual connections and layer normalization to stabilize the training process, the final output dimension is... Global Enhanced Temporal Feature Matrix .
8. The state parameter reconstruction method based on time-series fusion of multi-source heterogeneous sensors according to claim 7, characterized in that, In S25, the task-decoupled dual-branch regression head includes an angle reconstruction branch and a distance reconstruction branch, as detailed below: Angle reconstruction branch: Employs an independent multi-head self-attention module for... Angle feature enhancement is performed to obtain the angle-enhanced time series. ,Pick Features of the last time step The pitch and azimuth angles are reconstructed using a two-layer fully connected network. The calculation process is shown in equation (13): (13)， In the formula, the pitch angle Azimuth Reconstructed value , Learnable weights for reconstructing branches based on perspective. Learnable biases for angle reconstruction branches It is a linear rectification activation function; Distance reconstruction branch: Employs independent multi-head self-attention modules to globally enhance the temporal feature matrix. Distance feature enhancement is performed to obtain distance-enhanced time series sequences. ,Pick Features of the last time step , and the global enhanced temporal feature matrix characteristics of the last moment Weighted fusion is performed, as shown in equation (14): (14)， In the formula, The preset weighting coefficients range from 0 to 1; the fused features Input a two-layer fully connected network, reconstruct the distance, and output the distance. Reconstructed value .
9. The state parameter reconstruction method based on time-series fusion of multi-source heterogeneous sensors according to claim 8, characterized in that, Step 3, as follows: S31. Construct a weighted mean square error loss function Loss to balance the reconstruction accuracy of angle and distance parameters, as shown in equation (15): (15)， In the formula, These are the actual labeled values of the state parameters. These are the loss weight coefficients for the angle and distance parameters, respectively. Let the mean square error function be used. S32. Use the AdamW optimizer to iteratively update the network parameters, with weight decay set to... Set the batch size to B, the number of training rounds to N, and the learning rate to be dynamically adjusted using a cosine annealing strategy until the loss function converges, thus obtaining the reconstructed model of the state parameters after training.
10. A method for reconstructing state parameters based on time-series fusion of multi-source heterogeneous sensors according to claim 9, characterized in that, Step 4, as follows: S41. Input the test sample into the trained state parameter reconstruction model and output the standardized reconstruction values of pitch angle, azimuth angle and distance corresponding to the test sample. S42. Restore the standardized reconstructed values to their corresponding physical dimensions, including pitch and azimuth angles to degrees and distances to meters, ensuring that the timestamps of the output state parameters are completely consistent with the time base of the original observation data, and obtain the final state parameter reconstruction results. S43. Using root mean square error (RMSE) and mean absolute error (MSE) as evaluation metrics, compare the reconstruction results with the actual labeled values to complete the test and verification of model performance.