Maneuvering target tracking method based on MHS and GRU fusion network

Abstract

The present invention discloses a mobile target tracking method based on a fusion network of MHS and GRU, belongs to the technical field of mobile target tracking, and is used for mobile target tracking, including inputting a training set into a fusion network based on MHS and GRU for iterative training, inputting a verification set into a fusion network based on MHS and GRU after each iteration, returning to iterative training if the accuracy requirement is not met, and inputting a test set into a fusion network based on MHS and GRU if the accuracy requirement is met, obtaining a fusion gain FG, and inputting the local filter of each sensor together with the fusion gain FG into a mobile target tracking model to obtain a mobile target tracking result. The present invention automatically corrects and adapts to errors in the process of time registration, effectively reduces error accumulation, and achieves improvements in tracking accuracy, convergence speed, and stability; using a relatively compact RNN, relatively small data sets can be used for training.

Description

Technical Field

[0001] The invention discloses a mobile target tracking method based on a fusion network of MHS and GRU, belonging to the technical field of mobile target tracking. Background Art

[0002] In the autonomous driving system, sensors are key components for perceiving the external environment. Their performance directly affects the safety and reliability of autonomous driving vehicles. Different types of sensors have their own unique advantages. For example, radar sensors have the advantages of high ranging accuracy and high data rate, and can work stably in various weather conditions; laser sensors can provide detailed environmental information due to their high angular resolution; infrared sensors can work in day and night and low light conditions, can detect the thermal radiation of objects, identify temperature differences, and provide high-resolution target thermal image information. Therefore, the fusion of these multi-source heterogeneous sensor data can make full use of the advantages of various types of sensors and improve the maneuvering target detection and tracking performance of the autonomous driving system. In the multi-source heterogeneous sensor maneuvering target fusion tracking, there are usually two significant problems: time asynchrony problem and model mismatch problem.

[0003] The time asynchrony problem is caused by the different operating frequencies and data transmission rates of various types of sensors, sensor failures and communication delays. The time asynchronous sampling of sensor data will lead to inconsistent timing during data fusion, thus affecting the real-time and accuracy of the system. The methods to solve the time asynchrony problem can be roughly divided into iterative state equation method and pseudo-observation method. The iterative state equation method synchronizes the system at the observation sampling time by iterating the state space model of each sensor at the measurement sampling point, but when the system is complex or the number of sensors is large, the computational burden of this method is heavy and it is difficult to meet the requirements of real-time processing.

[0004] The model mismatch problem is caused by the inability to accurately model the motion of maneuvering targets due to their complex motion. In autonomous driving systems, the motion patterns of maneuvering targets are usually highly nonlinear and dynamically changing. Traditional state estimation methods, such as Kalman filters, rely on predefined motion models that assume that the target motion follows certain rules. However, in actual situations, the motion of maneuvering targets is often affected by multiple factors, such as road conditions, driving behavior, environmental changes, etc., resulting in significant deviations between the motion model and the actual situation. Summary of the invention

[0005] The purpose of the present invention is to provide a maneuvering target tracking method based on a fusion network of MHS and GRU, so as to solve the problems of time asynchrony and model mismatch in maneuvering target fusion tracking of multi-source heterogeneous sensors in the prior art.

[0006] Maneuvering target tracking method based on the fusion network of MHS and GRU, For multi-sensor heterogeneity, The method comprises the following steps: obtaining the trajectory data of the maneuvering target, performing time registration on the trajectory data of the maneuvering target, inputting the measurement data of each sensor into the corresponding local filter, generating the local filter through Kalman filtering, dividing the trajectory data of the maneuvering target after time registration into a training set, a validation set and a test set, and inputting the training set into a local filter based on and The fusion network is iteratively trained, and after each iteration, the validation set is input into the and If the accuracy requirement is not met, the fusion network returns to iterative training. If the accuracy requirement is met, the test set is input based on and The fusion network is obtained by , the local filtering and fusion gain of each sensor The two are input into the maneuvering target tracking model together to obtain the maneuvering target tracking result.

[0007] Time alignment includes: if the current time is the sampling point of the sensor, the Kalman filtering method is used to obtain the local filter and error covariance matrix; if the current time is not the sampling point of the sensor, one or more steps of prediction based on the local filter at the previous sampling moment is used as the local filter at the current moment, and the error covariance matrix at the previous sampling moment is used as the error covariance matrix at the current moment.

[0008] The maneuvering target tracking model includes:

[0009] ;

[0010] ;

[0011] ;

[0012] In the formula, is the maneuvering target tracking result, for Always The predicted value of the state at time It is a unit array. It is a block matrix composed of multiple unit matrices as blocks. is the local filter matrix, It is The local filtered value of each sensor, yes The estimated state value at time is the state transition matrix.

[0013] Each All form one layer, The modules include feedback input , time step assignment, , fully connected layer, output Feedback value ,in Feedback to , the fully connected layer has branches that produce output .

[0014] based on and The fusion network consists of three sub-networks , and ,give and The initial value will and Input First layer, and then fuse the input values ​​of the network based on MHS and GRU enter , Output preparation matrix , and Composition block matrix , the observed data is filtered by Kalman filter to obtain the error covariance matrix ,Will , , Enter together , and obtain the covariance matrix ,Will , , Enter together ,get , , , The three parameters are adjusted through feedback through the fully connected layer. is the fully connected layer, is the first elements, It is The local filtering of the sensor and the The estimated error cross-covariance matrix of the local filtering between sensors is: is based on and The estimated value of the fusion network and the The estimated error cross-covariance matrix of the local filtering of the sensors is, is based on and The covariance of the fusion network, , The diagonal elements of , , , and the remaining positions are , No. The local filter of each sensor is placed in the Row, No. The local filter of each sensor is placed in the List.

[0015] Including the second Layer, third Layer, fourth layer, Enter into three Layer, three The output of the layer is generated and , of which the second Layer Tracking ,third Layer Tracking ,fourth Layer Tracking .

[0016] Including the fifth Layer, sixth Layer, seventh Layer, fifth Layer Tracking ,sixth Layer Tracking ,seventh Layer Tracking ,Will and The covariance of the local filters Input to The fully connected layer of the layer, , the outputs of the three fully connected layers are fused into .

[0017] Including the eighth Floor, 8th Layer Tracking ,Will and Input to the first fully connected layer generates , It is fed back to the first fully connected layer through another fully connected layer.

[0018] First, the maneuvering target tracking model is used to calculate Used for Enter in Generated in is based on and The first gain value of the fusion network, and then start the iteration, the first calculation process Used for Enter in Generated in is based on and The secondary gain value of the fusion network is calculated and iterated repeatedly until the accuracy requirement is met.

[0019] Calculation using the maneuvering target tracking model include:

[0020] .

[0021] Compared with the prior art, the present invention has the following beneficial effects: it automatically corrects and adapts to errors in the temporal alignment process, effectively reduces error accumulation, and achieves improvements in tracking accuracy, convergence speed and stability; it uses a relatively compact recurrent neural network RNN, which can be trained with a relatively small data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a simulation diagram of the motion trajectory of a maneuvering target;

[0023] Figure 2 It is the mean square error (MSE) result of the x-coordinate of target tracking by each method under the noise level with a distance variance of 1;

[0024] Figure 3 It is the mean square error (MSE) result of the y coordinate of target tracking by each method under the noise level with a distance variance of 1;

[0025] Figure 4 The mean square error (MSE) of the x-coordinate of the target tracking by each method under the noise level with a distance variance of 2;

[0026] Figure 5 It is the mean square error (MSE) result of the y coordinate of target tracking by each method under the noise level with a distance variance of 2;

[0027] Figure 6 The mean square error (MSE) of the x-coordinate of target tracking by each method under the noise level of distance variance 3;

[0028] Figure 7 The mean square error (MSE) of the y coordinate of the target tracking method under the noise level of distance variance 3;

[0029] Figure 8 The statistical graphs of MSE results of various methods under different noise levels;

[0030] Fig. 9 The results obtained by each method on the real measurement data;

[0031] Fig.10 This is a graph showing the change of the maneuvering target trajectory quality assessment index over time. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] Maneuvering target tracking method based on the fusion network of MHS and GRU, For multi-sensor heterogeneity, The method comprises the following steps: obtaining the trajectory data of the maneuvering target, performing time registration on the trajectory data of the maneuvering target, inputting the measurement data of each sensor into the corresponding local filter, generating the local filter through Kalman filtering, dividing the trajectory data of the maneuvering target after time registration into a training set, a validation set and a test set, and inputting the training set into a local filter based on and The fusion network is iteratively trained, and after each iteration, the validation set is input into the and If the accuracy requirement is not met, the fusion network returns to iterative training. If the accuracy requirement is met, the test set is input based on and The fusion network is obtained by , the local filtering and fusion gain of each sensor The two are input into the maneuvering target tracking model together to obtain the maneuvering target tracking result.

[0034] Time alignment includes: if the current time is the sampling point of the sensor, the Kalman filtering method is used to obtain the local filter and error covariance matrix; if the current time is not the sampling point of the sensor, one or more steps of prediction based on the local filter at the previous sampling moment is used as the local filter at the current moment, and the error covariance matrix at the previous sampling moment is used as the error covariance matrix at the current moment.

[0035] The maneuvering target tracking model includes:

[0036] ;

[0037] ;

[0038] ;

[0039] In the formula, is the maneuvering target tracking result, for Always The predicted value of the state at time It is a unit array. It is a block matrix composed of multiple unit matrices as blocks. is the local filter matrix, It is The local filter value of each sensor, yes The estimated state value at time is the state transition matrix.

[0040] Each All form one layer, The modules include feedback input , time step assignment, , fully connected layer, output Feedback value ,in Feedback to , the fully connected layer has branches that produce output .

[0041] based on and The fusion network consists of three sub-networks , and ,give and The initial value will and Input First layer, and then fuse the input values ​​of the network based on MHS and GRU enter , Output preparation matrix , and Composition block matrix , the observed data is filtered by Kalman filter to obtain the error covariance matrix ,Will , , Enter together , and obtain the covariance matrix ,Will , , Enter together ,get , , , The three parameters are adjusted through feedback through the fully connected layer. is the fully connected layer, is the first elements, It is The local filtering of the sensor and the The estimated error cross-covariance matrix of the local filtering between sensors is: is based on and The estimated value of the fusion network and the The estimated error cross-covariance matrix of the local filtering of the sensors is, is based on and The covariance of the fusion network, , The diagonal elements of , , , and the remaining positions are , No. The local filter of each sensor is placed in the Row, No. The local filter of each sensor is placed in the List.

[0042] Including the second Layer, third Layer, fourth layer, Enter into three Layer, three The output of the layer is generated and , of which the second Layer Tracking ,third Layer Tracking ,fourth Layer Tracking .

[0043] Including the fifth Layer, sixth Layer, seventh Layer, fifth Layer Tracking ,sixth Layer Tracking ,seventh Layer Tracking ,Will and The covariance of the local filters Input to The fully connected layer of the layer, , the outputs of the three fully connected layers are fused into .

[0044] Including the eighth Floor, 8th Layer Tracking ,Will and Input to the first fully connected layer generates , It is fed back to the first fully connected layer through another fully connected layer.

[0045] First, the maneuvering target tracking model is used to calculate Used for Enter in Generated in is based on and The first gain value of the fusion network, and then start the iteration, the first calculation process Used for Enter in Generated in is based on and The secondary gain value of the fusion network is calculated and iterated repeatedly until the accuracy requirement is met.

[0046] Calculation using the maneuvering target tracking model include:

[0047] .

[0048] The present invention uses a simulated urban traffic state machine experiment to randomly generate the target trajectory data of the vehicle, which is used to verify the effectiveness of various autonomous vehicle tracking methods. In the data generation process, the random switching of the motion model is used to simulate the diverse driving modes of the vehicle. Multiple behavior mode switching points are embedded in the vehicle motion. After reaching these time points, the driving mode of the vehicle in the next time step is determined by uniform sampling from the motion model set. The parameters of the selected model (such as speed, acceleration, etc.) are randomly allocated within a reasonable range. During the model switching process, the continuity of the vehicle trajectory and speed must be ensured. The target vehicle moves on the urban road, and the starting point is within ±10 meters around the autonomous vehicle, and the autonomous vehicle is located at the coordinate origin. The total simulation time is 25 seconds, the sampling frequency of the vehicle-mounted radar is 20Hz (0.05 seconds / time), the sampling frequency of the laser radar is 4Hz (0.25 seconds / time), and the sampling frequency of the infrared radar is 10Hz (0.1 seconds / time). In order to simulate the uncertainty in the measurement, Gaussian white noise is introduced in the distance measurement, and its variance is set to 1 square meter, 2 square meters, and 3 square meters respectively. This means that the generated measurement data will fluctuate around the true value, with most fluctuations ranging from plus or minus 1 meter, plus or minus 2 meters, and plus or minus 3 meters from the true value. The motion trajectory of the maneuvering target is simulated as follows: Figure 1 shown.

[0049] The training data set can be obtained through the simulation of the model. Specifically, random observation targets are generated within the total observation time. By simulating the target trajectory, 400 random path data are generated based on different noise conditions. On the basis of these 400 random path data, the random path data are sampled using the sampling frequencies of the three sensors, and the corresponding Gaussian white noise is added to the sampled data to obtain a total of 400×3 observation data, which are used as observation data for radar sensors, laser sensors, and infrared sensors, respectively. Since the observation data contains data in the x and y directions, the data is split into data in the x direction and data in the y direction and trained separately. In this study, the simulated data set is randomly divided into training set, validation set, and test set in a ratio of 8:1:1, and experiments are carried out on the equipped workstation. In order to train the model, the back propagation time algorithm is used to obtain the gradient, and the gradient update is performed according to the commonly used Adam optimizer. The learning rate of the training structure is set to 1×10 -3 The number of iterations epoch is set to 500. The simulation experiment compares the impact of the proposed method, the traditional multi-source heterogeneous sensor fusion tracking and the Kalman filter method of the local sensor on the target trajectory of the autonomous driving vehicle under different distance variances.

[0050] The trained network is used to test the validation data, and the tracking errors of the target under different distance variance interferences are statistically analyzed by the present invention, the traditional multi-source heterogeneous sensor fusion tracking method, and the Kalman filter method of each sensor. The mean square error (MSE) of the x-coordinate of the target tracking by each method under the noise level of distance variance 1 is shown in the figure below: Figure 2 As shown in the figure, the mean square error (MSE) of the y coordinate of target tracking by each method under the noise level of distance variance 1 is as follows: Figure 3 The statistical results of the error are shown in Table 1. The unit of the mean square error in the table is m 2 , the unit of standard deviation is m.

[0051] Table 1 Error results of target tracking by various methods under the noise level of distance variance 1

[0052] ;

[0053] from Figure 2 and Figure 3It can be seen that the present invention shows the smallest tracking error fluctuation in both the x-direction and the y-direction for target tracking. In contrast, the error fluctuation of the target tracking method using radar sensor, infrared sensor and laser sensor directly applying Kalman filter and the traditional multi-source heterogeneous sensor fusion tracking method is larger. At the first 50 sampling points, it can be seen that the present invention has smaller volatility than the other methods. In the x direction, the target tracking error of the present invention is in the range of [-0.65m, 0.33m], with a span of 0.98m; the target tracking error of the traditional multi-source heterogeneous sensor fusion tracking method is in the range of [-0.95m, 0.89m], with a span of 1.84m; the target tracking error of the Kalman filter of the radar sensor is in the range of [-0.43m, 0.74m], with a span of 1.17m; the target tracking error of the Kalman filter of the infrared sensor is in the range of [-0.36m, 1.11m], with a span of 1.47m; the target tracking error of the Kalman filter of the laser sensor is in the range of [-0.30m, 0.77m], with a span of 1.07m. The span of the present invention is the smallest, and the absolute values ​​of its maximum positive error (0.33m) and maximum negative error (-0.65m) are smaller than those of other methods, indicating that the performance of the present invention in the x direction is more stable. In the y direction, the error of the present invention in tracking the target is within the range of [-0.79m, 0.40m], and the span is 1.19m; the error of the traditional multi-source heterogeneous sensor fusion tracking method in tracking the target is within the range of [-0.59m, 0.82m], and the span is 1.41m; the error of the radar sensor's Kalman filter in tracking the target is within the range of [-0.59m, 0.60m], and the span is 1.19m; the error of the infrared sensor's Kalman filter in tracking the target is within the range of [-0.76m, 1.09m], and the span is 1.85m; the error of the laser sensor's Kalman filter in tracking the target is within the range of [-0.72m, 0.50m], and the span is 1.22m. The span of the present invention is consistent with that of the Kalman filter method of the radar sensor and is smaller than that of other methods, which can indicate that the performance of the present invention is also stable in the y direction. The target tracking results in both the x and y directions indicate that the present invention can converge quickly and remain stable.

[0054] Looking at all the sampling moments in the x-direction and the y-direction, the error result obtained by the present invention in tracking the target can converge to within the error level of ±0.5m in the first 50 sampling points, and the error is kept within the error level of ±0.5m at the subsequent sampling moments; among the other methods, only the Kalman filter method of the laser sensor can converge within the error level of ±0.5m in the y-direction, but its error level in the x-direction exceeds ±0.5m at some moments, and the error levels of the other methods in the x-direction and the y-direction exceed ±0.5m. As can be seen from Table 1, the error and standard deviation of the present invention are both smaller than those of the other methods, indicating that the error of the present invention in tracking the target is smaller and more stable.

[0055] The mean square error (MSE) of the x-coordinate of target tracking by each method under a noise level of distance variance 2 is as follows: Figure 4 As shown in the figure, the mean square error (MSE) of the y coordinate of target tracking by each method under the noise level of distance variance 2 is as follows: Figure 5 The statistical results of the errors are shown in Table 2.

[0056] Table 2 Error results of target tracking by various methods under the noise level of distance variance 2

[0057] ;

[0058] from Figure 4 and Figure 5 It can be observed that under the noise level with a distance variance of 2, in the target tracking in the x direction, the present invention converges to an error range of ±0.5m near the 65th sampling point and maintains the error level within the range of ±0.5m at subsequent sampling moments, the Kalman filtering method of the laser sensor can converge to an error range of ±0.5m near the 200th sampling point and maintain it at subsequent sampling moments, the error of the traditional multi-source heterogeneous sensor fusion tracking method is always within the error range of ±1m, and the error results of the Kalman filtering method of the radar sensor and the Kalman filtering method of the infrared sensor converge to an error range of ±1m near the 400th sampling point. In the target tracking in the y direction, the present invention converges to an error range of ±0.5m near the 80th sampling point and maintains it at subsequent sampling times. The Kalman filter method of the laser sensor can converge to an error range of ±0.5m near the 300th sampling point and maintain it at subsequent sampling times. The error of the traditional multi-source heterogeneous sensor fusion tracking method has always been within the error range of ±1m, while the error results of the Kalman filter method of the radar sensor and the Kalman filter method of the infrared sensor jump greatly, and finally converge to an error range of ±1.5m. It can be seen that the present invention shows the smallest target tracking error fluctuation in both the x and y directions. As can be seen from Table 2, the MSE of the present invention is 0.1449m2 , while the MSE of the traditional multi-source heterogeneous sensor fusion tracking method is 0.2298m 2 , which is almost twice as high as that of the present invention. The MSEs of other methods are even larger. For example, the MSE of the infrared sensor Kalman filter method is 0.9588m 2 , which is 6.6 times higher than that of the present invention. For the standard deviation, the result of the present invention is also the smallest. In summary, at a noise level with a distance variance of 2, the present invention is significantly superior to other methods in terms of target tracking accuracy and stability. It can quickly converge to a smaller error range and maintain high accuracy throughout the tracking process.

[0059] The mean square error (MSE) of the x-coordinate of target tracking by each method under a noise level of distance variance 3 is as follows: Figure 5 As shown in the figure, the mean square error (MSE) of the y coordinate of target tracking by each method under the noise level of distance variance 3 is as follows: Figure 6 The statistical results of the errors are shown in Table 3.

[0060] Table 3 Error results of target tracking by various methods under the noise level of distance variance 3

[0061] ;

[0062] from Figure 5 and Figure 6 It can be seen that under the noise level of distance variance of 3, in the target tracking in the x direction, the present invention converges to an error range of ±0.6m near the 250th sampling point and maintains it at subsequent sampling moments, the Kalman filtering method of the laser sensor can converge to an error range of ±0.7m near the 400th sampling point and maintain it at subsequent sampling moments, the traditional multi-source heterogeneous sensor fusion tracking method converges to an error range of ±1m near the 370th sampling point, and the error of the Kalman filtering method of the radar sensor converges to an error range of ±1m near the 400th sampling point, and the error result of the Kalman filtering method of the infrared sensor converges to an error range of ±1m near the 450th sampling point. In the target tracking in the y direction, the present invention converges to and maintains within the error range of ±0.5m near the 350th sampling point, the Kalman filter method of the laser sensor finally converges to the error range of ±0.7m, the error of the traditional multi-source heterogeneous sensor fusion tracking method finally converges to the error range of ±1m, and the error result of the Kalman filter method of the radar sensor finally converges to the error range of [-1,1.2]m, and the error result of the Kalman filter method of the infrared sensor jumps greatly, and finally converges to the error range of ±1.5m. As can be seen from Table 3, the quantitative analysis results further confirm the superiority of the present invention, and the MSE of the present invention is 0.2168m 2In particular, it is better than the traditional multi-source heterogeneous sensor fusion tracking method (MSE=0.5162 m 2 ) is 2.3 times better than the worst performing infrared sensor Kalman filter method (MSE = 1.3983 m 2 ) is improved by about 6.4 times. The standard deviation indicator also shows that the present invention has the minimum value, which further proves its tracking stability. The above analysis results clearly show that at a noise level of distance variance of 3, the present invention is superior to the existing methods in terms of tracking accuracy and convergence speed. It can quickly achieve and maintain high-precision tracking, which is crucial for real-time tracking systems.

[0063] A detailed statistical performance analysis is performed on the Kalman filter method of each local filter under different noise levels, the traditional multi-source heterogeneous sensor fusion maneuvering target tracking method, and the present invention for tracking maneuvering targets. The mean square error (MSE) is used as the main performance indicator because it can effectively quantify the accuracy of state estimation under different noise conditions. The MSE results of various methods under different noise levels are shown in the following table. Figure 7 , Figure 8 As shown, the noise level in the experiment is expressed as the variance of the distance, ranging from 0.5m to 5m, representing different levels of measurement uncertainty of the sensor, and the MSE of each method increases with the increase of noise level. However, the present invention consistently outperforms other methods and shows the lowest MSE at all noise levels. The optimal MSE results of the present invention can be attributed to the ability to learn the model and adapt to the complex dynamics of the maneuvering target through GRU network training with a large amount of data, thereby minimizing error accumulation and improving tracking accuracy. Although the traditional Kalman filter-based and traditional multi-source heterogeneous sensor fusion maneuvering target tracking methods are effective at lower noise levels, the error increases at higher noise levels due to the reliance on the preset maneuvering target motion model and linear assumptions. In contrast, the present invention benefits from its data-driven architecture, which enables it to improve the model mismatch problem and effectively manage the error accumulation caused by the temporal registration process. The present invention combines deep learning technology with traditional multi-source heterogeneous sensor fusion tracking methods to improve stability and convergence speed, making it a more robust solution for tracking maneuvering targets in noisy environments.

[0064] The real experiment scene shows three sensors tracking a maneuvering target. In the real experiment, three radar sensors are used to track a maneuvering target at the same time. The three radar sensors are set at different positions and directions to capture the motion trajectory of the maneuvering target. The specific configuration is: the left radar sensor is located at the (-3,0,0) position of the coordinate axis, pointing to 30° (that is, 30° rotated to the right relative to the positive direction of the y-axis); the middle radar sensor is located at the (0,0,0) position of the coordinate axis, that is, facing the positive direction of the y-axis and pointing to 0°; the right radar sensor is located at the (3,0,0) position of the coordinate axis, pointing to -30° (that is, 30° rotated to the left relative to the positive direction of the y-axis). The distance unit is meter and the angle unit is degree. The radar sensor equipment tracks the maneuvering target through parameters such as radial distance, velocity, acceleration, and azimuth.

[0065] Since the true value of the maneuvering target cannot be obtained in real-world experiments, the maneuvering target trajectory smoothness quality is used to evaluate the maneuvering target trajectory obtained by the above method. The maneuvering target trajectory smoothness quality estimates the probability by calculating the mean square error of the distance and the distribution of these errors to judge the smoothness of the trajectory. The lower the value of the trajectory smoothness quality, the smaller the fluctuation of the maneuvering target position and the better the trajectory quality.

[0066] The real measurement data measured by each sensor are processed by the Kalman filtering method, the traditional multi-source heterogeneous sensor fusion mobile target tracking method and the present invention. The results obtained by each method on the real measurement data are as follows: Fig. 9 As shown. As can be seen from the figure, the Kalman filter method is applied separately to the three radar sensors. These local filters are based only on the measurement data of a single sensor, and can generate relatively smooth target trajectories after Kalman filtering. However, due to the different installation positions of each sensor and its measurement errors, there are still obvious differences between each trajectory. The traditional multi-source heterogeneous sensor fusion maneuvering target tracking method obtains a fused target trajectory by weighted fusion of the tracking results of the three local filters. In the figure, the maneuvering target tracking trajectory of the traditional multi-source heterogeneous sensor fusion maneuvering target tracking method is located between the three local filter trajectories, indicating that the traditional multi-source heterogeneous sensor fusion maneuvering target tracking method successfully fuses the data of multi-source sensors. The present invention combines GRU and FC to process sensor data. The model can automatically learn the complex relationship between multiple sensors by training on simulated data, thereby achieving effective fusion in actual data. It can be seen that the trajectory of the maneuvering target tracking of the present invention is also located between the results of the three local filters, and is close to the results of the traditional multi-source heterogeneous sensor fusion maneuvering target tracking method, which shows that the present invention can effectively fuse multi-sensor data. Fig.10It is a graph showing the change of the quality evaluation index of the maneuvering target trajectory over time. As can be seen from the figure, the smoothness quality result obtained by the local tracking result of the first radar sensor starts to decay from around 0.8 and eventually decays to around 0.3. The smoothness quality result obtained by the local tracking result of the second radar sensor starts to rise from around 0.8, reaches the highest value around the 10th sampling point, and the highest value is about 0.9, and then decays to around 0.1. The smoothness quality result obtained by the local tracking result of the third radar sensor also starts to rise from 0.2, and rises to around 0.4 around the 5th sampling point, and then decays all the time, approaching 0 around the 70th sampling point, and then only changes slightly. The smoothness quality result obtained by the traditional multi-source heterogeneous sensor fusion maneuvering target tracking method starts to rise from 0.6 at the beginning, and rises to around 0.75 around the 5th sampling point, and finally tends to around 0.1. The smoothness quality result obtained by the present invention is 0 value in the region near all sampling points, indicating that the fluctuation degree of the maneuvering target position obtained by the present invention is very small, and the trajectory quality is high. The real experimental results verify the effectiveness of the present invention proposed by the present invention. By processing time series data with GRU, the present invention can capture dynamic changes in sensor data. The GRU network can remember long-term historical information, model and correct the noise and drift of the sensor, and finally generate relatively accurate tracking results.

[0067] Traditional multi-source heterogeneous sensor fusion maneuvering target tracking methods have problems with model mismatch and error accumulation caused by the time alignment process. This paper proposes a method of the present invention that combines a multi-source heterogeneous sensor maneuvering target tracking algorithm with a GRU network and FC layer. The proposed invention can automatically correct and adapt to errors in the time alignment process, effectively reducing error accumulation. By introducing a data-driven neural network architecture, the present invention effectively alleviates the model mismatch problem in traditional methods, and also achieves improvements in tracking accuracy, convergence speed and stability. The present invention uses a relatively compact RNN and can be trained with a relatively small data set. Further work may optimize the structure of the present invention and better improve tracking performance.

[0068] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features may be replaced by equivalents, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A maneuvering target tracking method based on a fusion network of MHS and GRU, characterized in that: For multi-sensor heterogeneity, The method comprises the following steps: obtaining the trajectory data of the maneuvering target, performing time registration on the trajectory data of the maneuvering target, inputting the measurement data of each sensor into the corresponding local filter, generating the local filter through Kalman filtering, dividing the trajectory data of the maneuvering target after time registration into a training set, a validation set and a test set, and inputting the training set into a local filter based on and The fusion network is iteratively trained, and after each iteration, the validation set is input into the and If the accuracy requirement is not met, the fusion network returns to iterative training. If the accuracy requirement is met, the test set is input based on and The fusion network is obtained by , the local filtering and fusion gain of each sensor The two are input into the maneuvering target tracking model together to obtain the maneuvering target tracking result; The maneuvering target tracking model includes: ； ； ； In the formula, is the maneuvering target tracking result, for Always The predicted value of the state at time It is a unit array. It is a block matrix composed of multiple unit matrices as blocks. is the local filter matrix, It is The local filter value of each sensor, yes The estimated state value at time is the state transition matrix.

2. The method for tracking a maneuvering target based on a fusion network of MHS and GRU according to claim 1, characterized in that: Time alignment includes: if the current time is the sampling point of the sensor, the Kalman filtering method is used to obtain the local filter and error covariance matrix; if the current time is not the sampling point of the sensor, one or more steps of prediction based on the local filter at the previous sampling moment is used as the local filter at the current moment, and the error covariance matrix at the previous sampling moment is used as the error covariance matrix at the current moment.

3. The method for tracking a maneuvering target based on a fusion network of MHS and GRU according to claim 2, characterized in that: Each All form one layer, The modules include feedback input , time step assignment, , fully connected layer, output Feedback value ,in Feedback to , the fully connected layer has branches that produce output .

4. The method for tracking a maneuvering target based on a fusion network of MHS and GRU according to claim 3, characterized in that: based on and The fusion network consists of three sub-networks , and ,give and The initial value will and Input First layer, and then fuse the input values ​​of the network based on MHS and GRU enter , Output preparation matrix , and Composition block matrix , the observed data is filtered through Kalman filtering to obtain the error covariance matrix ,Will , , Enter together , and get the covariance matrix ,Will , , Enter together ,get , , , The three parameters are adjusted through feedback through the fully connected layer. is the fully connected layer, is the first elements, It is The local filtering of the sensor and the The estimated error cross-covariance matrix of the local filtering between sensors is: is based on and The estimated value of the fusion network and the The estimated error cross-covariance matrix of the local filtering of the sensors is, is based on and The covariance of the fusion network, , The diagonal elements of , , , and the remaining positions are , No. The local filter of each sensor is placed in the Row, No. The local filter of each sensor is placed in the List.

5. The method for tracking a maneuvering target based on a fusion network of MHS and GRU according to claim 4, characterized in that: Including the second Layer, third Layer, fourth layer, Enter into three Layer, three The output of the layer is generated and , of which the second Layer Tracking ,third Layer Tracking ,fourth Layer Tracking .

6. The method for tracking a maneuvering target based on a fusion network of MHS and GRU according to claim 5, characterized in that: Including the fifth Layer, sixth Layer, seventh Layer, fifth Layer Tracking ,sixth Layer Tracking ,seventh Layer Tracking ,Will and The covariance of the local filters Input to The fully connected layer of the layer, , the outputs of the three fully connected layers are fused into .

7. The method for tracking a maneuvering target based on a fusion network of MHS and GRU according to claim 6, characterized in that: Including the eighth Floor, 8th Layer Tracking ,Will and Input to the first fully connected layer generates , It is fed back to the first fully connected layer through another fully connected layer.

8. The method for tracking a maneuvering target based on a fusion network of MHS and GRU according to claim 7, characterized in that: First, the maneuvering target tracking model is used to calculate Used for Enter in Generated in is based on and The first gain value of the fusion network, and then start the iteration, the first calculation process Used for Enter in Generated in is based on and The secondary gain value of the fusion network is calculated and iterated repeatedly until the accuracy requirement is met.

9. The method for tracking a maneuvering target based on a fusion network of MHS and GRU according to claim 8, characterized in that: Calculation using the maneuvering target tracking model include: 。

Images