A convolution-bidirectional long short-term memory multi-source track association method and system based on spatiotemporal attention
By using the spatiotemporal attention module combined with CNN and BiLSTM network methods in track correlation, the spatial and timing characteristics of track data are extracted, and the problem of low stability and accuracy of track correlation technology in complex scenarios is solved, and the problem of low stability and accuracy of track correlation technology in complex scenarios is realized. More efficient track correlation effect.
Patent Information
- Application Number
- CN202410536907.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-30
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-04-30
AI Technical Summary
Existing track correlation technologies have problems of poor stability and low accuracy when dealing with complex track scenarios, especially when the track scenarios are expanded, the data sources are diversified and the complexity is increased.
The spatial and temporal attention-based convolution-bidirectional long and short-term memory multi-source track correlation method is adopted. The spatial and timing characteristics of the track data are extracted through the feature attention module and the temporal attention module combined with the CNN and BiLSTM networks, and the attention mechanism is used. Weighted calculations to improve the accuracy of track correlation.
It improves the accuracy and stability of track correlation, makes full use of the spatial and temporal characteristics of track data, and processes complex track data more efficiently than traditional methods.
Smart Images

Figure CN118520268B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of track association, and more specifically, relates to a convolution-bidirectional long short-term memory multi-source track association method and system based on spatiotemporal attention. Background Art
[0002] Modern technology has promoted the widespread use of monitoring equipment in modern warfare. These equipment can obtain important information, but also bring a large amount of data, making the battlefield situation more complicated. In order to accurately obtain the combat situation, it is not enough to rely on only one sensor. Since a single sensor has defects such as a single detection range, low measurement accuracy, and strong environmental interference, multi-source sensor track correlation technology has developed rapidly.
[0003] Traditional track correlation technology is mainly divided into two categories. One is based on traditional mathematical methods, using different mathematical models and mathematical methods to perform track linear programming and constraints, and improving various mathematical algorithms to achieve higher accuracy. However, since traditional mathematical methods need to consider a large number of complex mathematical models and algorithms, and have high requirements on the quality of track data, it is difficult to handle complex track correlation environments and has low efficiency in large-scale data processing; the other is a method based on grayscale correlation analysis, which determines whether the track sequence is associated by analyzing the similarities or differences of the track sequence curves. It does not have too many requirements on the accuracy, distribution type and correlation of the data, but the parameter setting is complex and requires a large number of simulation adjustment parameters. Summary of the invention
[0004] In view of the above defects or improvement needs of the prior art, the present invention provides a convolution-bidirectional long short-term memory multi-source track association method and system based on spatiotemporal attention, which aims to solve the problems of poor track association stability and low accuracy caused by the increasing scope of track scenes, the wide sources of track data and the increasing complexity of track data.
[0005] To achieve the above object, according to a first aspect of the present invention, a convolution-bidirectional long short-term memory multi-source track association method based on spatiotemporal attention is provided, comprising:
[0006] S1, obtain the track data Z observed by observation platforms A and B respectively A , Z B , for Z A , Z B Perform temporal and spatial registration to obtain Z A `、Z B `;
[0007] S2, Z A `、Z B`Input into the pre-trained multi-source track association model to obtain the association result;
[0008] Wherein, the multi-source track association model includes a feature attention module, a temporal attention module and a fully connected layer connected in sequence;
[0009] The feature attention module includes a CNN network and a first attention mechanism module connected in sequence, the CNN network is used to extract the feature vector of the input data, and the first attention mechanism module includes a first single-layer BP network, a first softmax layer and a first weight distribution layer connected in sequence, which are respectively used to calculate the feature attention weight vector of the feature vector of the input data, normalize the feature attention weight vector and perform weighted calculation on the feature vector of the input data to obtain the high-dimensional spatial features of the track data;
[0010] The temporal attention module includes a BiLSTM network and a second attention mechanism module connected in sequence, the BiLSTM network is used to extract the temporal feature vector of the high-dimensional spatial feature, and the second attention mechanism module includes a second single-layer BP network, a second softmax layer, and a second weight distribution layer connected in sequence, which are respectively used to calculate the temporal attention weight vector of the temporal feature vector, normalize the temporal attention weight vector, and perform weighted calculation on the temporal feature vector to obtain a track feature containing spatial information and temporal information;
[0011] The fully connected layer is used to classify and predict the track features containing spatial information and temporal information to obtain the association result of the input data.
[0012] According to a second aspect of the present invention, a convolution-bidirectional long short-term memory multi-source track association device based on spatiotemporal attention is provided, comprising:
[0013] The first processing module is used to obtain the track data Z observed by the observation platforms A and B respectively on the observation target A , Z B , for Z A , Z B Perform temporal and spatial registration to obtain Z A `、Z B `;
[0014] The second processing module is used to convert Z A `、Z B `Input into the pre-trained multi-source track association model to obtain the association result;
[0015] Wherein, the multi-source track association model includes a feature attention module, a temporal attention module and a fully connected layer connected in sequence;
[0016] The feature attention module includes a CNN network and a first attention mechanism module connected in sequence, the CNN network is used to extract the feature vector of the input data, and the first attention mechanism module includes a first single-layer BP network, a first softmax layer and a first weight distribution layer connected in sequence, which are respectively used to calculate the feature attention weight vector of the feature vector of the input data, normalize the feature attention weight vector and perform weighted calculation on the feature vector of the input data to obtain the high-dimensional spatial features of the track data;
[0017] The temporal attention module includes a BiLSTM network and a second attention mechanism module connected in sequence, the BiLSTM network is used to extract the temporal feature vector of the high-dimensional spatial feature, and the second attention mechanism module includes a second single-layer BP network, a second softmax layer, and a second weight distribution layer connected in sequence, which are respectively used to calculate the temporal attention weight vector of the temporal feature vector, normalize the temporal attention weight vector, and perform weighted calculation on the temporal feature vector to obtain a track feature containing spatial information and temporal information;
[0018] The fully connected layer is used to classify and predict the track features containing spatial information and temporal information to obtain the association result of the input data.
[0019] According to a third aspect of the present invention, there is provided a convolution-bidirectional long short-term memory multi-source track association system based on spatiotemporal attention, comprising: a computer-readable storage medium and a processor;
[0020] The computer-readable storage medium is used to store executable instructions;
[0021] The processor is used to read the executable instructions stored in the computer-readable storage medium and execute the method as described in the first aspect.
[0022] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the method described in the first aspect.
[0023] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:
[0024] 1. The present invention constructs a feature attention module and utilizes the CNN network feature extraction capability to extract the hidden information of the track in the spatial distribution. On this basis, an attention mechanism is added to highlight important spatial features, reduce redundant features and computational burden, and help improve the efficiency of the time attention module in extracting temporal features.
[0025] 2. The present invention constructs a time attention module, uses the BiLSTM network to extract time series feature vectors from high-dimensional spatial features, makes full use of the time series information of the track data, and uses the time attention module to further analyze the importance of track points at each historical moment, so as to better highlight the impact of important historical moments on the final result of track association.
[0026] 3. The method provided by the present invention fully utilizes the spatial and temporal characteristics of the track by constructing a CNN-BiLSTM neural network model based on spatiotemporal attention. Compared with the traditional track association algorithm, it can fully utilize the effective information contained in the local underwater acoustic track, and the association accuracy is higher. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 It is a flow chart of a convolution-bidirectional long short-term memory multi-source track association method based on spatiotemporal attention provided by an embodiment of the present invention;
[0028] Figure 2 It is a feature attention module model diagram of a convolution-bidirectional long short-term memory multi-source track association method based on spatiotemporal attention provided by an embodiment of the present invention;
[0029] Figure 3 It is a temporal attention module model diagram of a convolution-bidirectional long short-term memory multi-source track association method based on spatiotemporal attention provided by an embodiment of the present invention;
[0030] Figure 4 It is a flow chart of the overall network model training of the convolution-bidirectional long short-term memory multi-source track association method based on spatiotemporal attention provided by an embodiment of the present invention;
[0031] Figure 5 It is a correlation index result diagram of a convolution-bidirectional long short-term memory multi-source track correlation method based on spatiotemporal attention provided by an embodiment of the present invention;
[0032] Figure 6 It is a comparison chart of multi-source track association results using different neural network models provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0033] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0034] In recent years, deep learning has developed rapidly. Artificial neural networks have been widely used because of their strong fault tolerance, self-learning, self-organization and self-adaptation capabilities, and their ability to simulate complex nonlinear mappings. Since track data is generated by a series of spatiotemporal interactions during the target's motion, and contains multi-dimensional data in time and space, multi-layer neural networks and attention mechanisms can be used to improve the effect of track feature extraction, thereby improving the accuracy of track association. Based on this, an embodiment of the present invention provides a convolution-bidirectional long short-term memory multi-source track association method based on spatiotemporal attention, such as Figure 1 As shown, including:
[0035] S1, obtain the track data Z observed by observation platforms A and B respectively A , Z B , for Z A , Z B Perform temporal registration and spatial registration and transform them into the same spatiotemporal coordinate system to obtain Z A `、Z B `.
[0036] Specifically, the time registration and the space registration may adopt any existing algorithm. Preferably, a time registration algorithm based on interpolation and extrapolation is adopted for time registration, and a space registration algorithm based on geocentric coordinate system is adopted for space registration.
[0037] Firstly, a time registration algorithm based on interpolation and extrapolation is used to solve the problem of time misalignment of trajectory points caused by different sampling times of sensors. Then, a spatial registration algorithm based on the geocentric coordinate system (ECEF) is used to align the local track data in different spatial coordinate systems.
[0038] The interpolation and extrapolation method assumes that in the same spatial coordinate system, the measurement data obtained by the two observation platforms A and B are Y A (t i ) and Y B (t i ), where the sampling time of observation platform A is shorter, the trajectory points of observation platform A are unified to the trajectory data of observation platform B with a longer sampling period. For the time period of the local trajectory, the measurement data of each observation platform are arranged in sequence, and then the measurement trajectory points of observation platform A are interpolated and extrapolated to the trajectory points of observation platform B one by one to form the target measurement data aligned with observation platform B. The three-point parabola interpolation method is used for interpolation and extrapolation. Then, in the same spatial coordinate system, observation platform A is Bk The measurement value of the moment is
[0039]
[0040] When the time alignment of the sensor is completed, the spatial position information from the target needs to be processed. The origin of the ECEF Cartesian coordinate system is at the center of the earth, the rotation axis of the earth is the Z axis, the Greenwich meridian is the X axis, and the Y axis is in the equatorial plane. The three axes follow the right-hand rule. For a known sensor with a geodetic coordinate of (L s ,λ s ,H s ), then ECEF Cartesian coordinates (x s ,y s ,z s ) can be expressed as
[0041]
[0042] Where e and E q represent the eccentricity and equatorial radius of the Earth respectively, and C is defined as
[0043]
[0044] When the sensor processes the information from the target, it obtains the distance angle measurement information (r t ,θ t ,η t ), where r t represents the slope distance, θ t Represents the azimuth relative to true north, η t represents the pitch angle, then the local Cartesian coordinates can be expressed as
[0045]
[0046] Using the spatial registration algorithm based on the geocentric coordinate system, the local coordinate system (x1, y1, z1) of the target is converted into the ECEF coordinate system (x t ,y t ,z t ), conversion
[0047]
[0048] S2, Z A `、Z B `Input into the pre-trained multi-source track association model to obtain the association result;
[0049] Wherein, the multi-source track association model includes a feature attention module, a temporal attention module and a fully connected layer connected in sequence;
[0050] The feature attention module includes a CNN network and a first attention mechanism module connected in sequence. The CNN network is used to extract the feature vector of the input data. The first attention mechanism module includes a first single-layer BP network, a first softmax layer and a first weight distribution layer connected in sequence, which are respectively used to calculate the feature attention weight vector of the feature vector of the input data, normalize the feature attention weight vector and perform weighted calculation on the feature vector of the input data to obtain the high-dimensional spatial features of the track data.
[0051] The feature attention module is constructed by using the CNN neural network and the first attention mechanism module to convert the track data into a feature sequence. Specifically, the CNN network is constructed, and the attention mechanism is added on this basis to construct the feature attention module, so that the network pays more attention to the features that are more important to the current task. The CNN network extracts the high-dimensional complex features of the input data layer by layer with multi-layer convolution and pooling operations. At the same time, the first attention mechanism module is used in the entire process of CNN network output feature extraction, dynamically assigning attention weights to different input features of the model, mining the nonlinear relationship between input data and target prediction features, ensuring that important features play a key role, reducing or suppressing irrelevant features, and further improving the accuracy and robustness of the model.
[0052] Figure 2 The following is the feature attention module. First, a CNN network is constructed to extract local track features. At time t, the M feature single-step vectors x extracted from CNN are t =[x 1,t ,x 2,t ,x 3,t ,…,x M,t ] is input into the first single-layer BP network to calculate the attention weight vector:
[0053] e t =σ(W e x t +b e )
[0054] where e t =[e 1,t ,e 2,t ,e 3,t ,…,e M,t ] represents the input feature x in t time steps t The corresponding attention coefficient weight set; W e b is the weight matrix that can be learned and trained by back-propagating the model; b e is the bias vector; σ is the activation function Tanh. The obtained attention weights are normalized:
[0055]
[0056] Finally, the feature attention model weight coefficient [α 1,t ,α 2,t ,α 3,t ,…,α m,t ,…,α M,t ], then enter x t The output x is obtained by weighted calculation with the weight coefficient t ′:
[0057] x t ′=α t ⊙x t =[α 1,t x 1,t ,α 2,t x 2,t ,α 3,t x 3,t ,…,α M,t x M,t ]
[0058] Where ⊙ is the Hadamard product, that is, the product of the corresponding positions of the matrix. At the same time, the feature attention module is located in the shallow layer of the model, and the numerical range of the input sequence features is fixed.
[0059] The feature attention module parameters are shown in Table 1:
[0060] Table 1
[0061]
[0062] The temporal attention module includes a BiLSTM network and a second attention mechanism module connected in sequence, the BiLSTM network is used to extract the temporal feature vector of the high-dimensional spatial feature, and the second attention mechanism module includes a second single-layer BP network, a second softmax layer, and a second weight distribution layer connected in sequence, which are respectively used to calculate the temporal attention weight vector of the temporal feature vector of the high-dimensional spatial feature, normalize the temporal attention weight vector, and perform weighted calculation on the temporal feature vector to obtain a track feature containing spatial information and temporal information;
[0063] The BiLSTM neural network and the second attention mechanism module are used together to form a time attention module to capture the time series characteristics of the trajectory data. Specifically, it includes: building a BiLSTM network, adding an attention mechanism on this basis, and building a time attention module. The BiLSTM network has an excellent ability to learn time series data, especially in the recognition of periodic and trend characteristics. After the attention mechanism is trained on the hidden state weights, the network can automatically filter out the most important state information in the historical sequence. This process can effectively solve the problems of gradient explosion and information loss when the BiLSTM network bidirectionally mines long-term series information, thereby better highlighting the impact of important historical moments on the final results of track prediction.
[0064] Figure 3 The figure shows the temporal attention module. First, a BiLSTM network is constructed. The BiLSTM network inputs the feature information output by the feature attention module into two independent LSTM networks in the forward and reverse order of time to capture the features of the input sequence from the forward and reverse directions respectively. Specifically, the forward LSTM processes the input sequence from left to right according to the time step, and the hidden state p of each time step is t and the cell state c t They are calculated by the following formulas:
[0065] i t =σ(W it x t +W ih h t-1 +b i )
[0066] f t =σ(W fx x t +W fh h t-1 +b f )
[0067] o t =σ(W ot x t +W oh h t-1 +b o )
[0068]
[0069]
[0070] p t =o t ⊙tanh(c t )
[0071] Among them, i t、f t , o t and are the input gate, forget gate, output gate and candidate state representing the current information, σ and tanh are Sigmoid and hyperbolic tangent functions, and ⊙ represents element-by-element product. Similarly, the reverse LSTM processes the input sequence from right to left in time steps, and the hidden state p at each time step is t ′ and cell state c t ′ can be calculated by a similar formula. Finally, the output of BiLSTM is concatenated from the hidden states in two directions: t =[p t ,p t ′].
[0072] The hidden state of the BiLSTM network unit at time step t is taken as input h t =[h 1,t ,h 2,t ,h 3,t ,…,h M,t The time window length is represented by k, and the time attention weight vector l calculated by the second single-layer BP network is t Refers to the weight assigned to the correspondence between the current moment t and the historical moment:
[0073] l t =ReLU(W d h t +b d )
[0074] Among them l t =[l 1,t ,l 2,t ,l 3,t ,…,l k,t ],W d is the temporal attention weight matrix, b d is the bias vector; the ReLU function is used in temporal attention to expand the differences in module input features and make the weight distribution more centralized. The obtained attention weights are normalized:
[0075]
[0076] After the above normalization process, the weight coefficient β of the temporal attention model is obtained t =[β 1,t ,β 2,t ,β 3,t ,…,β τ,t ,…,β k,t ], β τ,tis the time attention weight value at time τ; the comprehensive temporal information state h is obtained by weighted summing of the hidden states at each historical moment t ′:
[0077]
[0078] in These weights are assigned through the temporal attention mechanism, which represents the relationship between the current moment and each historical moment.
[0079] The parameters of the temporal attention module are shown in Table 2:
[0080] Table 2
[0081]
[0082] The fully connected layer is used to classify and predict the track features containing spatial information and temporal information to obtain the association result of the input data.
[0083] Specifically, the output data is mapped to a new feature space using a fully connected layer and scored using a classifier. The correlation between multi-source track data is determined by calculating the correlation score.
[0084] That is, h t ′ is taken as input and input into two fully connected layers, where the number of neurons in the second layer is 1. After the second layer, the Sigmoid activation function is used to make the output association score between 0 and 1. If the output association score is less than 0.5, it is judged as unassociated; otherwise, it is associated.
[0085] Furthermore, the performance of the multi-source track association model can be evaluated by the following formula.
[0086] Let TP represent the track sample that is actually associated, but is judged as associated by the model, and the prediction result of the model is correct; FP represent the track sample that is actually unassociated, but is judged as associated by the model, and the prediction result of the model is wrong; FN represent the track sample that is actually associated, but is judged as unassociated by the model, and the prediction result of the model is wrong; TN represent the track sample that is actually unassociated, but is judged as unassociated by the model, and the prediction result of the model is correct. Among these four cases, two are judged correctly, and two are judged incorrectly.
[0087] Then the correlation accuracy formula of the model is:
[0088]
[0089] The formula for the missed association rate is:
[0090]
[0091] The precision formula is:
[0092]
[0093] In order to verify the effectiveness of the method provided by the present invention, the actual scene simulation is combined to generate track data, and model training and performance verification are performed based on the track data.
[0094] like Figure 4 As shown in the figure, it is the overall flow chart of the overall network model training. The specific steps are as follows:
[0095] (1) Simulation to generate track data
[0096] Model and analyze various target motion models and track detection scenarios to obtain local track data. Assuming that the target performs uniform linear motion (CV), uniform accelerated motion (CA) or uniform circular motion (CT) in a short period of time, these three target motion models are used to generate local track data.
[0097] In order to make the local track more consistent with the actual hydroacoustic positioning data and promote the track association algorithm to extract more hydroacoustic positioning data features, the moving target and the observation platform motion model are modeled and simulated. The target motion equation is:
[0098] x(k)=x(k-1)+ν×cos(o)×T
[0099] y(k)=y(k-1)+v×sin(o)×T
[0100] Where (x(k), y(k)) represents the spatial coordinates of a target in the ground coordinate system at time k, v is the speed of the target, o is the heading angle, T is the sensor sampling rate, and the radar sensor sampling rate is 0.02s.
[0101] The motion equation of the observation platform is:
[0102] o m (k) = o m (k-1)+θ m ×T
[0103] x m (k) = x m (k-1)+v m ×cos(o m (k-1))×T
[0104] y m (k) = y m (k-1)+v m ×sin(o m (k-1))×T
[0105] Where (x m (k),y m (k)) is the spatial coordinate of the observation platform at time k, v m is the navigation speed of the observation platform, o m (k) is the heading angle of the observation platform at time k, θ m (k) is the angular velocity of the observation platform.
[0106] Set up four observation platforms, namely O1, O2, O3, and O4, and four moving targets, namely T1, T2, T3, and T4. The parameter settings of each platform are shown in Table 3:
[0107] Table 3
[0108]
[0109] The parameters of each motion target are set as shown in Table 4:
[0110] Table 4
[0111]
[0112] The observation platform generates track data of each moving target based on the motion models of both parties.
[0113] (2) Track data preprocessing
[0114] The track data observed by different observation platforms are temporally and spatially aligned to eliminate the time and space differences between different sensors and ensure the uniformity of the track data in time and space.
[0115] (3) Build and train the model
[0116] First, three groups of convolutional layers and pooling layers of CNN network models are constructed to extract high-dimensional spatial features of track sequences. The number of convolutional kernels is 256, 100, and 50 respectively, and the feature attention layer is used to highlight the influence of important spatial features. Then, a BiLSTM network model with two hidden layers is constructed to mine the bidirectional temporal features of the track sequence output by the feature attention module. The hidden layers have 90 and 100 neurons respectively, and the time attention layer is used to further strengthen the influence of important historical information. Finally, a fully connected layer with 1 neuron is used to realize the association results of the track. In order to prevent the model from overfitting, two Dropout layers are added to the model, and the Dropout rate is 0.2573. In addition, the optimizer is Adam. According to the computer performance, the number of iterations is 100 and the batch size is 256. Finally, the output data is mapped to the new feature space by the fully connected layer, and the prediction result of the model is output by the classifier. The prediction result of the model is associated or not associated, and the association relationship between multi-source track data is determined. Figure 5 The figure shows the changing curves of association accuracy, missed association rate and precision rate obtained by changing the learning rate, so the final learning rate parameter is selected as 0.026.
[0117] (4) Comparison of different models
[0118] In order to verify the effectiveness of the spatiotemporal attention convolution-bidirectional long short-term memory network proposed in the present invention, the method of the present invention is applied to multi-source track association with CNN, BiLSTM, and CNN-BiLSTM. All network models are based on track data generated by the same motion model. The statistical results of 100 random experiments are compared. Figure 6 As shown, it can be seen that the track association accuracy of the method of the present invention based on spatiotemporal attention CNN-BiLSTM is higher than that of other methods.
[0119] According to the characteristics of CNN-BiLSTM and spatiotemporal attention, the present invention realizes multi-source track association, effectively improves the accuracy of track association, and solves the problems of long-term tracks and multi-source tracks in the track association process. The method proposed in this method is based on the idea of deep learning, and takes the detection platform track detection simulation scene as an experimental example. In the actual application process, it can be extended to application scenarios such as underwater hydroacoustic detection and underwater target track prediction, indicating that the present invention has good practicality.
[0120] The embodiment of the present invention provides a convolution-bidirectional long short-term memory multi-source track association device based on spatiotemporal attention, comprising:
[0121] The first processing module is used to obtain the track data Z observed by the observation platforms A and B respectively on the observation target A , Z B , for Z A , Z B Perform temporal and spatial registration to obtain Z A `、Z B `;
[0122] The second processing module is used to convert Z A `、Z B `Input into the pre-trained multi-source track association model to obtain the association result;
[0123] Wherein, the multi-source track association model includes a feature attention module, a temporal attention module and a fully connected layer connected in sequence;
[0124] The feature attention module includes a CNN network and a first attention mechanism module connected in sequence, the CNN network is used to extract the feature vector of the input data, and the first attention mechanism module includes a first single-layer BP network, a first softmax layer and a first weight distribution layer connected in sequence, which are respectively used to calculate the feature attention weight vector of the feature vector of the input data, normalize the feature attention weight vector and perform weighted calculation on the feature vector of the input data to obtain the high-dimensional spatial features of the track data;
[0125] The temporal attention module includes a BiLSTM network and a second attention mechanism module connected in sequence, the BiLSTM network is used to extract the temporal feature vector of the high-dimensional spatial feature, and the second attention mechanism module includes a second single-layer BP network, a second softmax layer, and a second weight distribution layer connected in sequence, which are respectively used to calculate the temporal attention weight vector of the temporal feature vector of the high-dimensional spatial feature, normalize the temporal attention weight vector, and perform weighted calculation on the temporal feature vector to obtain a track feature containing spatial information and temporal information;
[0126] The fully connected layer is used to classify and predict the track features containing spatial information and temporal information to obtain the association result of the input data.
[0127] The embodiment of the present invention provides a convolution-bidirectional long short-term memory multi-source track association system based on spatiotemporal attention, including: a computer-readable storage medium and a processor;
[0128] The computer-readable storage medium is used to store executable instructions;
[0129] The processor is used to read the executable instructions stored in the computer-readable storage medium and execute the method described in any of the above embodiments.
[0130] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the method described in any of the above embodiments.
[0131] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A convolution-bidirectional long short-term memory multi-source track association method based on spatiotemporal attention, It is characterized in that include: S1, obtain the track data Z observed by observation platforms A and B respectively A , Z B , for Z A , Z B Perform temporal and spatial registration to obtain Z A `、Z B `; S2, Z A `、Z B `Input into the pre-trained multi-source track association model to obtain the association result; Wherein, the multi-source track association model includes a feature attention module, a temporal attention module and a fully connected layer connected in sequence; The feature attention module includes a CNN network and a first attention mechanism module connected in sequence, the CNN network is used to extract the feature vector of the input data, and the first attention mechanism module includes a first single-layer BP network, a first softmax layer and a first weight distribution layer connected in sequence, which are respectively used to calculate the feature attention weight vector of the feature vector of the input data, normalize the feature attention weight vector and perform weighted calculation on the feature vector of the input data to obtain the high-dimensional spatial features of the track data; The temporal attention module includes a BiLSTM network and a second attention mechanism module connected in sequence, the BiLSTM network is used to extract the temporal feature vector of the high-dimensional spatial feature, and the second attention mechanism module includes a second single-layer BP network, a second softmax layer, and a second weight distribution layer connected in sequence, which are respectively used to calculate the temporal attention weight vector of the temporal feature vector, normalize the temporal attention weight vector, and perform weighted calculation on the temporal feature vector to obtain a track feature containing spatial information and temporal information; The fully connected layer is used to classify and predict the track features containing spatial information and temporal information to obtain the association result of the input data.
2. The method according to claim 1, It is characterized in that The first single-layer BP network is based on the formula e t =σ(W e x t +b e ) Calculate the feature attention weight vector of the feature vector of the input data; Among them, e t =[e 1,t , e 2,t , e 3,t , ..., e M,t ] represents the input feature x in t time steps t The corresponding attention coefficient weight set, x t =[x 1,t , x 2,t , x 3,t , …, x M,t ] is the M feature single-step vector extracted from CNN at time t, W e b is the weight matrix that can be learned and trained by back-propagating the model; b e is the bias vector; σ is the activation function Tanh.
3. The method according to claim 1 or 2, It is characterized in that The second single-layer BP network is based on formula l t =ReLU(W d h t +b d ) calculating the time attention weight vector of the temporal feature vector; Among them, l t =[l 1,t , l 2,t , l 3,t , ..., l k,t ],W d is the temporal attention weight matrix, b d is the bias vector, h t =[h 1,t ,h 2,t ,h 3,t , ..., h M,t ] is a high-dimensional space feature.
4. The method according to claim 1, It is characterized in that In step S1, a time registration algorithm based on interpolation and extrapolation is used to perform time registration.
5. The method according to claim 1, It is characterized in that In step S1, a spatial registration algorithm based on a geocentric coordinate system is used to perform spatial registration.
6. A convolution-bidirectional long short-term memory multi-source track association device based on spatiotemporal attention, It is characterized in that include: The first processing module is used to obtain the track data Z observed by the observation platforms A and B respectively on the observation target A , Z B , for Z A , Z B Perform temporal and spatial registration to obtain Z A `、Z B `; The second processing module is used to convert Z A `、Z B `Input into the pre-trained multi-source track association model to obtain the association result; Wherein, the multi-source track association model includes a feature attention module, a temporal attention module and a fully connected layer connected in sequence; The feature attention module includes a CNN network and a first attention mechanism module connected in sequence, the CNN network is used to extract the feature vector of the input data, and the first attention mechanism module includes a first single-layer BP network, a first softmax layer and a first weight distribution layer connected in sequence, which are respectively used to calculate the feature attention weight vector of the feature vector of the input data, normalize the feature attention weight vector and perform weighted calculation on the feature vector of the input data to obtain the high-dimensional spatial features of the track data; The temporal attention module includes a BiLSTM network and a second attention mechanism module connected in sequence, the BiLSTM network is used to extract the temporal feature vector of the high-dimensional spatial feature, and the second attention mechanism module includes a second single-layer BP network, a second softmax layer, and a second weight distribution layer connected in sequence, which are respectively used to calculate the temporal attention weight vector of the temporal feature vector, normalize the temporal attention weight vector, and perform weighted calculation on the temporal feature vector to obtain a track feature containing spatial information and temporal information; The fully connected layer is used to classify and predict the track features containing spatial information and temporal information to obtain the association result of the input data.
7. A convolution-bidirectional long short-term memory multi-source track association system based on spatiotemporal attention, It is characterized in that include: A computer readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is used to read the executable instructions stored in the computer-readable storage medium and execute the method according to any one of claims 1 to 5.
8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the method according to any one of claims 1 to 5.
Citation Information
Patent Citations
Track time sequence classification algorithm based on convolution self-attention mechanism
CN114219021A
Space-time attention LSTM vehicle trajectory prediction method based on position-speed
CN116872963A
Multi-ship trajectory prediction method and system based on dynamic space-time refinement network
CN117743795A
Passenger car trajectory planning method and system based on learning network and rule hybrid drive
CN117782127A
