Intelligent Multi-Target Association Tracking Method Based on Bayesian Inference Transformer Architecture

Abstract

This invention discloses an intelligent multi-target association tracking method (BAIT) based on a Bayesian inference Transformer architecture, combining the advantages of classical Bayesian filtering (BF) recursive inference with the Transformer's ability to handle long sequence tasks. First, BAIT employs a state predictive encoder during the prediction process to fully extract past target motion information. Then, mimicking the classical BF recursive inference structure, an association decoder is used between state prediction and filter update to achieve optimal matching and association between the target and the measurement. Next, based on the target's past motion information and the association result, a state update decoder is used to estimate the target's motion state in the current frame. Finally, by combining the classical BF iterative inference structure, BAIT can achieve high-precision target association tracking in complex data association scenarios. This method has advantages such as high precision, continuity, and accurate association, and can be applied in many fields, including military and civilian applications.

Description

Technical Field

[0001] This invention belongs to the fields of radar technology and deep learning, specifically relating to an intelligent multi-target correlation tracking method based on the Bayesian inference Transformer architecture. Background Technology

[0002] Multi-target tracking (MTT) refers to predicting the trajectory of an unknown, time-varying target using a series of imprecise sensor measurements. MTT has wide applications, including pedestrian tracking, autonomous driving, and underwater warfare. Currently, the most widely used method in MTT is Bayesian filtering-based algorithms. When an accurate multi-target model is known and the observations consist of low-dimensional, single-target detections, model-based Bayesian methods achieve the best performance. Kalman filtering (KF), based on Bayesian filtering, is a classic method that uses Bayes' theorem and a linear system dynamics model to recursively update the probability distribution of the target state to estimate and predict the target trajectory. Furthermore, the random finite set (RFS) framework is also widely used to model MTT problems in a Bayesian manner. For example, the Probabilistic Hypothesis Density (PHD) filter and its extended Cardinalized PHD (CPHD) filter are powerful tools for tracking multiple targets in complex environments without explicit data association. Representative methods also include PMBM filters and GLMB filters, both of which utilize the RFS framework to solve tracking problems, theoretically making Bayesian optimal estimation possible.

[0003] The Bayesian filtering framework demonstrates significant advantages in multi-target tracking tasks, possessing a solid theoretical foundation and an efficient computational architecture. Through its prediction and update mechanism, it effectively handles the inherent uncertainties and data correlation problems in complex tracking environments, and can provide accurate predictions by recursively improving target state estimation based on Bayesian inference, thereby enhancing tracking accuracy and robustness.

[0004] However, the Bayesian filtering algorithms mentioned above are all model-based (MB) methods, which typically exhibit significant limitations. In the filtering process, the state-space model (SSM) plays a crucial role in describing state evolution and sensor observations. The optimality of traditional Bayesian filters depends on the correctness of the prior SSM; therefore, their performance deteriorates significantly in cases of model mismatch. Furthermore, obtaining a complete and correct prior state-space model is difficult in complex, non-cooperative environments. In addition, Bayesian filtering algorithms suffer from first-order Markov property and the inability to learn from offline data.

[0005] In recent years, an attractive approach to addressing the aforementioned problems has been to utilize deep learning (DL) methods. DL typically optimizes models with a large number of parameters by minimizing the empirical risk on labeled datasets. Indeed, with breakthroughs in performance, deep learning has been increasingly applied to the MTT (Multi-Target Tracking) domain. On one hand, DL is used to handle certain sub-tasks of MTT, such as KalmanNet proposed in the paper "Kalmannet: Neural network aided kalman filtering for partially known dynamics, IEEE Transactions on Signal Processing, vol.70, pp.1532-1547, 2022," which uses RNNs for training and Kalman gain inference to assist the Kalman filter. Furthermore, recent advancements have employed DL to complete the full MTT task. The paper "Next generation multitarget trackers: Random finite set methods vs transformer-based deep learning, in Proc. Int. Conf. Inform. Fusion. IEEE, 2021, pp.1-8" proposes a high-performance, type-specific MTT neural network based on the Transformer architecture, called MultiTarget TrackingTransformer (MT3). Experimental results show that MT3 outperforms state-of-the-art Bayesian filters in complex scenes. However, MT3 is a single-frame prediction algorithm and cannot achieve continuous target state estimation. The paper "Transformer based online continuous multi-target tracking with state regression," in the 2023 12th International Conference on Control, Automation and Information Sciences (ICCAIS). IEEE, 2023, pp. 393-398, proposes an SR-MT3 method to address the problem of continuous target state estimation. This method achieves online continuous target tracking through recursive state autoregressive queries. However, it still predicts isolated target states without data association and does not form a true trajectory, failing to perform recursive prediction in a manner consistent with tracking theories such as Bayesian inference.Currently, no research in the field of deep learning has been able to integrate data association and trajectory filtering tasks into a single algorithm and train it simultaneously to solve the complete multi-target tracking task. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an intelligent multi-target association tracking method based on a Bayesian inference Transformer architecture. This invention uses an encoder to analyze the target motion state over a past period during the prediction process and employs a serial dual-decoder architecture to perform data association and filtering update processes separately. It combines the advantages of recursive inference in Bayesian architecture with the Transformer's ability to handle long sequence tasks, thus achieving high-precision target tracking in complex data association scenarios.

[0007] This invention combines the advantages of recursive inference in traditional Bayesian filtering architectures with the Transformer's ability to handle long sequence tasks, forming a novel Bayesian Inference Transformer (BAIT) multi-target association tracking processing architecture. First, BAIT employs a state predictive encoder during the prediction process to fully extract past motion information from multiple targets. Then, mimicking the classic Bayesian filtering recursive inference structure, an association decoder is used between state prediction and filter updates to achieve optimal matching and association between multiple target states and multiple measurement data. Next, a state update decoder is used to estimate the motion states of multiple targets in the current frame based on past target motion information and target measurement association results. Combining the classic Bayesian filtering iterative inference structure, the target prediction state in the current frame can also be used as input to the state predictive encoder for recursive state prediction and target state estimation in the next frame. Finally, through continuous iterative processing, BAIT can achieve high-precision target association tracking tasks in complex data association scenarios.

[0008] The objective of this invention is achieved through the following technical solution: an intelligent multi-target correlation tracking method based on Bayesian inference Transformer architecture, the specific steps of which are as follows:

[0009] S1. Generate simulation measurement data based on the target motion state and sensor observation model;

[0010] S2. Overall architecture design of BAIT network model: The BAIT network model adopts an encoder-decoder architecture that includes one Transformer encoder and two Transformer decoders; one Transformer encoder is responsible for state prediction in the algorithm, one Transformer decoder is responsible for data association in the algorithm, and the other Transformer decoder is responsible for state update in the algorithm.

[0011] S3, Design State Prediction Encoder;

[0012] S4. Design a data association decoder;

[0013] S5, Design State Update Decoder;

[0014] S6. Design a loss function and input the measurement data obtained in step S1 into the overall model constructed through steps S2, S3, S4, and S5 to train the network model.

[0015] S7. Use the trained BAIT network model to identify unknown data and output multi-objective state estimation results with numbers.

[0016] The beneficial effects of this invention are as follows: The method innovatively combines the recursive advantages of neural networks with the prediction and updating of classical Bayesian architecture. It uses neural networks to replace pre-assumed motion and observation models while retaining the excellent features of the Bayesian architecture. BAIT uses an encoder to analyze the target's motion state over a past period during the prediction process and employs a serial dual-decoder architecture to perform data association and filtering update processes separately. It combines the advantages of recursive inference in the Bayesian architecture with the Transformer's ability to handle long sequence tasks, thus achieving high-precision target tracking in complex data association scenarios.

[0017] This invention, from a deep learning data-driven perspective, breaks free from the limitations of traditional methods based on motion and observation models. It innovatively integrates with an excellent Bayesian architecture, exhibiting good interpretability and forming a multi-target, end-to-end intelligent tracking algorithm with data association. This algorithm achieves high-precision target tracking in complex data association scenarios. With advantages such as high precision, continuity, and accurate association, this method can be applied to numerous fields including autonomous driving, traffic control, and military operations. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the implementation of an intelligent multi-target correlation tracking method based on a Bayesian inference Transformer architecture according to the present invention.

[0019] Figure 2 This is a schematic diagram of the overall architecture of the BAIT network, the core model of this invention.

[0020] Figure 3 A flowchart showing the relationship between the association decoder and the state update decoder;

[0021] Figure 4 This is a comparison curve of the OSPA index for Task 1 in this embodiment of the invention;

[0022] Figure 5This is an example of the evaluation of Task 1 in an embodiment of the present invention;

[0023] Figure 6 OSPA for Task 2 in this embodiment of the invention (2) Indicator comparison curve;

[0024] Figure 7 This is an example of the evaluation of Task 2 in an embodiment of the present invention. Detailed Implementation

[0025] This embodiment primarily employs simulation experiments for verification. All steps and conclusions have been verified correctly using Matlab 2021b and Python 3.9. The technical solution of this invention is further described below with reference to the accompanying drawings.

[0026] like Figure 1 As shown, this invention discloses an intelligent multi-target correlation tracking method based on a Bayesian inference Transformer architecture. First, radar simulation data is designed and acquired to generate two-dimensional measurement data. Then, the measurement data is preprocessed, and high-dimensional vector information is extracted before being input into a pre-constructed BAIT network model for training. Finally, test data is input into the trained network model to obtain high-precision multi-target tracking results in scenarios where data correlation is difficult. The specific steps of this invention are as follows:

[0027] S1. Generate simulation measurement data based on the target motion state and sensor observation model. The specific method is as follows: First, generate simulation measurement data based on the target motion state equation and sensor observation model as the dataset for the deep learning network. Then, divide the dataset into a training set for training the BAIT network model and a test set for testing the model's performance.

[0028] Approximating the target's motion as uniform motion, the state vector of target i at frame t is: The state vector of target i at frame t-1 is The equation of motion is then expressed as:

[0029]

[0030] in, Let x and y represent the x-axis position and y-axis position of target i at frame t, respectively. F represents the x-direction velocity and y-direction velocity of target i at frame t, respectively; t W represents the state transition matrix. t-1 This represents the process noise matrix, where the process noise is Gaussian noise with zero mean and a covariance of Q. t-1 ;

[0031]

[0032] q s The process noise variance is represented by T, the sensor sampling period is represented by I², and the second-order identity matrix is ​​represented by I². Indicates the Kronecker product;

[0033] The new target is reached via a Poisson point process, with a birth intensity of λ. b , Represents the state of all targets in frame t. A set of.

[0034] Consider a group of sensors with the same field of view (FOV) detecting the same set of targets. Then, an analog sensor is used to generate measurements in a two-dimensional Cartesian coordinate system, and each existing target can generate at most one true measurement. The observation equation for the actual measurements captured by the sensor s is expressed as follows:

[0035]

[0036] in, This represents the measurement value of target i obtained by the sensor at frame t; n t The measurement noise of the sensor has a mean of zero and a variance of R. t H represents the sensor's measurement matrix, specifically represented as:

[0037]

[0038] Clutter measurement is based on intensity λ c The Poisson point process arrives independently of the existing target or actual measurement, and the set of all measurement values ​​in frame t is:

[0039]

[0040] in, This represents the set of clutter generated by the sensor in frame t.

[0041] This embodiment sets up two tasks to evaluate the performance of BAIT in two tasks of different complexities and compares it with the original MT3 and Kalman filter (KF). Due to the large number of measurements in each frame, the inexpensive joint probabilistic data association (CJPDA) was chosen as the most suitable data association algorithm for KF to avoid the exponential growth of computational explosion. In Task 1, the tracking accuracy of BAIT, MT3, and KF was compared in common scenarios with moderate clutter. In Task 2, the focus was on comparing the tracking of BAIT and KF in scenarios with more clutter, which presents a more complex challenge for data association. The parameters for the two tasks are shown in Tables 1 and 2, respectively; then, the sensor's two-dimensional measurement data was normalized within the field of view, and a linear layer was used to extract the XY coordinate measurements into a high-dimensional feature vector.

[0042] Table 1

[0043] parameter symbol numerical values Target speed v ±U(10,20)m Field of view FOV [-30m, 30m] × [-30m, 30m] Sliding window length τ 5 Sampling period Δt 0.1s Poisson parameter of the initial number of targets <![CDATA[λ0]]> 8 Target motion duration T 2s Process noise <![CDATA[q s ]]> <![CDATA[0.09m 2 / s 2 ]]> Measurement noise R <![CDATA[0.01m 2 ]]> Detection probability <![CDATA[P d ]]> 0.95 Poisson parameter of clutter number <![CDATA[λ c ]]> 10

[0044] Table 2

[0045]

[0046]

[0047] S2. Overall Architecture Design of BAIT Network Model: The BAIT network model adopts an encoder-decoder architecture that includes one Transformer encoder and two Transformer decoders. One Transformer encoder is responsible for state prediction in the algorithm, one Transformer decoder is responsible for data association in the algorithm, and the other Transformer decoder is responsible for state update in the algorithm.

[0048] Step S2 is as follows: To clearly explain the data correlation issue, the measurement... Expanded to:

[0049]

[0050] in Let m represent the i-th measurement in frame t, and m t This represents the number of measurements in frame t; and the target state estimation set in frame t is defined as:

[0051]

[0052] in This represents the state of the j-th target in frame t, where n is the number of targets. t It represents the number of targets in frame t.

[0053] To address the issue of false alarms, a virtual track was introduced. Therefore, a pseudo-state with index j=0 was added to each set. The state estimation set is redefined as follows:

[0054]

[0055] The target tracking task is further divided into data association task and state filtering task;

[0056] The problem of state estimation for a multi-object with label numbers is solved by using measurements of the current frame T and a series of estimated states from the past time step τ up to the last T-1 frame.

[0057] The sequence of estimated states from the past time step τ up to the last T-1 frame is represented as:

[0058]

[0059] in

[0060]

[0061] Where ′ represents the matrix transpose, n t It is the estimated number in frame t. It estimates the X and Y coordinates of the target k. It is a unique label for estimating the target k, where Let i represent the set of positive integers, and the label parameter α for different values ​​of i. i They are different (labels (numbers) used to represent the estimated target); t represents the t-th frame from the start of sensor detection;

[0062] The set of measurements for the current T-frame is represented as follows:

[0063]

[0064] Where, m T It is the number of measurements in the current frame T. These are the X and Y coordinates of the measurement k.

[0065] The result of the target tracking task is to obtain a set of estimated labels and states for the target in the current frame T, in the form of:

[0066]

[0067] BAIT employs an encoder-decoder architecture comprising one Transformer encoder and two Transformer decoders, specifically, according to formula (10) in the set The estimated state is collected within a period τ; and the current frame Each measurement is added to the sequence in formula (13) in a random order. middle.

[0068] The two Transformer decoders are a data association decoder and a state filtering decoder; the embedded sequence is obtained from the Transformer encoder during the state prediction process. Generated during the transformation; then it is combined with the query sequence. Together, they are fed into the data association decoder to generate the probability of which trajectory each measurement belongs to, and further generate association results as measurement-trajectory matching pairs through a matching and sorting mechanism; finally, these matching pairs are rearranged and input into the state filter decoder to generate estimated states during the update process. result It will return to the Transformer encoder as the recursive input at the next frame T+1.

[0069] Figure 2 This is a schematic diagram of the overall architecture of the BAIT network of the present invention. Figure 3 This is a flowchart illustrating the relationship between the association decoder and the state update decoder. Figure 2 As shown, the estimated state of the last τ frame, after flattening and preprocessing, is first fed into the Transformer encoder to generate an embedding vector representing the motion features during the previous specific time period, a process called implicit prediction. Then, the data association decoder performs data association based on these features to match measurements and tracks in the current frame. After converting the measurement-track pairs into permuted measurements through a matching and reordering mechanism, the filter update decoder performs an update process and outputs the estimated state for the current frame T. The estimated state result for the current frame T is then returned to the Transformer encoder for recursive estimation.

[0070] like Figure 3 As shown, the data association decoder will convert the measurement values and embedded e 1:N This is converted into a matching probability matrix (10×4 table on the left). Each row in the matrix represents a measurement detected in the current frame, each column represents a trajectory, and each grid represents the probability of the corresponding measurement from the corresponding trajectory. The highest probability in each row is colored, and their corresponding measurements are also colored in the visualization scene graph (dashed squares above the table). The matching and reordering mechanism selects and arranges measurement values ​​according to the matching probability matrix to populate the filter update query matrix (3×4 table on the right). Each row in the matrix represents a matching tracking measurement value with a unique label (visible in the dashed box above the table); clutter is excluded from the queries. These queries are ultimately fed into the filter update decoder to perform the update process.

[0071] S3. Design the state prediction encoder; specifically as follows: The formula for the BAIT state prediction process is expressed as:

[0072]

[0073] Where p represents the probability density function. Indicates in under conditions The probability density function; This represents a part of the deep learning network in the algorithm, so as to replace Bayesian implementations like the Chapman-Komogorov equation with similar functions;

[0074] A Transformer encoder is used to handle state prediction in the algorithm; after flattening and preprocessing, the encoder extracts features from the previously estimated states into embedding vectors; the embedding vectors are the state evolution probability models. Another representation of this allows the encoder to perform implicit predictions.

[0075] S4. Design a data association decoder; specifically as follows: The formula for the data association process in BAIT is expressed as:

[0076]

[0077] Where p represents the probability density function (pdf); This represents a part of the deep learning network in the algorithm, so as to replace Bayesian implementations like the Chapman-Komogorov equation with similar functions;

[0078] The data association decoder connects the prediction and update steps, matching the current frame measurement with different previous trajectories and providing these matching pair combinations to the state filtering update decoder; the output of the association decoder is a matching probability matrix (MPM), which represents the probability that each measurement comes from each different trajectory; the formula is expressed as:

[0079]

[0080] in,

[0081]

[0082] in, This indicates the probability that measurement i comes from trajectory j, m is the number of measurements in the current frame; s represents the maximum number of trajectories, and pseudotrack s=0 indicates clutter;

[0083] The matching and sorting mechanism first performs softmax along the rows of the matching probability matrix to normalize the probabilities; then, it selects the most likely generated measurement value by performing argmax along each column of the matching probability matrix, thereby producing a query sequence q with state filtering updates. 1:s In addition, in another subtask, the linear layer processes the matching probability to calculate an existence threshold, and tracks with a probability below this value are considered terminated.

[0084] S5. Design the state update decoder; specifically as follows: The formula for the state update process in BAIT is expressed as:

[0085]

[0086] Where p represents the probability density function; This represents a part of the deep learning network in the algorithm, so as to replace Bayesian implementations like the Chapman-Komogorov equation with similar functions;

[0087] After state prediction and data association, the state filter decoder transforms the filtered query and embedding vector into an estimated state. 1:k This is responsible for the state update process, where k represents the number of existing tracks in the current frame; these state update output sets This will be incorporated into the prediction process for recursion in the next frame after appending the current timestamp T.

[0088] S6. Design the loss function and input the measurement data obtained in step S1 into the overall model constructed in steps S2, S3, S4, and S5 to train the network model; specifically as follows: all tracking predictions are supervised using correlation loss and filtering loss; given the matching probability matrix in (19) and the ground true probability of all measurements at frame T:

[0089]

[0090] The correlation loss includes cross-entropy (CE) loss and MCD loss, and the specific formulas are as follows:

[0091]

[0092] Where γ is a smoothing term to prevent loss overflow;

[0093] For the filtering loss, given the predictions and ground truth states for all estimated states at frame T:

[0094]

[0095] in It is the number of real targets in frame T; therefore, the filtering loss is expressed as:

[0096]

[0097] Loss SL1 It is a smoothed L1 loss, σ * Represents pairing o i and Matching;

[0098] The total loss of backpropagation is the sum of the association loss and the filtering loss and their respective factors. The training process of the entire network model is divided into forward propagation and backpropagation. A large amount of training data is obtained through step S1 and input into the network model. At the same time, the weights of each unit are continuously adjusted using the loss function.

[0099] The estimation performance of the network model of this invention was verified using data from the test set.

[0100] In this embodiment, the key parameter information in the MT3 network model of the comparison algorithm is shown in Table 3, while the key parameters in the BAIT network model are shown in Table 4.

[0101] Table 3

[0102] parameter numerical values Encoder layers 6 Decoder layers 6 Number of multi-head attention 8 Number of hidden units in FFN 2048 State Dimension 256 Gradient descent step size 800k Batch size 16

[0103] Table 4

[0104] parameter numerical values Predictive encoder layer number 6 Number of associated decoder layers 3 Number of filter decoder layers 6 Number of multi-head attention 8 Number of hidden units in FFN 1024 State Dimension 256 Gradient descent step size 800k Batch size 16

[0105] The test data is input into the trained network model to obtain the estimated tracking results for multiple targets, and the tracking results are evaluated. Specifically, the optimal subpattern allocation (OSPA) metric is used to evaluate the model's performance, as follows:

[0106]

[0107] d c (x,y)=min{c,d(x,y)} (28)

[0108] Among them, D p,c Indicates OSPA error. Indicates the prediction result. Γ represents the true value, and Γ represents a set of values ​​corresponding to equation (30); p represents the distance sensitivity parameter; c represents a parameter for cutting off distance, i.e., the target state estimation error threshold, which is used to adjust the ratio between the estimation error of the set potential and the position error; d(·) represents the function for calculating the distance;

[0109] In addition, OSPA was also used. (2) Measurement, definition and Two sets of trajectories are used simultaneously, employing the basic distance method. To define and Distance between sets

[0110]

[0111] Where t∈{1,...,K}, p represents the distance sensitivity parameter, c represents a cutoff distance parameter consistent with the OSPA metric; q is the order of the basic distance, and w is the set of convex weights. For all test tasks, c=1.5 and p=1 were chosen, and 1000 rounds of Monte Carlo testing were performed for each experiment. This embodiment sets up two tasks. In Task 1, the tracking accuracy of BAIT, MT3, and KF in common scenarios with moderate clutter was compared. In Task 2, the tracking accuracy of BAIT and KF in scenarios with more clutter was compared, which implies a complex challenge in data correlation. The evaluation results are as follows: Figure 4 , Figure 5 and Figure 6 As shown.

[0112] Figure 4 This is a comparison curve of the OSPA index for Task 1 in this embodiment of the invention. Since MT3 and BAIT are initialized in the first four frames, the comparison begins from the fifth frame. The upward-facing triangle, downward-facing triangle, and solid circle represent KF, MT3, and BAIT, respectively. The OSPA index shows that BAIT's tracking accuracy remains stable throughout the tracking process, significantly outperforming KF and MT3. The Kalman filter error is slightly higher at the beginning but gradually decreases over time, with the decreasing trend slowing down near the 10th frame. MT3 is a single-frame prediction algorithm without independent data association; therefore, prediction errors during track crossings accumulate in subsequent predictions, interfering with tracking accuracy.

[0113] Figure 5 This is an evaluation example of Task 1 in this embodiment of the invention. The sample contains 8 trajectories, and the image is a magnified view involving three targets. Measurement results are represented by crosses, while circles represent the true ground location. The prediction results of the three methods are given: KF is represented by an upward-facing triangle, MT3 by a downward-facing triangle, and BAIT by a solid circle. It can be seen that for most points, BAIT has higher positioning accuracy than KF and MT3. BAIT fully utilizes the recursive advantages of the Bayesian algorithm and the long-sequence understanding capability of the Transformer architecture to accurately grasp the trajectory trend. Due to this accurate grasp, BAIT can maintain stable performance without interference, while KF has more significant errors due to competition between measurement targets, and MT3 exhibits poor prediction due to track intersections.

[0114] Figure 6 OSPA for Task 2 in this embodiment of the invention (2) Indicator comparison curve. For clarity, a dashed line with an upward-pointing triangle is used to represent KF in OSPA. (2) The distance error in the indicator is represented by a dashed line with a downward-pointing triangle, indicating KF at OSPA. (2) The positioning error in the indicator is represented by a dashed line with a solid circle, indicating KF at OSPA. (2) Basis error in the indicator. BAFT in OSPA is represented by a solid line with an upward-pointing triangle. (2) The distance error in the indicator is represented by a solid line with a downward-pointing triangle, indicating BAFT in OSPA. (2) The positioning error in the indicator is represented by a solid line with a filled circle, indicating BAFT's position in OSPA. (2) The base error in the indicator.

[0115] OSPA (2) The distance error of the metric is the sum of the localization error and the baseline error. Localization error represents the distance between the ground truth and the estimated state. It can be seen that before frame 14, BAIT's localization error is lower than KF's, after which the errors become similar. Baseline error represents the accuracy of the target quantity estimation. When the estimated target quantity deviates from the true target quantity, the baseline error increases. As shown in the figure, BAIT's baseline error remains stable at a lower level than KF. This means that BAIT has superior tracking performance in this challenging scenario.

[0116] Figure 7 This is an evaluation example of Task 2 in this embodiment of the invention. There are a total of 8 trajectories in this example; this image is a magnified view of a portion involving four targets. In the right image, all measurement points in frame 9 of this example are plotted, with different colors representing the tracking trajectories of different targets. The left image is a visualization of the MPM in frame 9, where higher color saturation indicates a greater association probability, and 'C' represents clutter. As can be seen from the right image, when the trajectories of target 3 and target 4 intersect in frame 9, KF exhibits a significant tracking error due to adjacent measurement interference and association failure, while BAIT can still perform accurate measurement-target association and maintain excellent tracking performance. The results show that this invention can select the maximum measurement-target matching probability from the MPM calculated by the association decoder, thereby overcoming interference and ensuring excellent estimation accuracy.

[0117] S7. Use the trained BAIT network model to identify unknown data and output multi-objective state estimation results with numbers.

[0118] In summary, as can be seen from the embodiments of this invention, the method of this invention can effectively achieve high-precision target tracking in complex data association scenarios. This invention innovatively combines the advantages of classical Bayesian filtering (BF) recursive inference with the Transformer's ability to handle long sequence tasks, using a neural network to replace the pre-assumed motion and observation models of traditional methods, while retaining the excellent recursive inference Bayesian architecture. To achieve sufficient and effective extraction of past target motion information, BAIT uses a predictive encoder to analyze past motion states during the prediction process. Since accurate data association can improve the accuracy of subsequent track filtering, we follow the idea of ​​the classical BF method and implement accurate data association between state prediction and filter update through an association decoder. Subsequently, the state update decoder captures target motion feature information and outputs the predicted target state, and the target predicted state of the current frame is used for recursive state prediction and update in the next frame.

[0119] This invention's method, starting from a data-driven perspective of deep learning, breaks free from the limitations of traditional methods based on motion and observation models. It also innovatively combines the advantages of a superior Bayesian architecture, resulting in excellent interpretability. This method boasts advantages such as high precision, continuity, accurate correlation, and good interpretability, making it applicable to numerous fields including autonomous driving, traffic management, and military operations.

[0120] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. An intelligent multi-target correlation tracking method based on Bayesian inference Transformer architecture, characterized in that, The specific steps are as follows: S1. Generate simulation measurement data based on the target motion state and sensor observation model; S2. Overall architecture design of BAIT network model: The BAIT network model adopts an encoder-decoder architecture that includes one Transformer encoder and two Transformer decoders; one Transformer encoder is responsible for state prediction in the algorithm, one Transformer decoder is responsible for data association in the algorithm, and the other Transformer decoder is responsible for state update in the algorithm. S3, Design State Prediction Encoder; S4. Design a data association decoder; S5, Design State Update Decoder; S6. Design a loss function and input the measurement data obtained in step S1 into the overall model constructed through steps S2, S3, S4, and S5 to train the network model. S7. Use the trained BAIT network model to identify unknown data and output multi-objective state estimation results with numbers.

2. The intelligent multi-target association tracking method based on Bayesian inference Transformer architecture according to claim 1, characterized in that, The specific method of step S1 is as follows: First, generate simulation measurement data based on the target motion state equation and sensor observation model, which serves as the dataset for the deep learning network; Approximating the target's motion as uniform motion, the state vector of target i at frame t is: The state vector of target i at frame t-1 is The equation of motion is then expressed as: Among them, F t W represents the state transition matrix. t-1 This represents the process noise matrix, where the process noise is Gaussian noise with zero mean and a covariance of Q. t-1 ; q s The process noise variance is represented by T, the sensor sampling period is represented by I², and the second-order identity matrix is ​​represented by I². Indicates the Kronecker product; The new target is reached via a Poisson point process, with a birth intensity of λ. b , Represents the state of all targets in frame t. A set; Consider a group of sensors with the same field of view detecting the same set of targets. Then, an analog sensor is used to generate measurements in a two-dimensional Cartesian coordinate system, and each existing target can generate at most one true measurement. The observation equation for the actual measurements captured by sensor s is expressed as follows: in, This represents the measurement value of target i obtained by the sensor at frame t; n t The measurement noise of the sensor has a mean of zero and a variance of R. t H represents the sensor's measurement matrix, specifically expressed as: Clutter measurement is based on intensity λ c The Poisson point process arrives independently of the existing target or actual measurement, and the set of all measurement values ​​in frame t is: in, This represents the set of clutter generated by the sensor in frame t.

3. The intelligent multi-target association tracking method based on Bayesian inference Transformer architecture according to claim 2, characterized in that, The specific steps of step S2 are as follows: To clearly explain the data correlation problem, the measurement... Expanded to: in Let m represent the i-th measurement in frame t, and m t This represents the number of measurements in frame t; and the target state estimation set in frame t is defined as: in This represents the state of the j-th target in frame t, where n is the number of targets. t It is the number of targets in frame t; By adding the pseudo-state with index j=0 to each set The state estimation set is redefined as follows: The target tracking task is subdivided into a data association task and a state filtering task; the state estimation problem of multi-targets with label numbers is solved by using the measurement of the current frame T and a series of estimated states from the past time step τ up to the last T-1 frame. The estimated sequence of states is represented as follows: in Where ′ represents the matrix transpose, n t It is the estimated number in frame t. It estimates the X and Y coordinates of the target k. It is the unique label for estimating the target k, where N represents the set of positive integers, and the parameter α is... i They are different; The set of measurements for the current frame is represented as follows: Where, m T It is the number of measurements in the current frame T. It measures the X and Y coordinates of k; The result of the target tracking task is to obtain a set of estimated labels and states for the target in the current frame T, in the form of: BAIT employs an encoder-decoder architecture comprising one Transformer encoder and two Transformer decoders, specifically, according to formula (10) in the set The estimated state within the collection period τ; and the current frame Each measurement is added to the sequence in formula (13) in a random order. middle; The two Transformer decoders are a data association decoder and a state filtering decoder; the embedded sequence is obtained from the Transformer encoder during the state prediction process. Generated during the transformation; then it is combined with the query sequence. Together, they are fed into the data association decoder to generate the probability of which trajectory each measurement belongs to, and further generate association results as measurement-trajectory matching pairs through a matching and sorting mechanism; finally, these matching pairs are rearranged and input into the state filter decoder to generate estimated states during the update process. result It will return to the Transformer encoder as the recursive input at the next frame T+1.

4. The intelligent multi-target association tracking method based on Bayesian inference Transformer architecture according to claim 3, characterized in that, Step S3 is as follows: The formula for the BAIT state prediction process is expressed as: Where p represents the probability density function; This represents a part of the deep learning network in the algorithm, so as to replace Bayesian implementations like the Chapman-Komogorov equation with similar functions; A Transformer encoder is used to handle state prediction in the algorithm; after flattening and preprocessing, the encoder extracts features from the previously estimated states into embedding vectors; the embedding vectors are the state evolution probability models. Another representation of this allows the encoder to perform implicit predictions.

5. The intelligent multi-target association tracking method based on Bayesian inference Transformer architecture according to claim 3, characterized in that, Step S4 is as follows: The formula for the data association process in BAIT is expressed as: Where p represents the probability density function; This represents a part of the deep learning network in the algorithm, so as to replace Bayesian implementations like the Chapman-Komogorov equation with similar functions; The data association decoder connects the prediction and update steps, matching the current frame measurement with different previous trajectories and providing these matching pair combinations to the state filtering update decoder; the output of the association decoder is a matching probability matrix, which represents the probability that each measurement comes from each different trajectory; the formula is expressed as: in, in, This indicates the probability that measurement i comes from trajectory j, m is the number of measurements in the current frame; s represents the maximum number of trajectories, and pseudotrack s=0 indicates clutter; The matching and sorting mechanism first performs softmax along the rows of the matching probability matrix to normalize the probabilities; then, it selects the most likely generated measurement value by performing argmax along each column of the matching probability matrix, thereby producing a query sequence q with state filtering updates. 1:s .

6. The intelligent multi-target association tracking method based on Bayesian inference Transformer architecture according to claim 3, characterized in that, Step S5 is as follows: The formula for the state update process in BAIT is expressed as: Where p represents the probability density function; This represents a part of the deep learning network in the algorithm, so as to replace Bayesian implementations like the Chapman-Komogorov equation with similar functions; After state prediction and data association, the state filter decoder transforms the filtered query and embedding vector into an estimated state. 1:k This is responsible for the state update process, where k represents the number of existing tracks in the current frame; these state update output sets This will be incorporated into the prediction process for recursion in the next frame after appending the current timestamp T.

7. The intelligent multi-target association tracking method based on Bayesian inference Transformer architecture according to claim 3, characterized in that, The specific steps of step S6 are as follows: all tracking predictions are supervised using correlation loss and filtering loss; given the matching probability matrix in (19) and the ground true probabilities of all measurements at frame T: The correlation loss includes cross-entropy loss and MCD loss, and the specific formula is as follows: Where γ is a smoothing term to prevent loss overflow; For the filtering loss, given the predictions and ground truth states for all estimated states at frame T: in It is the number of real targets in frame T; therefore, the filtering loss is expressed as: Loss SL1 It is a smoothed L1 loss, σ * Represents pairing o i and Matching; The total loss of backpropagation is the sum of the association loss and the filtering loss and their respective factors. The training process of the entire network model is divided into forward propagation and backpropagation. A large amount of training data is obtained through step S1 and input into the network model. At the same time, the weights of each unit are continuously adjusted using the loss function.

Images