Position prediction method in multi-target tracking based on random noise and attention mechanism

Abstract

The invention provides a position prediction method in multi-target tracking based on random noise and an attention mechanism, and belongs to the field of multi-target tracking. In the technical scheme provided by the invention, a traditional Kalman filtering method is replaced by constructing a target driving network, and the target driving network comprises an LSTM encoder unit, a rough target estimation unit, a CVAE-based target position prediction unit and a random noise and attention mechanism-based accurate target estimation unit; target position prediction in a non-mismatching scene is carried out through a CVAE target position prediction unit, and target position prediction in a mismatching scene is carried out through an accurate target position prediction unit based on random noise and a multi-head attention mechanism. And the deviation of target position prediction in an actual complex scene is effectively reduced. Therefore, the number of mismatching between the targets is reduced, and the precision of multi-target tracking in an actual complex scene is obviously improved.

Description

Technical Field

[0001] The present invention belongs to the field of multi-target tracking, and in particular relates to a position prediction method in multi-target tracking based on random noise and attention mechanism. Background Art

[0002] Multi-Object Tracking (MOT) is an important research direction in the field of computer vision. Its main task is to predict the position of the target in the subsequent time frame and its corresponding relationship through the target detection results in the video or image sequence. In the prior art, Kalman filter is a widely used target position prediction method. Kalman filter is a recursive least squares estimation algorithm based on linear dynamic model and Gaussian distribution assumption. Its core is to combine the current observation value and the predicted value to predict the position of the target through the state estimation equation and the state update equation. However, in actual complex scenes, Kalman filter has the following technical problems:

[0003] 1. Insufficient ability to model nonlinear dynamics: Kalman filtering assumes that the target motion conforms to a linear dynamic model, which is difficult to establish in actual complex scenes. Targets in complex scenes usually have nonlinear and diverse motion patterns, such as sudden acceleration, turning or occlusion. Kalman filtering is difficult to effectively model these nonlinear motions, resulting in large prediction errors.

[0004] 2. Insufficient robustness to data noise and occlusion: In multi-target tracking tasks, the target positions output by the detector may contain noise, or even mismatch or occlusion. Kalman filtering relies on the Gaussian distribution assumption and cannot fully cope with these complex noise interferences, thus reducing the robustness and accuracy of the prediction.

[0005] 3. Unable to model the uncertainty of the target position: The Kalman filter outputs a deterministic prediction value, which fails to capture the probability distribution characteristics of the target position. This makes it difficult to provide more potential trajectory information when there is a large uncertainty in the target position, limiting the flexibility of subsequent decision-making. Summary of the invention

[0006] In order to solve the problem of large position prediction error in complex scenarios, the present invention introduces a long short-term memory network (LSTM) to extract target position features, which can better extract dynamic features in historical trajectories and enhance the ability of nonlinear motion modeling.

[0007] In order to deal with the mismatch problem in actual complex scenes, the present invention proposes an accurate target position prediction unit based on random noise and attention mechanism, which can simulate mismatch scenes through random noise, thereby improving the robustness of the model in complex scenes and reducing the impact of mismatch on target position prediction.

[0008] In order to model the uncertainty of target position in actual complex scenarios, the present invention constructs a conditional variational autoencoder (CVAE) in the target position prediction scenario to model the probability distribution of the target position, and uses historical trajectory information as a condition to generate multiple possible target position distributions to comprehensively capture the uncertainty of the target position.

[0009] In order to solve the above technical problems, the specific technical solution of the position prediction method in multi-target tracking based on random noise and attention mechanism of the present invention is as follows:

[0010] Step 1: Obtain multi-target tracking data; collect the target's τ-frame historical position information, including the position of the target's bounding box in the video frame in the two-dimensional image coordinates, and divide the data into a training set and a test set;

[0011] Step 2: Input the historical position of the target trajectory into the target-driven network, where the target-driven network includes an LSTM encoder unit, a coarse target estimation unit, a target position prediction unit based on CVAE, and a precise target estimation unit with random noise and attention mechanism;

[0012] The LSTM encoder unit extracts the temporal information of the target in the past τ frames and outputs the hidden state and cell state to the rough target estimation unit;

[0013] The coarse target estimation unit generates coarse target position predictions through an internal LSTM decoder and a multi-head attention mechanism;

[0014] The target position prediction unit based on CVAE includes a recognition network, a priori network, a generative network and an LSTM decoder. Specifically, the input vector containing the target historical position information is passed through a gated recurrent unit to obtain a common input vector, and the common input vector is input into the priori network to generate a priori latent variables. At the same time, the common input vector and the target real position are input into the recognition network together to generate recognition latent variables. Then, the priori latent variables and the recognition latent variables are selected, and the recognition latent variables are selected during training, and the priori latent variables are selected during testing or inference. They are input into the generative network to generate an initial latent state and an initial cell state, which are then input into the LSTM decoder to generate accurate target position predictions in conventional scenarios.

[0015] The precise target estimation unit of random noise and attention mechanism includes a multi-head attention mechanism and a multi-layer perceptron. First, the hidden state of the LSTM encoder unit and the output of the rough target estimation unit are respectively passed through the fully connected layer to obtain a vector containing the target historical position information and a vector containing the target predicted position information. Then, these two vectors are concatenated and passed through the fully connected layer to obtain a query vector, a key vector, and a value vector based on the attention mechanism, wherein the value vector contains random noise. Then, the final trajectory prediction vector is obtained through the multi-head attention mechanism, and the target position prediction in the scenario of random noise output is obtained through the multi-layer perceptron.

[0016] Step 3: Use the collected data to train and test the target-driven network to obtain a trained target-driven network model, and apply the model to actual target position prediction reasoning.

[0017] The specific workflow of the rough target estimation unit is as follows:

[0018] First, the hidden state and cell state from the LSTM encoder are decoded through the LSTM decoder to generate the decoded hidden state and cell status Generate the supplementary input of the LSTM encoder unit for the next time step through the fully connected layer and the input of the LSTM decoder

[0019]

[0020] Among them, FC represents the fully connected layer;

[0021] Then, the query vector Q, key vector K, and value vector V based on the attention mechanism are obtained through the fully connected layer:

[0022]

[0023] Among them, W Q is the weight matrix of the fully connected layer that generates the query vector Q, W K is the weight matrix of the fully connected layer that generates the key vector K, W V is the weight matrix of the fully connected layer that produces the value vector V;

[0024] Finally, a rough target position prediction is obtained through the multi-head attention mechanism

[0025]

[0026] Among them, Multi_Attention represents the multi-head attention mechanism.

[0027] The specific process of the target position prediction unit based on CVAE is as follows:

[0028] First, the hidden state from the LSTM encoder passes through the fully connected layer to generate an input vector containing the target historical position information, and then passes through the gated recurrent unit to generate the common input vector X of the prior network and the recognition network. P , input the true position Y of the feature t Generate the input vector Y of the recognition network through the fully connected layer Q ;X P and Y Q Input recognition network to output recognition latent variable z Q :

[0029]

[0030] X P Generate a priori latent variables z through the prior network P :

[0031] z P =MLP(X P )

[0032] Among them, MLP stands for multi-layer perceptron, Represents a splicing operation;

[0033] Then the prior latent variable z is selected by the selection operation P Or identify the latent variable z Q As the hidden variable z, the initial hidden state of the LSTM decoder is obtained by generating the network and initial cell state The generated network includes a fully connected layer and two parallel multi-layer perceptrons, which are expressed as follows:

[0034] h z =FC(z)

[0035]

[0036] Where z is P and z P One of them is that during training, z=z Q , in testing or reasoning, z = z P ;h z is the intermediate vector obtained by the fully connected layer;

[0037] Finally, the LSTM decoder generates accurate target position predictions in common scenarios

[0038]

[0039] Where goal_for_dec is the output of the coarse target estimation unit Obtained through the fully connected layer.

[0040] The specific process of the precise target estimation unit based on random noise and attention mechanism is as follows:

[0041] First, the hidden state of the LSTM encoder unit is passed through the fully connected layer to obtain the vector enc_hidden containing the target historical position information, and the output of the rough target estimation unit After the fully connected layer, we get the vector goal_for_p containing the target prediction position information;

[0042] The vector enc_hidden containing the target historical position information and the vector goal_for_p containing the target predicted position information are converted into intermediate vectors through the fully connected layer. and

[0043]

[0044] and The query vectors Q = {q1, q2…, q k}, key vector K = {k1, k2…, k k} and a vector of values ​​V = {v1,v2…,v k}, k is the number of different semantics specified artificially, that is, the final output and the intermediate output are increased by one dimension to represent the semantic information. The process is expressed as follows:

[0045]

[0046] Where W Q , W K and W V is a randomly initialized matrix from the fully connected layer, Represents a concatenation operation, and random noise random_noise randomly scales the feature vector to generate the effect of a feature vector in the wrong position;

[0047] Then the final target position is predicted through the multi-head attention mechanism, and the calculation of each head is as follows:

[0048]

[0049] in, is the scaling factor, T is the matrix transpose; thus, h different heads are obtained, namely head1, head2, …, head h, the input of each head is a set of k features; the h heads are concatenated to obtain the final trajectory prediction vector F:

[0050] F=concat(head1,head2,…,head h )

[0051] Finally, the target position prediction in the scene with random noise is output by the multi-layer perceptron

[0052]

[0053] When training the target-driven network, the root mean square error is used as the loss function to supervise the loss of the CVAE and Attention position prediction units; for each training sample, the final loss is as follows:

[0054]

[0055] Loss total =Loss Global +Loss CVAE +Loss KLD +Loss Attention

[0056] Among them, Loss total Represents the final loss function, Loss Global Represents the loss function of the predicted value and the true value of the rough target estimation unit, Loss CVAE Represents the loss function of the rough target estimation unit, Loss Attention Represents the loss function of the accurate target estimation unit, Loss KLD represents the KLD loss function.

[0057] In the technical solution provided by the present invention, the traditional Kalman filter method is replaced by constructing a target-driven network, and the target position prediction of the non-mismatching scene is performed by the CVAE target position prediction unit, and the target position prediction of the mismatching scene is performed by the precise target position prediction unit based on random noise and multi-head attention mechanism. The deviation of the target position prediction in the actual complex scene is effectively reduced. In addition, the number of mismatches between targets is reduced, so that the accuracy of multi-target tracking in the actual complex scene is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of the structure of a target-driven network provided by an embodiment of the present invention;

[0059] Figure 2 Estimating unit structure diagram for rough target;

[0060] Figure 3 This is the structure diagram of the CVAE target position prediction unit;

[0061] Figure 4 The structure diagram of the precise target position prediction unit for the attention mechanism and random noise;

[0062] Figure 5 Comparison of athlete trajectories before and after adding noise trajectories;

[0063] Figure 6 Iteration plots of the loss during training and testing. DETAILED DESCRIPTION

[0064] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a position prediction method in multi-target tracking based on random noise and attention mechanism of the present invention in conjunction with the accompanying drawings.

[0065] like Figure 1 As shown in FIG. 1 , the target-driven network of the present invention includes: an LSTM encoder unit, a rough target estimation unit based on an attention mechanism, a target position prediction unit based on CVAE and random noise, and a precise target estimation unit based on an attention mechanism and random noise. The input of the LSTM encoder unit is the latent vector X of the historical position information t ={x t-τ+1 ,x t-τ+2 ,...,x t}, where x t represents the bounding box information, τ is the artificially specified input time step, and the output contains the hidden state of the historical position feature and cell state C t At time step t, the coarse target estimation unit receives the output from the LSTM encoder unit and outputs a coarse target position prediction and decoder input Complementary input to the LSTM encoder at the next time step (t+1) The target position prediction unit based on CVAE accepts the hidden state from the LSTM encoder unit as The output of the fully connected layer and the coarse target estimation unit The output of the fully connected layer, goal_for_dcc, is used as input, and the LSTM decoder is used to output accurate target position prediction. Accurate target estimation unit with random noise and attention mechanism accepts hidden states from LSTM encoder Through the output of the fully connected layer enc_hidden, random noise random_noise and the rough target estimation unit The output goal_for_p of the fully connected layer is used as input to output the target position prediction in the noisy scene. Next, the method provided in this embodiment is further described.

[0066] Step 1: Obtain multi-target tracking data; the evaluation dataset uses SportsMOT, in which 240 video sequences are from three sports: football, basketball, and volleyball. The dataset is divided into training, validation, and test subsets, consisting of 45, 45, and 150 video sequences, respectively. The training and validation sets also include manually annotated conditions for all targets in each frame, including which frame the target appears in. The target is identified by assigning a unique ID to the object belonging to the track. The position of the target's bounding box in the two-dimensional image coordinates. The position is represented by the upper left corner and the width and height of the bounding box. The detection confidence score of the target. Target category and visibility. The SportsMOT dataset is divided into a training set and a test set with a ratio of 8:2. Collect the τ-frame historical position information of each target. The position information is in the form of (top, left, weight, height), which represents the upper left corner coordinates of the target's bounding box in the video frame and the width and height of the bounding box, respectively.

[0067] Step 2: Input the historical position of the target trajectory into the target-driven network, where the target-driven network includes an LSTM encoder unit, a coarse target estimation unit, a target position prediction unit based on CVAE, and a precise target estimation unit with random noise and attention mechanism; the LSTM encoder unit can extract the temporal information in the past τ frames; the coarse target estimation unit can generate rough target position information based on the hidden state and cell state output by the LSTM encoder, and assist in the precise estimation of the target position in a coarse-to-fine manner; then, the target position prediction unit based on CVAE in the generative network is used to make precise target position predictions under normal circumstances; finally, in order to prevent the mistracking of the target at a certain moment in complex scenarios, resulting in the input of the target position at a certain time step in the input sequence not being the real input, resulting in the deviation of the target prediction position, the precise target position prediction unit based on random noise and multi-head attention mechanism is used to accurately estimate the target position. The specific steps are as follows:

[0068] Step 2.1: The LSTM encoding unit uses the latent vector X of the target's continuous position information in the past τ frames t =(x t-τ+1 ,x t-τ ...,x t ) as input, input feature x t The target information output by the rough target estimation unit in the previous time step t-1 Connect together and combine the hidden state of the previous time step With cell state C t-1 Then it is updated to a new hidden state through the LSTM encoder unit: and cell state C t , for the first time step, the hidden state Cell state and target information are set to zero.

[0069] Step 2.2: In order to provide guidance for precise target estimation, a coarse target position prediction unit is introduced. The main idea is to generate rough target position information to assist the precise estimation of the target position in a coarse-to-fine manner.

[0070] The rough target estimation unit includes an LSTM decoder, a multi-head attention mechanism, and a fully connected layer. First, the hidden state from the LSTM encoder is input and cell status The internal LSTM decoder and fully connected layers generate a rough target position prediction At the same time, the decoder input is generated through the fully connected layer Generate target information through multi-head attention mechanism and fully connected layer As complementary input to the LSTM encoder.

[0071] like Figure 2 As shown, and is the hidden state and cell state from the LSTM encoder, which is decoded by the internal LSTM decoder to produce the decoded hidden state and cell status Hidden state after decoding and cell status Generate the target information of the next time step through the fully connected layer and As the supplementary input of LSTM encoder and LSTM decoder respectively, the decoded cell state The query vector Q, key vector K and value vector V related to the multi-head attention mechanism are extracted through the fully connected layer, and finally the rough target position prediction is obtained through the multi-head attention mechanism and then through a fully connected layer.

[0072] Specifically, first the hidden state from the LSTM encoder and cell status The decoded hidden state is generated by the LSTM decoder of the coarse target position prediction unit and cell status Generate the supplementary input of the LSTM encoder for the next time step through the fully connected layer and the input of the LSTM decoder

[0073]

[0074] FC stands for fully connected layer.

[0075] Rough target position prediction After the multi-head attention mechanism, we get:

[0076]

[0077] W Q is the weight matrix of the fully connected layer that generates the query vector Q, W K is the weight matrix and W of the fully connected layer that generates the key vector K V The weight matrix of the fully connected layer that produces the value vector V. Multi_Attention represents the multi-head attention mechanism. and is the final output of the coarse target estimation unit.

[0078] Step 2.3: In order to make accurate target position prediction in general cases, the conditional autoencoding unit (CVAE) in the generative network is introduced.

[0079] like Figure 3 As shown, CVAE learns to observe the trajectory X by introducing a latent variable z t is the initial hidden state of the conditional LSTM decoder and initial cell state The conditional automatic encoding unit mainly consists of four parts: recognition network, prior network, generation network, and LSTM decoder; the recognition network, prior network and generation network are all composed of multi-layer perceptrons.

[0080] Figure 3 is the structural diagram of the CVAE target position prediction unit, X t is the hidden state from the LSTM encoder, Y t is the future true target position from the input, the output of the coarse target estimation unit After the fully connected layer, goal_for_dec is obtained as the input vector of the LSTM decoder.

[0081] ⊙ represents the selection symbol (the training process selects z Q , test and reasoning process selection z P ), and are the hidden states and cell states provided to the LSTM decoder. With KLD(z Q ,zP ) is the final output of the CVAE target position prediction unit.

[0082] First, the hidden state from the LSTM encoder Generate an input vector containing the target's historical position information through the fully connected layer Then, the common input vector X of the prior network and the recognition network is generated through GRU (Gated Recurrent Unit) P , as the input of the recognition network and the prior network; the true position Y of the input feature t (Known) Generate the input vector Y of the recognition network through the fully connected layer Q , as the input of the recognition network; X P and Y Q Input recognition network to output recognition latent variable z Q :

[0083]

[0084] X P Generate a priori latent variables z through the prior network P :

[0085] z P =MLP(X P )

[0086] Here, MLP stands for Multi-Layer Perceptron. Represents a splicing operation. Q and z P They are all hidden variables. During the training process, it is necessary to minimize the distribution difference between them. The reasoning process only uses the prior hidden variable z P .X P is the common input vector for generating the prior network and the recognition network, Y Q is the input vector of the recognition network.

[0087] Then the hidden variable z is passed through the generative network to obtain the initial hidden state of the LSTM decoding unit and initial cell state The generated network includes a fully connected layer and two parallel multi-layer perceptrons, which are expressed as follows:

[0088] h z =FC(z)

[0089]

[0090] Where z is P and z P One of them is that during training, z=z Q , in the reasoning process z=z P ;hz It is the intermediate vector obtained by the fully connected layer, which is used to generate the initial hidden state of the LSTM decoding unit and initial cell state

[0091] Finally, the target trajectory is generated through the LSTM decoder:

[0092]

[0093] In order to measure z Q and z P The distribution difference between them is minimized, which is measured by the KLD divergence and optimized by the KLD loss function:

[0094]

[0095] During inference, the prior network is used to generate the prior latent variable z P , generate the initial hidden state of the LSTM decoding unit through the generative network and cell status By combining the CVAE generative network and the LSTM decoder unit, accurate target position prediction can be achieved in conventional scenarios to obtain the final predicted target position.

[0096] Step 2.4: In order to deal with the phenomenon of mistracking of the target at a certain moment in complex scenes with complex motion and similar appearance, resulting in the input of the target position at a certain time step in the input sequence not being the real input, resulting in the deviation of the target prediction position, this embodiment designs an accurate target position prediction unit based on random noise and multi-head attention mechanism. Figure 4 As shown, the hidden state of the LSTM encoder unit The vector enc_hidden containing the target's historical position information is obtained through the fully connected layer, and the output of the rough target estimation unit After the fully connected layer, we get the vector goal_for_p containing the target predicted position information.

[0097] Specifically, the precise target position prediction unit receives the vector enc_hidden containing the target historical position information and the vector goal_for_p containing the target predicted position information, and converts them into intermediate vectors through the fully connected layer. and

[0098]

[0099] A normally distributed random noise is used, which is defined as follows:

[0100] random_noise=α*N(0,I d )

[0101] Among them, random_noise is the randomly generated noise, α is the scaling weight, N(0,I d ) represents a normally distributed random vector with mean 0 and variance 1, and the dimension d is The dimensions of are the same. Figure 5 The figure shows the comparison of the target trajectory before and after adding noise. The box represents the trajectory information of the football player in 8 consecutive frames. The original trajectory aiming frame is offset by scaling the random noise (the blue aiming frame in the figure).

[0102] and Through concatenation and three fully connected layers, k Q containing target position features under different semantics are obtained respectively. k}, K={k1,k2…,k k} and V={v1,v2…,v k}. k is the number of different semantics artificially defined, that is, the final output and the intermediate output are increased by one dimension to represent the semantic information.

[0103]

[0104] Where W Q , W K and W V is a randomly initialized matrix from the fully connected layer, Represents a splicing operation, and random noise random_noise randomly scales the feature vector of a trajectory to generate the effect of a feature vector at the wrong position.

[0105] In order to obtain the target position prediction features with different semantics and their importance to the final target position prediction, a multi-head attention mechanism is used to obtain the weight score between them:

[0106]

[0107] in, is a scaling factor used to alleviate the gradient vanishing problem introduced by the softmax activation function; T is the matrix transpose.

[0108]

[0109] In order to fully extract the global dependencies of each fragment from multiple perspectives, the above formula is repeated to obtain h (the same number as k, both 8) different heads, namely head1, head2, …, headh The input of each head is a set of k features. In order to better integrate the information from multiple angles, h heads are spliced ​​to obtain the final trajectory prediction vector F:

[0110] F=concat9head1,head2,…,head h )

[0111] Finally, the target position prediction in the scenario of random noise is output by the multi-layer perceptron:

[0112]

[0113] Step 3: Use the root mean square error (RMSE) as the loss function to supervise the loss of CVAE and Attention position prediction unit. CVAE and Loss Attention ). At the same time, in order to ensure that the coarse target estimation unit predicts the accurate target position, the target prediction and the true value Y t The RMSE between the two is used to optimize the prediction of the rough target estimation unit (Loss global Finally, KL divergence loss (KLD) is added to optimize the prior network in the conditional autoencoder unit (Loss KLD ). Therefore, for each training example, the final loss is summarized as follows:

[0114]

[0115] Loss KLD =KLD(Q θ (z|X t ,Y t ),P υ (z|X t ))

[0116] Loss total =Loss Global +Loss CVAE +Loss KLD +Loss Attention

[0117] The different loss updates during training and testing are as follows Figure 6 As shown in the figure, 200 rounds of training are used, and the test set is immediately verified after each round of training. The horizontal axis represents the training round, and the vertical axis represents the loss size. In the left figure, LossGlobal, Loss CVAE, Loss Attention, and Loss KLD represent the loss on the training set respectively. global 、Loss CVAE、Loss Attention and Loss KLD The right figure shows Loss Global, Loss CVAE, Loss Attention, and Loss KLD, which represent the Loss on the test set. global 、Loss CVAE 、Loss Attention and Loss KLD The test set Loss KLD remains unchanged.

[0118] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A position prediction method for multi-target tracking based on random noise and attention mechanism, characterized in that: The following steps are involved: Step 1: Obtain multi-target tracking data; collect the target's τ-frame historical position information, including the position of the target's bounding box in the video frame in the two-dimensional image coordinates, and divide the data into a training set and a test set; Step 2: Input the historical position of the target trajectory into the target-driven network, where the target-driven network includes an LSTM encoder unit, a coarse target estimation unit, a target position prediction unit based on CVAE, and a precise target estimation unit with random noise and attention mechanism; The LSTM encoder unit extracts the temporal information of the target in the past τ frames and outputs the hidden state and cell state to the rough target estimation unit; The coarse target estimation unit generates coarse target position predictions through an internal LSTM decoder and a multi-head attention mechanism; The target position prediction unit based on CVAE includes a recognition network, a priori network, a generation network and an LSTM decoder. Specifically, the input vector containing the target historical position information is obtained through a gated recurrent unit to obtain a common input vector, and the common input vector is input into the priori network to generate a priori latent variables. At the same time, the common input vector and the target real position are input into the recognition network together to generate recognition latent variables. Then, the priori latent variables and the recognition latent variables are selected, and the recognition latent variables are selected during training, and the priori latent variables are selected during testing or inference. They are input into the generation network to generate an initial latent state and an initial cell state, which are then input into the LSTM decoder to generate accurate target position predictions in conventional scenarios. The precise target estimation unit of random noise and attention mechanism includes a multi-head attention mechanism and a multi-layer perceptron; first, the hidden state obtained by the LSTM encoder unit and the output of the rough target estimation unit are respectively passed through the fully connected layer to obtain a vector containing the target historical position information and a vector containing the target predicted position information, and then these two vectors are spliced, and after passing through the fully connected layer, a query vector, a key vector, and a value vector based on the attention mechanism are obtained, where the value vector contains random noise; then, after passing through the multi-head attention mechanism, the final trajectory prediction vector is obtained, and the target position prediction in the scenario of random noise output is output by the multi-layer perceptron; Step 3: Use the collected data to train and test the target-driven network to obtain a trained target-driven network model, and apply the model to actual target position prediction reasoning.

2. According to claim 1, a method for position prediction in multi-target tracking based on random noise and attention mechanism is characterized in that: The specific workflow of the rough target estimation unit is as follows: First, the hidden state and cell state from the LSTM encoder are decoded through the LSTM decoder to generate the decoded hidden state and cell status Generate the supplementary input of the LSTM encoder unit for the next time step through the fully connected layer and the input of the LSTM decoder Among them, FC represents the fully connected layer; Then, the query vector Q, key vector K, and value vector V based on the attention mechanism are obtained through the fully connected layer: Among them, W Q is the weight matrix of the fully connected layer that generates the query vector Q, W K is the weight matrix of the fully connected layer that generates the key vector K, W V is the weight matrix of the fully connected layer that produces the value vector V; Finally, a rough target position prediction is obtained through the multi-head attention mechanism Among them, Multi_Attention represents the multi-head attention mechanism.

3. According to claim 2, a method for position prediction in multi-target tracking based on random noise and attention mechanism is characterized in that: The specific process of the target position prediction unit based on CVAE is as follows: First, the hidden state from the LSTM encoder passes through the fully connected layer to generate an input vector containing the target historical position information, and then passes through the gated recurrent unit to generate the common input vector X of the prior network and the recognition network. P , input the true position Y of the feature t Generate the input vector Y of the recognition network through the fully connected layer Q ;X P and Y Q Input recognition network to output recognition latent variable z Q : X P Generate a priori latent variables z through the prior network P : With P =MLP(X P ) Among them, MLP stands for multi-layer perceptron, Represents a splicing operation; Then the prior latent variable z is selected by the selection operation P Or identify the latent variable z Q As the hidden variable z, the initial hidden state of the LSTM decoder is obtained by generating the network and initial cell state The generated network includes a fully connected layer and two parallel multi-layer perceptrons, which are expressed as follows: h z =FC(z) Where z is P and z P One of them is that during training, z=z Q , in the test or reasoning process z = z P ;h z is the intermediate vector obtained by the fully connected layer; Finally, the LSTM decoder generates accurate target position predictions in common scenarios Where goal_for_dec is the output of the coarse target estimation unit Obtained through the fully connected layer.

4. According to claim 3, a method for position prediction in multi-target tracking based on random noise and attention mechanism is characterized in that: Measure z by KLD divergence Q and z P The distribution difference between them is minimized and optimized by the KLD loss function:

5. According to claim 4, a method for position prediction in multi-target tracking based on random noise and attention mechanism is characterized in that: The specific process of the precise target estimation unit based on random noise and attention mechanism is as follows: First, the hidden state of the LSTM encoder unit is passed through the fully connected layer to obtain the vector enc_hidden containing the target historical position information, and the output of the rough target estimation unit After the fully connected layer, we get the vector goal_for_p containing the target prediction position information; The vector enc_hidden containing the target historical position information and the vector goal_for_p containing the target predicted position information are converted into intermediate vectors through the fully connected layer. and and The query vectors Q = {q1, q2…, q k }, key vector K = {k1, k2…, k k } and a vector of values ​​V = {v1,v2…,v k }, k is the number of different semantics specified artificially, that is, the final output and the intermediate output are increased by one dimension to represent the semantic information. The process is expressed as follows: Where W Q , W K and W V is a randomly initialized matrix from the fully connected layer, Represents a concatenation operation, and random noise random_noise randomly scales the feature vector to generate the effect of a feature vector in the wrong position; Then the final target position is predicted through the multi-head attention mechanism, and the calculation of each head is as follows: in, is the scaling factor, T is the matrix transpose; thus, h different heads are obtained, namely head1, head2, …, head h , the input of each head is a set of k features; the h heads are concatenated to obtain the final trajectory prediction vector F: F=concat(head1,head2,...,head h ) Finally, the target position prediction in the scene with random noise is output by the multi-layer perceptron 6. The position prediction method in multi-target tracking based on random noise and attention mechanism according to claim 5 is characterized in that: When training the target-driven network, the root mean square error is used as the loss function to supervise the loss of the CVAE and Attention position prediction units; for each training sample, the final loss is as follows: Loss total =Loss Global +Loss CVAE +Loss KLD +Loss Attention Among them, Loss total Represents the final loss function, Loss Global Represents the loss function of the predicted value and the true value of the rough target estimation unit, Loss CVAE Represents the loss function of the rough target estimation unit, Loss Attention Represents the loss function of the accurate target estimation unit, Loss KLD represents the KLD loss function.

Images