A position prediction method in multi-target tracking based on random noise and attention mechanism

Abstract

The application provides a position prediction method based on random noise and attention mechanism in multi-target tracking, and belongs to the field of multi-target tracking.In the technical scheme, the traditional Kalman filtering method is replaced by constructing a target driving network, the target driving network comprises an LSTM encoder unit, a rough target estimation unit, a target position prediction unit based on CVAE and an accurate target estimation unit based on random noise and attention mechanism; the target position prediction unit based on CVAE is used for target position prediction in a non-mismatching scene, and the accurate target position prediction unit based on random noise and multi-head attention mechanism is used for target position prediction in a mismatching scene.The deviation of target position prediction in an actual complex scene is effectively reduced, the number of target mismatches is reduced, and the accuracy of multi-target tracking in an actual complex scene is obviously improved.

Description

Technical Field

[0001] The present invention belongs to the field of multi-target tracking, and in particular relates to a position prediction method in multi-target tracking based on random noise and attention mechanism. Background Art

[0002] Multi-Object Tracking (MOT) is an important research direction in the field of computer vision. Its main task is to predict the position of the target in the subsequent time frame and its corresponding relationship based on the target detection results in the video or image sequence. In the existing technology, Kalman filter is a widely used target position prediction method. Kalman filter is a recursive least squares estimation algorithm based on linear dynamic model and Gaussian distribution assumption. Its core is to combine the current observation value and the predicted value to predict the target position through the state estimation equation and the state update equation. However, in actual complex scenarios, Kalman filter has the following technical problems:

[0003] 1. Insufficient ability to model nonlinear dynamics: The Kalman filter assumes that target motion conforms to a linear dynamic model, an assumption that is difficult to establish in complex real-world scenarios. Targets in complex scenes often exhibit nonlinear and diverse motion patterns, such as sudden acceleration, steering, or occlusion. The Kalman filter struggles to effectively model these nonlinear motions, resulting in large prediction errors.

[0004] 2. Inadequate robustness to data noise and occlusion. In multi-target tracking tasks, the target positions output by the detector may contain noise, or even mismatches or occlusions. Kalman filtering relies on the Gaussian distribution assumption and cannot adequately handle these complex noise interferences, thus reducing the robustness and accuracy of predictions.

[0005] 3. Unable to model the uncertainty of target location: The Kalman filter outputs a deterministic prediction value, which fails to capture the probabilistic distribution characteristics of the target location. This makes it difficult to provide more potential trajectory information when there is significant uncertainty in the target location, limiting the flexibility of subsequent decision-making. Summary of the Invention

[0006] In order to solve the problem of large position prediction errors in complex scenarios, the present invention introduces a long short-term memory network (LSTM) to extract target position features, which can better extract dynamic features in historical trajectories and enhance the nonlinear motion modeling capability.

[0007] In order to deal with the mismatching problem in actual complex scenarios, the present invention proposes an accurate target position prediction unit based on random noise and attention mechanism, which can simulate mismatching scenarios through random noise, thereby improving the robustness of the model in complex scenarios and reducing the impact of mismatching on target position prediction.

[0008] In order to model the uncertainty of target position in actual complex scenarios, this paper constructs a conditional variational autoencoder (CVAE) in the target position prediction scenario to model the probability distribution of the target position, and uses historical trajectory information as a condition to generate multiple possible target position distributions to fully capture the uncertainty of the target position.

[0009] To solve the above technical problems, the specific technical solution of the present invention is a method for position prediction in multi-target tracking based on random noise and attention mechanism as follows:

[0010] Step 1: Obtain multi-target tracking data; collect the target's τ-frame historical position information, including the position of the target's bounding box in the video frame in the two-dimensional image coordinates, and divide the data into training and test sets;

[0011] Step 2: Input the historical position of the target trajectory into the target-driven network, which includes an LSTM encoder unit, a coarse target estimation unit, a target position prediction unit based on CVAE, and a precise target estimation unit with random noise and attention mechanism.

[0012] The LSTM encoder unit extracts the temporal information of the target in the past τ frames and outputs the hidden state and cell state to the coarse target estimation unit;

[0013] The coarse target estimation unit generates coarse target position predictions through an internal LSTM decoder and a multi-head attention mechanism;

[0014] The target position prediction unit based on CVAE includes a recognition network, a priori network, a generative network and an LSTM decoder. Specifically, the input vector containing the target's historical position information is converted into a common input vector through a gated recurrent unit. The common input vector is input into the priori network to generate a priori latent variable. At the same time, the common input vector and the target's true position are input into the recognition network together to generate a recognition latent variable. Then, the priori latent variable and the recognition latent variable undergo a selection operation. The recognition latent variable is selected during training, and the prior latent variable is selected during testing or inference. The variables are input into the generative network to generate an initial latent state and an initial cell state, which are then input into the LSTM decoder to generate an accurate target position prediction in a conventional scenario.

[0015] The precise target estimation unit with random noise and attention mechanism includes a multi-head attention mechanism and a multi-layer perceptron. First, the hidden state of the LSTM encoder unit and the output of the coarse target estimation unit are respectively passed through a fully connected layer to obtain a vector containing the target's historical position information and a vector containing the target's predicted position information. These two vectors are then concatenated and passed through a fully connected layer to obtain the query vector, key vector, and value vector based on the attention mechanism. The value vector contains random noise. The final trajectory prediction vector is then obtained through the multi-head attention mechanism, and the target position prediction in the scenario with random noise is output by the multi-layer perceptron.

[0016] Step 3: Use the collected data to train and test the target-driven network to obtain a trained target-driven network model, and apply the model to actual target position prediction reasoning.

[0017] The specific workflow of the rough target estimation unit is as follows:

[0018] First, the hidden state and cell state from the LSTM encoder are generated through the LSTM decoder to generate the decoded hidden state and cell status Generate the supplementary input of the LSTM encoder unit for the next time step through the fully connected layer and the input of the LSTM decoder

[0019]

[0020] Among them, FC represents the fully connected layer;

[0021] Then, the query vector Q, key vector K, and value vector V based on the attention mechanism are obtained through the fully connected layer:

[0022]

[0023] Among them, W Q is the weight matrix of the fully connected layer that generates the query vector Q, W K is the weight matrix of the fully connected layer that generates the key vector K, W V is the weight matrix of the fully connected layer that produces the value vector V;

[0024] Finally, a rough target position prediction is obtained through the multi-head attention mechanism

[0025]

[0026] Among them, Multi_Attention represents the multi-head attention mechanism.

[0027] The specific process of the target position prediction unit based on CVAE is as follows:

[0028] First, the hidden state from the LSTM encoder passes through the fully connected layer to generate an input vector containing the target historical position information, and then passes through the gated recurrent unit to generate the common input vector X for the prior network and the recognition network. P , the true position Y of the input feature t Generate the input vector Y of the recognition network through the fully connected layer Q ;X P and Y Q Input recognition network to output recognition latent variable z Q :

[0029]

[0030] X P Generate the prior latent variable z through the prior network P :

[0031] z P =MLP(X P )

[0032] Among them, MLP stands for multi-layer perceptron, Represents a splicing operation;

[0033] Then the prior latent variable z is selected by the selection operation P Or identify the latent variable z Q As the hidden variable z, the initial hidden state of the LSTM decoder is obtained by generating the network and initial cell state The generative network consists of a fully connected layer and two parallel multi-layer perceptrons, which are expressed as follows:

[0034] h z =FC(z)

[0035]

[0036] Where z is z P and z P One of them is that during training, z=z Q , z=z during testing or reasoning P ;h z is the intermediate vector obtained by the fully connected layer;

[0037] Finally, the LSTM decoder generates accurate target position predictions in common scenarios

[0038]

[0039] Among them, goal_for_dec is the output of the coarse target estimation unit Obtained through the fully connected layer.

[0040] The specific process of the precise target estimation unit based on random noise and attention mechanism is as follows:

[0041] First, the hidden state of the LSTM encoder unit is passed through the fully connected layer to obtain the vector enc_hidden containing the target historical position information, and the output of the rough target estimation unit After the fully connected layer, we get the vector goal_for_p containing the target predicted position information;

[0042] The vector enc_hidden containing the target historical position information and the vector goal_for_p containing the target predicted position information are converted into intermediate vectors through the fully connected layer. and

[0043]

[0044] and The query vectors Q={q1,q2…,q k}、Key vector K={k1,k2…,k k} and a value vector V containing random noise = {v1,v2…,v k}, k is the number of different semantics specified artificially, that is, the final output and the intermediate output are increased by one dimension to represent the semantic information. The process is expressed as follows:

[0045]

[0046] Where W Q 、W K and W V is a randomly initialized matrix from the fully connected layer, Represents the splicing operation, random noise random_noise randomly scales the feature vector to generate the effect of the feature vector in the wrong position;

[0047] The final target position is then predicted through a multi-head attention mechanism, and the calculation of each head is as follows:

[0048]

[0049] in, is the scaling factor, T is the matrix transpose; thus, h different heads are obtained, namely head1, head2,…, head h, the input of each head is a set of k features; the h heads are concatenated to obtain the final trajectory prediction vector F:

[0050] F=concat(head1,head2,…,head h )

[0051] Finally, the target position prediction in the scene with random noise is output by the multi-layer perceptron

[0052]

[0053] When training the target-driven network, the root mean square error is used as the loss function to supervise the loss of the CVAE and Attention position prediction units; for each training sample, the final loss is as follows:

[0054]

[0055] Loss total =Loss Global +Loss CVAE +Loss KLD +Loss Attention

[0056] Among them, Loss total Represents the final loss function, Loss Global Represents the loss function of the predicted value and the true value of the rough target estimation unit, Loss CVAE Represents the loss function of the rough target estimation unit, Loss Attention Represents the loss function of the accurate target estimation unit, Loss KLD represents the KLD loss function.

[0057] The technical solution provided by this invention replaces the traditional Kalman filter method by constructing a target-driven network. The CVAE target position prediction unit performs target position prediction in non-mismatch scenarios, while the precise target position prediction unit based on random noise and a multi-head attention mechanism performs target position prediction in mismatch scenarios. This effectively reduces the deviation in target position prediction in complex real-world scenarios. This, in turn, reduces the number of mismatches between targets, significantly improving the accuracy of multi-target tracking in these complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of the structure of a target-driven network provided by an embodiment of the present invention;

[0059] Figure 2 Estimate the unit structure diagram for rough targets;

[0060] Figure 3 Structure diagram of CVAE target position prediction unit

[0061] Figure 4 Structure diagram of precise target position prediction unit with attention mechanism and random noise

[0062] Figure 5 Comparison diagram of player trajectory before and after adding noise trajectory

[0063] Figure 6 Loss iteration diagram in training process and test process DETAILED DESCRIPTION

[0064] In order to better understand the purpose, structure and function of the present application, a kind of position prediction method in multi-target tracking based on random noise and attention mechanism will be further described in detail in combination with the drawings.

[0065] As shown in Figure 1 The target driving network of the present application includes: LSTM encoder unit, rough target estimation unit based on attention mechanism, target position prediction unit based on CVAE and random noise, and precise target estimation unit based on attention mechanism and random noise. The input of the LSTM encoder unit is the latent vector X t ={x t-τ+1 ,x t-τ+2 ,...,x t}, wherein x t represents the bounding box information, and τ is the artificially specified time step of input. The output contains hidden state and cell state C t of historical position features. At time step t, the rough target estimation unit receives the output from the LSTM encoder unit, outputs rough target position prediction and decoder input and the supplementary input of next time step (t+1) LSTM encoder The target position prediction unit based on CVAE accepts the hidden state from the LSTM encoder unit as The output of the full connection layer and the output of the rough target estimation unit Through full connection layer, goal_for_dcc as input, cooperate with LSTM decoder to output precise target position prediction The precise target estimation unit with random noise and attention mechanism accepts the hidden state from the LSTM encoder as The output of the full connection layer enc_hidden, random noise random_noise and the output of the rough target estimation unit The output goal_for_p of the fully connected layer is used as input to output the target position prediction in the noise scene. Next, the method provided in this embodiment is further described.

[0066] Step 1: Obtain multi-target tracking data. The evaluation dataset used is SportsMOT, which contains 240 video sequences from three sports: football, basketball, and volleyball. The dataset is divided into training, validation, and test subsets, consisting of 45, 45, and 150 video sequences, respectively. The training and validation sets also include manually annotated information about all objects in each frame, including the frame in which the object appears. Objects are assigned unique IDs to identify them as belonging to a track. The position of the object's bounding box in 2D image coordinates is represented by the top-left corner and the width and height of the bounding box. The detection confidence score of the object is also displayed. The object category and visibility are also displayed. The SportsMOT dataset is split into training and test sets with an 8:2 ratio. The historical position information of each object is collected over τ frames. The position information is in the form of (top, left, weight, height), which represents the coordinates of the top-left corner and the width and height of the bounding box of the object in the video frame, respectively.

[0067] Step 2: Input the historical position of the target trajectory into the target-driven network, where the target-driven network includes an LSTM encoder unit, a coarse target estimation unit, a target position prediction unit based on CVAE, and a precise target estimation unit with random noise and attention mechanism; the LSTM encoder unit can extract temporal information in the past τ frames; the coarse target estimation unit can generate coarse target position information based on the hidden state and cell state output by the LSTM encoder, assisting in the precise estimation of the target position in a coarse-to-fine manner; then, the target position is accurately predicted under normal circumstances through the CVAE-based target position prediction unit in the generative network; finally, in order to prevent the target from being mistracked at a certain moment in complex scenarios, resulting in the input of the target position at a certain time step in the input sequence not being the real input, resulting in the deviation of the target prediction position, the precise target position prediction unit based on random noise and multi-head attention mechanism is applied to accurately estimate the target position. The specific steps are as follows:

[0068] Step 2.1: The LSTM encoding unit uses the latent vector X of the target's continuous position information in the past τ frames t =(x t-τ+1 ,x t-τ ...,x t ) as input, input feature x t The target information output by the rough target estimation unit in the previous time step t-1 Connect together and combine the hidden state of the previous time step With cell state C t-1 Then it is updated to the new hidden state through the LSTM encoder unit and cell state C t , for the first time step, the hidden state Cell state and target information are set to zero.

[0069] Step 2.2: In order to provide guidance for precise target estimation, a coarse target position prediction unit is introduced. Its main idea is to generate coarse target position information to assist the precise estimation of the target position in a coarse-to-fine manner.

[0070] The rough target estimation unit includes LSTM decoder, multi-head attention mechanism, and fully connected layer. First, the hidden state from the LSTM encoder is input and cell status Generate rough target position prediction through internal LSTM decoder and fully connected layer At the same time, the decoder input is generated through the fully connected layer Generate target information through multi-head attention mechanism and fully connected layer Serves as a complementary input to the LSTM encoder.

[0071] like Figure 2 As shown, and is the hidden state and cell state from the LSTM encoder, and the decoded hidden state is generated by the internal LSTM decoder and cell status Hidden state after decoding and cell status Generate the target information of the next time step through the fully connected layer and As the supplementary input of LSTM encoder and LSTM decoder respectively, the decoded cell state The query vector Q, key vector K and value vector V related to the multi-head attention mechanism are extracted through the fully connected layer, and finally the rough target position prediction is obtained through the multi-head attention mechanism and then through a fully connected layer.

[0072] Specifically, first the hidden state from the LSTM encoder and cell status The decoded hidden state is generated by the LSTM decoder of the coarse target position prediction unit and cell status Generate the supplementary input of the LSTM encoder for the next time step through the fully connected layer and the input of the LSTM decoder

[0073]

[0074] FC stands for fully connected layer.

[0075] Coarse target position prediction After the multi-head attention mechanism, we get:

[0076]

[0077] W Q is the weight matrix of the fully connected layer that generates the query vector Q, W K is the weight matrix and W of the fully connected layer that produces the key vector K V The weight matrix of the fully connected layer that produces the value vector V, Multi_Attention represents the multi-head attention mechanism. and is the final output of the coarse target estimation unit.

[0078] Step 2.3: In order to perform accurate target position prediction in general cases, the conditional autoencoding unit (CVAE) in the generative network is introduced.

[0079] like Figure 3 As shown, CVAE learns to observe the trajectory X by introducing the latent variable z t The initial hidden state of the conditional LSTM decoder and initial cell state The conditional automatic encoding unit mainly consists of four parts: recognition network, prior network, generation network, and LSTM decoder; the recognition network, prior network and generation network are all composed of multi-layer perceptrons.

[0080] Figure 3 It is the structural diagram of the CVAE target position prediction unit, X t is the hidden state from the LSTM encoder, Y t is the future true target position from the input, the output of the coarse target estimation unit After the fully connected layer, goal_for_dec is obtained as the input vector of the LSTM decoder.

[0081] ⊙ represents the selection symbol (the training process selects z Q , testing and reasoning process selection z P ), and are the hidden state and cell state provided to the LSTM decoder. With KLD(z Q ,zP ) is the final output of the CVAE target position prediction unit.

[0082] First, the hidden state from the LSTM encoder Generate an input vector containing the target's historical position information through the fully connected layer Then, the common input vector X of the prior network and the recognition network is generated by GRU (Gated Recurrent Unit) P , as the input of the recognition network and the prior network; the true position Y of the input feature t (Known) Generate the input vector Y of the recognition network through the fully connected layer Q , as the input of the recognition network; X P and Y Q Input recognition network to output recognition latent variable z Q :

[0083]

[0084] X P Generate the prior latent variable z through the prior network P :

[0085] z P =MLP(X P )

[0086] Here, MLP stands for Multi-Layer Perceptron. Represents a splicing operation. Q and z P They are all latent variables. During the training process, the distribution difference between them needs to be minimized. The reasoning process only uses the prior latent variable z P .X P is the common input vector for generating the prior network and the recognition network, Y Q is the input vector of the recognition network.

[0087] Then the hidden variable z is used to generate the initial hidden state of the LSTM decoding unit through the generative network and initial cell state The generative network consists of a fully connected layer and two parallel multi-layer perceptrons, which are expressed as follows:

[0088] h z =FC(z)

[0089]

[0090] Where z is z P and z P One of them is that during training, z=z Q , in the reasoning process z=z P ;hz It is the intermediate vector obtained by the fully connected layer, which is used to generate the initial hidden state of the LSTM decoding unit and initial cell state

[0091] Finally, the target trajectory is generated through the LSTM decoder:

[0092]

[0093] In order to measure z Q and z P The distribution difference between them and minimizes the difference, which is measured by KLD divergence and optimized by KLD loss function:

[0094]

[0095] During inference, the prior network is used to generate the prior latent variable z P , generate the initial hidden state of the LSTM decoding unit by generating the network and cell status By combining the CVAE generative network and the LSTM decoder unit, accurate target position prediction can be achieved in conventional scenarios, and the final predicted target position can be obtained.

[0096] Step 2.4: In order to cope with the phenomenon of mistracking of the target at a certain moment in complex scenes with complex motion and similar appearance, which results in the input of the target position at a certain time step in the input sequence not being the real input, resulting in the deviation of the target predicted position, this embodiment designs an accurate target position prediction unit based on random noise and multi-head attention mechanism. Figure 4 As shown, the hidden state of the LSTM encoder unit The vector enc_hidden containing the target's historical position information is obtained through the fully connected layer, and the output of the rough target estimation unit After the fully connected layer, the vector goal_for_p containing the target predicted position information is obtained.

[0097] Specifically, the precise target position prediction unit receives the vector enc_hidden containing the target historical position information and the vector goal_for_p containing the target predicted position information, and converts them into intermediate vectors through the fully connected layer. and

[0098]

[0099] Normally distributed random noise is used, which is defined as follows:

[0100] random_noise=α*N(0,I d )

[0101] Among them, random_noise is the randomly generated noise, α is the scaling weight, N(0,I d ) represents a normally distributed random vector with mean 0 and variance 1, and the dimension d is The dimensions of are the same. Figure 5 The figure shows the comparison of the target trajectory before and after adding noise. The box represents the trajectory information of the football player in 8 consecutive frames. The original trajectory aiming box is offset by scaling the random noise (the blue aiming box in the figure).

[0102] and By splicing and passing through three fully connected layers, k different semantic features containing target position features are obtained, namely Q = {q1,q2…,q k}、K={k1,k2…,k k} and V={v1,v2…,v k k is an artificially defined number of different semantics, that is, the final output and the intermediate output are both increased by one dimension to represent semantic information.

[0103]

[0104] Where W Q 、W K and W V is a randomly initialized matrix from the fully connected layer, Represents a splicing operation. Random noise random_noise randomly scales the feature vector of a trajectory to generate the effect of a feature vector at the wrong position.

[0105] In order to obtain the target position prediction features with different semantics and their importance to the final target position prediction, a multi-head attention mechanism is used to obtain the weight score between them:

[0106]

[0107] in, is a scaling factor used to alleviate the gradient vanishing problem introduced by the softmax activation function; T is the matrix transpose.

[0108]

[0109] In order to fully extract the global dependencies of each fragment from multiple perspectives, the above formula is repeated to obtain h (the same number as k, both 8) different heads, namely head1, head2, ..., headh The input of each head is a set of k features. To better integrate the information from multiple angles, h heads are spliced ​​together to obtain the final trajectory prediction vector F:

[0110] F=concat(head1,head2,…,head h )

[0111] Finally, the target position prediction in the scenario where the multi-layer perceptron outputs random noise is:

[0112]

[0113] Step 3: Use the root mean square error (RMSE) as the loss function to supervise the loss of CVAE and Attention position prediction unit (Loss CVAE and Loss Attention ). At the same time, in order to ensure that the coarse target estimation unit predicts the accurate target position, the target prediction and the true value Y t RMSE between to optimize the prediction of the rough target estimation unit (Loss global Finally, KL divergence loss (KLD) is added to optimize the prior network in the conditional autoencoder unit (Loss KLD ). Therefore, for each training example, the final loss is summarized as follows:

[0114]

[0115] Loss KLD =KLD(Q θ (z|X t ,Y t ),P υ (z|X t ))

[0116] Loss total =Loss Global +Loss CVAE +Loss KLD +Loss Attention

[0117] Different loss updates during training and testing are as follows Figure 6 As shown in the figure, 200 rounds of training are used, and the test set is immediately verified after each round of training. The horizontal axis represents the training round, and the vertical axis represents the loss size. global 、Loss CVAE、Loss Attention and Loss KLD The right figure Loss Global, Loss CVAE, Loss Attention and Loss KLD represent the Loss on the test set. global 、Loss CVAE 、Loss Attention and Loss KLD The test set Loss KLD remains unchanged.

[0118] It will be understood that the present invention is described by way of some embodiments, and it will be appreciated by those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are intended to be protected by the present invention.

Claims

1. A position prediction method for multi-target tracking based on random noise and attention mechanism, characterized by: The following steps are involved: Step 1: Obtain multi-target tracking data; collect the target's τ-frame historical position information, including the position of the target's bounding box in the video frame in the two-dimensional image coordinates, and divide the data into training and test sets; Step 2: Input the historical position of the target trajectory into the target-driven network, where the target-driven network includes an LSTM encoder unit, a coarse target estimation unit, a target position prediction unit based on CVAE, and a precise target estimation unit with random noise and attention mechanism; The LSTM encoder unit extracts the temporal information of the target in the past τ frames and outputs the hidden state and cell state to the coarse target estimation unit; The coarse target estimation unit generates coarse target position predictions through an internal LSTM decoder and a multi-head attention mechanism; The target position prediction unit based on CVAE includes a recognition network, a priori network, a generative network and an LSTM decoder. Specifically, the input vector containing the target's historical position information is converted into a common input vector through a gated recurrent unit. The common input vector is input into the priori network to generate a priori latent variable. At the same time, the common input vector and the target's true position are input into the recognition network together to generate a recognition latent variable. Then, the priori latent variable and the recognition latent variable undergo a selection operation. The recognition latent variable is selected during training, and the prior latent variable is selected during testing or inference. The variables are input into the generative network to generate an initial latent state and an initial cell state, which are then input into the LSTM decoder to generate an accurate target position prediction in a conventional scenario. The precise target estimation unit with random noise and attention mechanism includes a multi-head attention mechanism and a multi-layer perceptron. First, the hidden state obtained by the LSTM encoder unit and the output of the rough target estimation unit are respectively passed through a fully connected layer to obtain a vector containing the target's historical position information and a vector containing the target's predicted position information. These two vectors are then concatenated and passed through a fully connected layer to obtain a query vector, a key vector, and a value vector based on the attention mechanism. The value vector contains random noise. The final trajectory prediction vector is then obtained through the multi-head attention mechanism, and the target position prediction in the scenario with random noise is output by the multi-layer perceptron. Step 3: Use the collected data to train and test the target-driven network to obtain a trained target-driven network model, and apply the model to actual target position prediction reasoning.

2. The position prediction method for multi-target tracking based on random noise and attention mechanism according to claim 1 is characterized in that: The specific workflow of the rough target estimation unit is as follows: First, the hidden state and cell state from the LSTM encoder are generated through the LSTM decoder to generate the decoded hidden state and cell status Generate the supplementary input of the LSTM encoder unit for the next time step through the fully connected layer and the input of the LSTM decoder Among them, FC represents the fully connected layer; Then, the query vector Q, key vector K, and value vector V based on the attention mechanism are obtained through the fully connected layer: Among them, W Q is the weight matrix of the fully connected layer that generates the query vector Q, W K is the weight matrix of the fully connected layer that generates the key vector K, W V is the weight matrix of the fully connected layer that produces the value vector V; Finally, a rough target position prediction is obtained through the multi-head attention mechanism Among them, Multi_Attention represents the multi-head attention mechanism.

3. The position prediction method for multi-target tracking based on random noise and attention mechanism according to claim 2 is characterized in that: The specific process of the target position prediction unit based on CVAE is as follows: First, the hidden state from the LSTM encoder passes through the fully connected layer to generate an input vector containing the target historical position information, and then passes through the gated recurrent unit to generate the common input vector X for the prior network and the recognition network. P , the true position Y of the input feature t Generate the input vector Y of the recognition network through the fully connected layer Q ;X P and Y Q Input recognition network to output recognition latent variable z Q : X P Generate the prior latent variable z through the prior network P : With P =MLP(X P ) Among them, MLP stands for multi-layer perceptron, Represents a splicing operation; Then the prior latent variable z is selected by the selection operation P Or identify the latent variable z Q As the hidden variable z, the initial hidden state of the LSTM decoder is obtained by generating the network and initial cell state The generative network consists of a fully connected layer and two parallel multi-layer perceptrons, which are expressed as follows: h z =FC(z) Where z is z P and z P One of them is that during training, z=z Q , z=z during testing or reasoning P ;h z is the intermediate vector obtained by the fully connected layer; Finally, the LSTM decoder generates accurate target position predictions in common scenarios Among them, goal_for_dec is the output of the coarse target estimation unit Obtained through the fully connected layer.

4. The position prediction method for multi-target tracking based on random noise and attention mechanism according to claim 3 is characterized in that: Measure z by KLD divergence Q and z P The distribution difference between them is minimized and optimized by the KLD loss function:

5. The position prediction method for multi-target tracking based on random noise and attention mechanism according to claim 4 is characterized in that: The specific process of the precise target estimation unit based on random noise and attention mechanism is as follows: First, the hidden state of the LSTM encoder unit is passed through the fully connected layer to obtain the vector enc_hidden containing the target historical position information, and the output of the rough target estimation unit After the fully connected layer, we get the vector goal_for_p containing the target predicted position information; The vector enc_hidden containing the target historical position information and the vector goal_for_p containing the target predicted position information are converted into intermediate vectors through the fully connected layer. and and The query vectors Q={q1,q2…,q k }、Key vector K={k1,k2…,k k } and a value vector V containing random noise = {v1,v2…,v k }, k is the number of different semantics specified artificially, that is, the final output and the intermediate output are increased by one dimension to represent the semantic information. The process is expressed as follows: Where W Q 、W K and W V is a randomly initialized matrix from the fully connected layer, Represents the splicing operation, random noise random_noise randomly scales the feature vector to generate the effect of the feature vector in the wrong position; The final target position is then predicted through a multi-head attention mechanism, and the calculation of each head is as follows: in, is the scaling factor, T is the matrix transpose; thus, h different heads are obtained, namely head1, head2,…, head h , the input of each head is a set of k features; the h heads are concatenated to obtain the final trajectory prediction vector F: F=concat(head1,head2,...,head h ) Finally, the target position prediction in the scene with random noise is output by the multi-layer perceptron 6. The method for position prediction in multi-target tracking based on random noise and attention mechanism according to claim 5, characterized in that: When training the target-driven network, the root mean square error is used as the loss function to supervise the loss of the CVAE and Attention position prediction units; for each training sample, the final loss is as follows: Loss total =Loss Global +Loss CVAE +Loss KLD +Loss Attention Among them, Loss total Represents the final loss function, Loss Global Represents the loss function of the predicted value and the true value of the rough target estimation unit, Loss CVAE Represents the loss function of the rough target estimation unit, Loss Attention Represents the loss function of the accurate target estimation unit, Loss KLD represents the KLD loss function.

Images