A target tracking method and device based on a multipath selection attention mechanism

Abstract

This invention discloses a target tracking method and apparatus based on a multi-path attention mechanism. The target tracking method based on the multi-path attention mechanism includes: predicting the category of the concatenated vector through a category classification network branch to obtain a classification loss L. cls Identity prediction is performed on the concatenated vector through an identity authentication network branch, resulting in the identity authentication loss L. id The regression loss L is obtained by predicting the bounding box of the concatenated vector through the bounding box of the regression network branch. reg According to classification loss L cls Authentication loss L id and regression loss L reg The total loss L is obtained total The tracking network is trained iteratively multiple times until the total loss L is reached. total The network converges, resulting in the trained tracking network. This network is then used to track targets in real-time footage. By modifying the network structure and loss function, the training effect is optimized, thereby enhancing the model's tracking accuracy and stability.

Description

Technical Field

[0001] This invention belongs to the field of target tracking, and more specifically, relates to a target tracking method and apparatus based on a multi-path selection attention mechanism. Background Technology

[0002] In recent years, with rapid economic development and continuous technological progress, artificial intelligence (AI) technology has received significant attention from various industries. Scenarios that previously required substantial human resources for monitoring can now be gradually replaced by machines. Research indicates that humans acquire over 70% of their information through vision; therefore, computer vision technology is crucial for the development of the AI ​​industry. Specifically, computer vision refers to using electronic imaging devices to replace the human eye in tasks such as target classification and recognition. In recent years, with the widespread availability of high-performance computing resources, deep learning technology has been explored for application in the field of computer vision.

[0003] Currently, multi-target tracking technology is gradually attracting attention from academia and industry due to its wide range of applications and numerous technical challenges. At present, multi-target tracking technology has seen initial applications in fields such as intelligent monitoring, autonomous driving, and traffic control. However, due to numerous unfavorable factors during the tracking process, such as frequent occlusion between targets, complex background interference in the scene, and scale and attitude changes during target movement, it is difficult to achieve stable tracking results. Summary of the Invention

[0004] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a target tracking method based on a multi-path selection attention mechanism. The aim is to optimize the training effect of the network by modifying the network structure and loss function, thereby enhancing the tracking accuracy and stability of the model. This solves the technical problems of trajectory fragmentation caused by frequent occlusion between targets in the tracking scenario, inaccurate target localization caused by complex background interference in the scene, and dynamic changes in the scale and posture of the target.

[0005] To achieve the above objectives, according to one aspect of the present invention, a target tracking method based on a multi-path selection attention mechanism is provided, the target tracking method based on the multi-path selection attention mechanism comprising:

[0006] Two adjacent input images are obtained from the dataset. A tracking network is used to extract and fuse features for each input image to obtain an aggregate vector corresponding to each input image. The two aggregate vectors are then concatenated to obtain a concatenated vector.

[0007] The concatenated vector is input into the category classification network branch, the identity verification network branch, and the bounding box pair regression network branch of the tracking network, respectively.

[0008] The category classification network branch is used to predict the category of the concatenated vector, resulting in a classification loss L. cls The identity prediction is performed on the concatenated vector through the identity authentication network branch, resulting in the identity authentication loss L. id The regression loss L is obtained by predicting the bounding box of the concatenated vector using the bounding box of the regression network branch. reg ;

[0009] According to the classification loss L cls The authentication loss L id and regression loss L reg The total loss L is obtained total The tracking network is trained iteratively multiple times until the total loss L is reached. total The training network converges, resulting in the trained tracking network.

[0010] The trained tracking network tracks targets in real-time footage.

[0011] Furthermore, calculate the total loss L according to Formula 1. total ;

[0012]

[0013] Where ω1 and ω2 are learnable parameters used to balance classification and regression tasks.

[0014] Further, the step of obtaining two adjacent input images from the dataset, using a tracking network to perform feature extraction and feature fusion on each input image to obtain an aggregate vector corresponding to each input image, and concatenating the two aggregate vectors to obtain a concatenated vector includes:

[0015] Two adjacent input images are obtained from the dataset, and a tracking network is used to extract features from each input image to obtain high-level and low-level features.

[0016] The high-level features and the low-level features are aggregated from top to bottom by upsampling, and the high-level features are passed to the low-level features to complete the fusion of the target's speech information.

[0017] Add a bottom-up aggregation path above the top-down aggregation path to pass the low-level features to the high-level features, complete the fusion of the target's location information, and obtain the aggregation vector corresponding to each frame of input image.

[0018] The high-level features include the semantic information of the target, and the low-level features include the location information and texture information of the target.

[0019] Furthermore, each of the category classification network branch, the authentication network branch, and the bounding box pair regression network branch includes at least one convolutional layer;

[0020] A multi-path attention mechanism is added after the convolutional layer of each branch; the multi-path attention mechanism includes a spatial attention module, a motion activation module, and a spatiotemporal attention module.

[0021] Furthermore, the classification loss L is obtained by predicting the category of the concatenated vector through the category classification network branch. cls include:

[0022] The concatenated vector is convolved by at least one convolutional layer in the category classification network branch to obtain the input vector.

[0023] The input vector is processed by a spatial attention module to obtain channel excitations;

[0024] The motion excitation module processes the input vector to obtain the motion excitation.

[0025] The spatiotemporal attention module processes the input vector to obtain spatiotemporal excitation;

[0026] The channel excitation is multiplied by weight α, the motion excitation is multiplied by weight β, and the spatiotemporal excitation is multiplied by weight λ. The excitations multiplied by weights are then summed by weight to obtain the predicted output vector. Here, α, β, and λ are the network parameters, which are adaptively and dynamically adjusted during the network training process.

[0027] The classification loss L is obtained based on the predicted output vector and the actual output vector. cls .

[0028] Furthermore, the identity prediction is performed on the concatenated vector through the identity verification network branch to obtain the identity verification loss L. id include:

[0029] The concatenated vector is convolved by at least one convolutional layer in the category classification network branch to obtain the input vector.

[0030] The input vector is processed by a spatial attention module to obtain channel excitations;

[0031] The motion excitation module processes the input vector to obtain the motion excitation.

[0032] The spatiotemporal attention module processes the input vector to obtain spatiotemporal excitation;

[0033] The channel excitation is multiplied by weight α, the motion excitation is multiplied by weight β, and the spatiotemporal excitation is multiplied by weight λ. The excitations multiplied by weights are then summed by weight to obtain the predicted output vector. Here, α, β, and λ are the network parameters, which are adaptively and dynamically adjusted during the network training process.

[0034] The authentication loss L is obtained based on the predicted output vector and the actual output vector. id .

[0035] Furthermore, the regression network branch predicts the bounding box of the concatenated vector using the bounding box to obtain the regression loss L. reg include:

[0036] The input vector is obtained by multiplying the concatenated vector, the predicted output vector output by the category classification network branch, and the predicted output vector output by the identity verification network branch.

[0037] The input vector is processed by a spatial attention module to obtain channel excitations;

[0038] The motion excitation module processes the input vector to obtain the motion excitation.

[0039] The spatiotemporal attention module processes the input vector to obtain spatiotemporal excitation;

[0040] The channel excitation is multiplied by weight α, the motion excitation is multiplied by weight β, and the spatiotemporal excitation is multiplied by weight λ. The excitations multiplied by weights are then summed by weight to obtain the predicted output vector. Here, α, β, and λ are the network parameters, which are adaptively and dynamically adjusted during the network training process.

[0041] The regression loss L is obtained based on the predicted output vector and the actual output vector. reg .

[0042] Furthermore, the process of processing the input vector through the spatial attention module to obtain the channel excitation includes:

[0043] The spatial attention module performs global average pooling on the feature map of each channel of the input vector to obtain the average value of all pixels. The average value is then convolved by a one-dimensional convolution with a k×k kernel to obtain the channel activation.

[0044] The process of processing the input vector through the motion excitation module to obtain motion excitation includes:

[0045] The motion excitation module performs channel dimensionality reduction on the input vector through a 2D convolution with kernel m, dividing the input vector into a first path vector and a second path vector. The first path vector is then subjected to a 2D convolution with kernel n, and the difference is calculated between the first path vector and the second path vector to obtain a difference vector. The difference vector is padded with zeros to obtain an intermediate vector. The intermediate vector is then subjected to average pooling and then subjected to a 2D convolution with kernel m to restore the original number of channels and generate motion excitation.

[0046] The process of processing the input vector through the spatiotemporal attention module to obtain the spatiotemporal excitation includes:

[0047] The input vector to the spatiotemporal attention module is split into two paths. One path is average pooled along the channel dimension and then convolved using a 3D convolution with kernel p. The convolved vector is multiplied by the other path vector to generate spatiotemporal excitation.

[0048] Furthermore, the dataset includes a public dataset and a self-collected dataset. The public dataset includes MOT17 or MOT20. The self-collected dataset is collected from one or more of crowded train stations, indoor shopping malls, and bustling commercial streets. The collected data includes both fixed-camera and moving-camera modes. The sample annotation format of the self-collected dataset is the same as that of the MOT17 dataset, and includes the target's location information, category information, and identity information.

[0049] According to another aspect of the present invention, a target tracking device based on a multiple-choice attention mechanism is provided, comprising at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, wherein the instructions, after being executed by the processor, are used to complete the target tracking method based on the multiple-choice attention mechanism described in the first aspect.

[0050] Overall, the technical solutions conceived in this invention have the following beneficial effects compared with the prior art: In this invention, a dynamically weighted loss function is used to promote balanced training of the network, optimize the training effect of the network, and improve the expressive power of the tracking model.

[0051] Furthermore, the channel stimulation generated by the spatial attention module enables the network to focus on learning target features and suppress complex background interference in the image; the motion stimulation module can generate motion stimulation of the target between adjacent video frames, helping the network to better learn the pose changes of pedestrian targets between frames; the spatiotemporal attention module can generate spatiotemporal stimulation, which helps the network to grasp the complete motion trajectory of the target in the time series, thereby solving problems such as trajectory fragmentation and identity switching caused by frequent occlusion between targets. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating a target tracking method based on a multi-path selection attention mechanism provided in an embodiment of the present invention.

[0053] Figure 2 This is a schematic diagram of a basic framework based on a multi-path selection attention mechanism provided in an embodiment of the present invention;

[0054] Figure 3 This is provided by the embodiments of the present invention. Figure 1 A detailed flowchart of step 10 is shown below;

[0055] Figure 4 This is a schematic diagram of the basic framework of a multi-attention mechanism provided in an embodiment of the present invention;

[0056] Figure 5 This is a schematic diagram of a specific process of a multi-attention mechanism provided in an embodiment of the present invention;

[0057] Figure 6 This is a schematic diagram of the structure of a target tracking device based on a multi-path selection attention mechanism provided in an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0059] To address the challenges of trajectory fragmentation caused by frequent occlusion between targets in tracking scenarios, inaccurate target localization due to complex background interference, and dynamic changes in target scale and pose, the Chained-Tracker algorithm was selected as the baseline after careful analysis. By modifying the network structure and loss function, the training effect of the network was optimized, thereby enhancing the tracking accuracy and stability of the model.

[0060] In this embodiment, a tracking network is first established. After the tracking network is built, it is trained end-to-end using labeled samples in a data-driven manner. After training, the model file of the tracking network can be obtained, and the target can be tracked through the trained tracking network.

[0061] Based on the improved tracking network, this invention proposes a target tracking method based on a multi-path selection attention mechanism to improve the accuracy and stability of target tracking in typical scenarios and optimize the robustness of the tracking model. The target can specifically be a pedestrian or other object.

[0062] Example 1:

[0063] This embodiment provides a target tracking method based on a multi-path selection attention mechanism, such as... Figure 1 and Figure 2 As shown, the target tracking method based on the multi-path selection attention mechanism includes:

[0064] Step 10: Obtain two adjacent input images from the dataset, use a tracking network to perform feature extraction and feature fusion on each input image to obtain the aggregate vector corresponding to each input image, and concatenate the two aggregate vectors to obtain the concatenated vector.

[0065] In this embodiment, the tracking network needs to undergo training and testing phases. The training phase mainly includes sample labeling, building the tracking network, and training the tracking algorithm. The testing phase mainly includes testing and optimizing the tracking network.

[0066] The dataset includes a public dataset and a self-collected dataset. The public dataset includes MOT17 or MOT20. The self-collected dataset is collected from one or more of the following locations: crowded train stations, indoor shopping malls, and bustling commercial streets. The collected data includes both fixed-camera and moving-camera modes. The sample annotation format of the self-collected dataset is the same as that of the MOT17 dataset, and includes the target's location information, category information, and identity information.

[0067] The self-collected dataset can also be collected in other densely populated areas.

[0068] After completing sample collection and annotation, the tracking network was constructed. The tracking network consists of three sub-modules: a feature extraction module, a feature fusion module, and three prediction branches (a category classification network branch, an identity verification network branch, and a bounding box pair regression network branch).

[0069] In one embodiment, the feature extraction module follows the original Chained-Tracker structure, using ResNet50 as the backbone network for feature extraction. Two adjacent frames in the video frame are used as samples and fed into the feature extraction module to extract relevant information such as the target's location, texture, and semantics.

[0070] In one embodiment, a bidirectional feature fusion network is used for the feature fusion module.

[0071] In one embodiment, a feature extraction module is used to extract features, and a feature fusion module is used to fuse the features to obtain an aggregate vector corresponding to each frame of the input image. Then, the two aggregate vectors are concatenated to obtain a concatenated vector.

[0072] Step 20: Input the spliced ​​vector into the category classification network branch, identity verification network branch, and bounding box pair regression network branch of the tracking network, respectively.

[0073] In one embodiment, each of the category classification network branch, the authentication network branch, and the bounding box pair regression network branch includes at least one convolutional layer; a multi-way attention mechanism is added after the convolutional layer of each branch; the multi-way attention mechanism includes a spatial attention module, a motion activation module, and a spatiotemporal attention module.

[0074] Among them, the channel stimulation generated by the spatial attention module enables the network to focus on learning target features and suppress complex background interference in the image; the motion stimulation module can generate motion stimulation of the target between adjacent video frames, helping the network to better learn the pose changes of pedestrian targets between frames; the spatiotemporal attention module can generate spatiotemporal stimulation, which can help the network grasp the complete motion trajectory of the target in the time series, thereby solving problems such as trajectory fragmentation and identity switching caused by frequent occlusion between targets.

[0075] Step 30: Perform category prediction on the concatenated vector through the category classification network branch to obtain the classification loss L. cls The identity prediction is performed on the concatenated vector through the identity authentication network branch, resulting in the identity authentication loss L. id The regression loss L is obtained by predicting the bounding box of the concatenated vector using the bounding box of the regression network branch. reg .

[0076] Among them, the classification loss L cls Category loss for classifying targets; authentication loss L id The authentication loss is calculated by identifying the target, as the same target will be assigned the same ID; the regression loss is L. reg The loss is used for bounding box prediction of the target. After the target is identified, a bounding box is generated to represent the location of the target. Bounding boxes in two adjacent images form a bounding box pair.

[0077] Classification loss L of the network cls Using focal loss as an expression, the authentication loss L id Also using the focal loss expression, the bounding box is used to express the regression loss L. regIt adopts the expression form of Balanced L1 loss.

[0078] Step 40: Based on the classification loss L cls The authentication loss L id and regression loss L reg The total loss L is obtained total The tracking network is trained iteratively multiple times until the total loss L is reached. total The network converges, resulting in the trained tracking network.

[0079] In one embodiment, considering that the network model designed in this embodiment involves multi-task learning, in order to ensure that the loss values ​​of the regression task and the classification task are balanced.

[0080] Calculate the total loss L according to Formula 1. total Formula 1 is as follows:

[0081]

[0082] Where ω1 and ω2 are learnable parameters used to balance classification and regression tasks.

[0083] In one embodiment, the tracking network is trained iteratively through multiple rounds according to steps 10-40 described above to update the classification loss L. cls The authentication loss L id and regression loss L reg And update the total loss L according to Formula 1. total Until the total loss L total Convergence, where the total loss L total Convergence refers to the total loss L total Less than the set loss threshold.

[0084] A dynamically weighted loss function is used to promote balanced training of the network, optimize the training effect of the network, and improve the expressive power of the tracking model.

[0085] Step 50: Track the target in the real-time video using the trained tracking network.

[0086] In one embodiment, the standard Adam optimizer is used to train the network for 150 epochs with an initial learning rate of 5e-5 and a batch size of 16 during training. After training, the final multi-object tracking model is obtained.

[0087] After training the tracking network, it is tested. Specifically, the trained tracking network is used to track real-time images captured by the sensor. The tracking network outputs bounding box pairs for each target between adjacent frames. When a target first appears, the algorithm assigns a target ID and maintains this ID in subsequent frames. Over time, a smooth curve is used to plot the target's historical motion trajectory until the target disappears from the monitored scene. If the historical motion trajectory output by the tracking network is substantially the same as the target's actual motion trajectory, it indicates that the tracking network has high accuracy, and the trained tracking network can be used to predict targets.

[0088] In real-world scenarios, real-time video feeds into a tracking network. The network tracks every two adjacent frames, outputting bounding box pairs of the target within those frames. As time progresses, the network obtains bounding box pairs for the target in (frame 1, frame 2), (frame 2, frame 3), (frame 3, frame 4), and so on. Connecting these trajectory segments yields the complete trajectory.

[0089] In this embodiment, the spatial attention module, motion activation module, and spatiotemporal attention module can respectively address adverse factors such as frequent pedestrian target occlusion, complex background interference, and dynamic changes in pedestrian scale and posture. The weights generated by the three modules are dynamically adjusted through adaptive weighting. By introducing an adaptive multipath attention mechanism, interference from complex backgrounds during network training is suppressed, and a spatiotemporal relationship is established for the target between frames. This improves the accuracy of pedestrian tracking, reduces the number of identity switching in the tracking trajectory, and enhances the robustness of the tracking model.

[0090] In one embodiment, such as Figure 3 As shown, step 10 specifically includes:

[0091] Step 101: Obtain two adjacent input images from the dataset, and use a tracking network to extract features from each input image to obtain high-level and low-level features.

[0092] The high-level features include the semantic information of the target, and the low-level features include the location information and texture information of the target.

[0093] Typically, high-level features in convolutional neural networks contain more semantic information about the target, while low-level features contain more location and texture information. For multi-target tracking tasks, the prediction network needs both location and semantic information of the target to achieve stable and accurate target recognition. Therefore, after the feature extraction network, a bidirectional feature fusion network is constructed to aggregate the deep texture information and shallow location information of the target.

[0094] Step 102: The high-level features and the low-level features are aggregated from top to bottom by upsampling, and the high-level features are passed to the low-level features to complete the fusion of the target's speech information.

[0095] Step 103: Add a bottom-up aggregation path above the top-down aggregation path to pass the low-level features to the high-level features, complete the fusion of the target's location information, and obtain the aggregation vector corresponding to each frame of input image.

[0096] After the feature fusion module, the two feature maps output by the backbone network are concatenated into vectors using a concatenation operation, and the concatenated feature vectors are then fed into the three prediction branches in the prediction network.

[0097] This invention employs a bidirectional feature fusion network. First, high-level features with low resolution and high semantic information are aggregated from top to bottom with low-level features with high resolution and low semantic information through upsampling, so that features at all scales contain rich target semantic information. Then, a bottom-up feature aggregation path is added on the basis of the top-down aggregation path, thereby transferring the position information of the lower level to the higher level features and completing the fusion of target position information.

[0098] In one embodiment, such as Figure 4 and Figure 5 As shown, step 20 specifically includes:

[0099] Step 201: Perform convolution processing on the concatenated vector through at least one convolutional layer in the category classification network branch to obtain the input vector.

[0100] Step 202: Process the input vector using the spatial attention module to obtain channel excitation.

[0101] Step 203: Process the input vector through the motion excitation module to obtain motion excitation.

[0102] Step 204: Process the input vector through the spatiotemporal attention module to obtain spatiotemporal excitation.

[0103] Step 205: Multiply the channel excitation by weight α, multiply the motion excitation by weight β, multiply the spatiotemporal excitation by weight λ, and perform a weighted summation of the excitations multiplied by weights to obtain the prediction output vector.

[0104] Here, α, β, and λ are the network parameters, which are adaptively and dynamically adjusted during the network training process.

[0105] Step 206: Obtain the classification loss L based on the predicted output vector and the actual output vector. cls .

[0106] In this embodiment, the three modules operate in parallel and are introduced into the residual module of the three output branches of the prediction network (i.e., the category classification network branch, the identity verification network branch, and the bounding box pair regression network branch) using an adaptive weighted summation method.

[0107] In one embodiment, in step 20, the identity prediction of the concatenated vector is performed through the authentication network branch to obtain the authentication loss L. id Specifically, this includes: convolving the concatenated vector through at least one convolutional layer in the category classification network branch to obtain an input vector; processing the input vector through a spatial attention module to obtain channel activation; processing the input vector through a motion activation module to obtain motion activation; processing the input vector through a spatiotemporal attention module to obtain spatiotemporal activation; multiplying the channel activation by a weight α, the motion activation by a weight β, and the spatiotemporal activation by a weight λ, and then performing a weighted sum of the activations multiplied by the weights to obtain a predicted output vector, where α, β, and λ are network parameters that are adaptively and dynamically adjusted during the network training process; and obtaining the authentication loss L based on the predicted output vector and the actual output vector. id .

[0108] In one embodiment, in step 20, the bounding box prediction of the concatenated vector using the bounding box of the regression network branch is used to obtain the regression loss L. reg Specifically, this includes: multiplying the concatenated vector, the predicted output vector output by the category classification network branch, and the predicted output vector output by the identity verification network branch to obtain an input vector; processing the input vector through a spatial attention module to obtain channel excitation; processing the input vector through a motion excitation module to obtain motion excitation; processing the input vector through a spatiotemporal attention module to obtain spatiotemporal excitation; multiplying the channel excitation by a weight α, the motion excitation by a weight β, and the spatiotemporal excitation by a weight λ; and then weighted summing the excitations multiplied by the weights to obtain a predicted output vector, where α, β, and λ are network parameters that are adaptively and dynamically adjusted during the network training process; and obtaining the regression loss L based on the predicted output vector and the actual output vector. reg .

[0109] In one embodiment, step 202, processing the input vector using the spatial attention module to obtain channel activation specifically includes: performing global average pooling on the feature map of each channel of the input vector using the spatial attention module to obtain the average value of all pixels; and convolving the average value using a one-dimensional convolution with a k×k kernel to obtain the channel activation. The feature vector output from the one-dimensional convolution, after being mapped by the sigmoid function, can generate channel activation between 0 and 1.

[0110] The spatial attention module includes global average pooling and one-dimensional convolutional layers. The value of k can be 5 or other values, without specific restrictions here.

[0111] In this embodiment, global average pooling is used to reduce the number of parameters, thereby reducing computational cost. Cross-channel interaction is achieved through a one-dimensional convolution with a k×k kernel. The kernel k of the one-dimensional convolution determines the range of channel interaction.

[0112] In one embodiment, in step 203, the process of processing the input vector through the motion excitation module to obtain motion excitation specifically includes: performing channel dimensionality reduction on the input vector through a two-dimensional convolution with kernel m, dividing the input vector after channel dimensionality reduction into a first path vector and a second path vector; performing a two-dimensional convolution operation with kernel n on the first path vector and then subtracting it from the second path vector to obtain a difference vector; padding the difference vector with zeros to obtain an intermediate vector; performing average pooling on the intermediate vector and then performing a two-dimensional convolution with kernel m again to restore the original number of channels and generate motion excitation.

[0113] Where m can be 1 or any other value, without specific restrictions. n can be 3 or any other value, without specific restrictions.

[0114] In one embodiment, in step 204, the process of processing the input vector through the spatiotemporal attention module to obtain spatiotemporal excitation includes: splitting the input vector to the spatiotemporal attention module into two paths, one of which performs average pooling on the channel dimension and performs convolution operation using a three-dimensional convolution with a kernel of p; multiplying the convolved vector with the other path vector to generate spatiotemporal excitation.

[0115] Where p can be 1 or other values, without any specific restrictions.

[0116] In this invention, a multi-attention mechanism is designed, which consists of three sub-modules: a spatial attention module that generates channel excitations to enable the network to focus on learning target features and suppress complex background interference in the image; a motion excitation module that generates motion excitations of the target between adjacent video frames to help the network better learn the pose changes of pedestrian targets between frames; and a spatiotemporal attention module that generates spatiotemporal excitations to help the network grasp the complete motion trajectory of the target in the time series, thereby solving problems such as trajectory fragmentation and identity switching caused by frequent occlusion between targets.

[0117] Furthermore, in the three tasks of target regression, target classification, and identity verification, a dynamically weighted loss function is used to promote balanced training of the network, optimize the training effect of the network, and improve the expressive power of the tracking model.

[0118] Example 2:

[0119] Based on the target tracking method based on the multi-path selection attention mechanism provided in Embodiment 1 above, the present invention also provides a target tracking device based on the multi-path selection attention mechanism, such as... Figure 6 The diagram shown is a schematic representation of the device architecture according to an embodiment of the present invention. The device in this embodiment includes one or more processors 21 and a memory 22. Figure 6 Take a processor 21 as an example.

[0120] The processor 21 and the memory 22 can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0121] The memory 22, as a non-volatile computer-readable storage medium for a target tracking method based on a multiple-choice attention mechanism, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the target tracking method based on a multiple-choice attention mechanism in Embodiment 1. The processor 21 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in the memory 22, thereby implementing the target tracking method based on a multiple-choice attention mechanism in the embodiment.

[0122] The memory 22 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 22 may optionally include memory remotely located relative to the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0123] The program instructions / modules are stored in the memory 22 and, when executed by one or more processors 21, execute the target tracking method based on the multi-way attention mechanism in the above embodiments.

[0124] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0125] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A target tracking method based on a multiple selection attention mechanism, characterized in that, The target tracking method based on the multi-path selection attention mechanism comprises the following steps: Two adjacent input images are obtained from a data set, and a tracking network is used to extract features and fuse features of each input image to obtain an aggregated vector corresponding to each input image, and the two aggregated vectors are spliced to obtain a spliced vector; The spliced vector is input into a category classification network branch, an identity verification network branch and a bounding box pair regression network branch of the tracking network; The category classification network branch is used for performing category prediction on the splicing vector to obtain a category loss L cls The identity authentication network branch is used for performing identity prediction on the splicing vector to obtain an identity authentication loss L id The bounding box regression network branch is used for performing bounding box prediction on the splicing vector to obtain a regression loss L reg ​ According to the classification loss L cls The authentication loss L id and regression loss L reg The total loss L is obtained total The tracking network is trained iteratively multiple times until the total loss L is reached. total The training network converges, resulting in the trained tracking network. A target in a real-time image is tracked according to the trained tracking network; Each of the category classification network branch, the identity verification network branch and the bounding box pair regression network branch comprises at least one convolutional layer, and a multi-path selection attention mechanism is added after the convolutional layer of each branch; the multi-path selection attention mechanism comprises a spatial attention module, a motion excitation module and a space-time attention module; the input vector is processed by the spatial attention module to obtain channel excitation; the input vector is processed by the motion excitation module to obtain motion excitation; the input vector is processed by the space-time attention module to obtain space-time excitation; The processing of the input vector by the spatial attention module to obtain the channel excitation comprises: The feature map of each channel of the input vector is globally averaged and pooled by the spatial attention module to obtain the average value of all pixels, and the average value is convolved by one-dimensional convolution with a convolution kernel of k x k to obtain channel excitation; The processing of the input vector by the motion excitation module to obtain the motion excitation comprises: the input vector is channel-reduced by two-dimensional convolution with a convolution kernel of m by the motion excitation module, and the input vector after channel reduction is divided into a first vector and a second vector; the first vector is operated by two-dimensional convolution with a convolution kernel of n, and the difference between the first vector and the second vector is obtained to obtain a difference vector; the difference vector is supplemented by 0 to obtain an intermediate vector, the intermediate vector is averaged and pooled, and the original channel number is restored by two-dimensional convolution with a convolution kernel of m to generate motion excitation; The processing of the input vector by the space-time attention module to obtain the space-time excitation comprises: the input vector input into the space-time attention module is divided into two paths, one of which is averaged and pooled in the channel dimension and is operated by three-dimensional convolution with a convolution kernel of p; the convolved vector is multiplied by the other vector to generate space-time excitation. 2.The target tracking method based on the multiple selection attention mechanism according to claim 1, characterized in that, The total loss L is calculated according to equation one total ; Formula One; Wherein, ω1 and ω2 are learnable parameters for balancing classification and regression tasks. 3.The target tracking method based on the multiple selection attention mechanism of claim 1, wherein, The two adjacent input images are obtained from the data set, and the tracking network is used to extract features of each input image to obtain high-level features and low-level features; The high-level features and the low-level features are aggregated from top to bottom by upsampling to pass the high-level features to the low-level features, and the fusion of the speech information of the target is completed; ​ A bottom-up aggregation path is added on the top of the top-down aggregation path to pass the bottom layer features into the high layer features, complete the fusion of the position information of the target, and obtain the aggregation vector corresponding to each input image; The high layer features include semantic information of the target, and the bottom layer features include position information and texture information of the target. 4.The target tracking method based on the multiple selection attention mechanism of claim 1, wherein, The category prediction of the splicing vector by the category classification network branch obtains a category loss L cls comprises: The splicing vector is processed by at least one convolution layer in the category classification network branch to obtain an input vector; The input vector is processed by a spatial attention module to obtain channel excitation; The input vector is processed by a motion excitation module to obtain motion excitation; The input vector is processed by a space-time attention module to obtain space-time excitation; The channel excitation is multiplied by a weight α, the motion excitation is multiplied by a weight β, and the space-time excitation is multiplied by a weight λ, and the excitations multiplied by the weights are weighted and summed to obtain a predicted output vector, wherein α, β and λ are parameters of the network and are adaptively and dynamically adjusted during the training process of the network. from the predicted output vector and the actual output vector cls .

5. The target tracking method based on multiple selection attention mechanism according to claim 1, characterized in that, The identity authentication network branch predicts the identity of the splicing vector, and obtains an identity authentication loss L id Comprising: The splicing vector is processed by at least one convolution layer in the category classification network branch to obtain an input vector; The input vector is processed by a spatial attention module to obtain channel excitation; The input vector is processed by a motion excitation module to obtain motion excitation; The input vector is processed by a space-time attention module to obtain space-time excitation; The channel excitation is multiplied by a weight α, the motion excitation is multiplied by a weight β, and the space-time excitation is multiplied by a weight λ, and the excitations multiplied by the weights are weighted and summed to obtain a predicted output vector, wherein α, β and λ are parameters of the network and are adaptively and dynamically adjusted during the training process of the network. An authentication loss L is obtained from the predicted output vector and the actual output vector id . 6.The target tracking method based on the multiple selection attention mechanism of claim 1, wherein, The regression network branch performs a bounding box prediction on the splicing vector through the bounding box, to obtain a regression loss L reg comprises: The splicing vector, the predicted output vector output by the category classification network branch, and the predicted output vector output by the identity verification network branch are multiplied to obtain an input vector. The input vector is processed by a spatial attention module to obtain channel excitation; The input vector is processed by a motion excitation module to obtain motion excitation; The input vector is processed by a space-time attention module to obtain space-time excitation; The channel excitation is multiplied by a weight α, the motion excitation is multiplied by a weight β, and the space-time excitation is multiplied by a weight λ, and the excitations multiplied by the weights are weighted and summed to obtain a predicted output vector, wherein α, β and λ are parameters of the network and are adaptively and dynamically adjusted during the training process of the network. According to the predicted output vector and the actual output vector, a regression loss L is obtained reg .

7. The method of claim 1, wherein, The data set includes a public data set and a self-collected data set, wherein the public data set includes MOT17 or MOT20; the collection place of the self-collected data set includes one or more of crowded stations, indoor shopping malls and bustling commercial streets, and the collected data includes both camera fixation and camera motion; the sample annotation format of the self-collected data set is the same as that of the MOT17 data set, and includes position information, category information and identity information of the target.

8. A target tracking device based on a multiple selection attention mechanism, characterized in that, The application relates to a device for implementing the target tracking method based on the multiple selection attention mechanism, comprising at least one processor and a memory, wherein the at least one processor and the memory are connected through a data bus, the memory stores instructions which can be executed by the at least one processor, and the instructions are used for completing the target tracking method based on the multiple selection attention mechanism according to any one of claims 1-7 after being executed by the processor.

Images