Target tracking method based on improved rednet network

Abstract

The invention relates to the technical field of target tracking, in particular to a target tracking method based on an improved rednet network, and the method comprises the steps: collecting video data of a scene to be detected, and carrying out the framing processing, thereby obtaining a plurality of images; the method comprises the following steps: constructing an improved Resnet network, replacing a common convolution in stage0 of the Resnet network with a partial convolution, and introducing a CBAM after the partial convolution; the Bottleneck in the stage1 to the stage4 is improved by using the SAAN (Sequence Amplified Amplified Number); and training the improved Resnet network by using the training set, and converging the model by using a loss function. According to the method, different image data expansion data sets are synthesized by using the cGAN; in the network structure, partial conv is used to replace standard convolution, and CBAM is introduced after convolution; the SAAN is integrated in a residual block to capture a long-distance dependency relationship, and the precision of continuous tracking is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target tracking, and in particular to a target tracking method based on an improved ResNet network. Background Art

[0002] With the rapid development of computer vision technology, multi-target tracking (MOT) has become one of the core functions of intelligent monitoring systems. In public areas or inside buildings with complex structures, multi-target tracking methods across cameras are particularly important. Factors such as lighting changes, occlusion, and changes in target appearance place higher requirements on the stability of tracking algorithms.

[0003] ByteTrack first identifies the target through the target detector, and uses the improved Kalman filter method to predict the target in the tracking stage; at the same time, it adopts the method of dynamically adjusting the association threshold to adapt to the density and movement speed of the target in different scenes; although ByteTrack has improved the tracking effect, it still depends on the detection accuracy of the detector, and may cause tracking loss for occluded targets or disappeared targets.

[0004] MOTR is based on the Transformer structure for multi-target tracking. It improves the accuracy of target association through the global attention mechanism. However, the tracking performance decreases in the case of occlusion or target disappearance, and it cannot accurately identify the tracked target in long-term tracking or cross-camera scenarios. Summary of the invention

[0005] In view of the shortcomings of existing methods, the present invention uses the cGAN network to synthesize different image data to expand the data set; replaces the standard convolution with partial conv in the network structure, and introduces CBAM after the convolution; integrates SAAN in the residual block to capture long-distance dependencies and improve the accuracy of continuous tracking.

[0006] The technical solution adopted by the present invention is: a target tracking method based on an improved resnet network comprises the following steps:

[0007] Step 1: Collect video data of the scene to be tested and perform frame processing to obtain several images;

[0008] As a preferred embodiment of the present invention, a cGAN network is used to expand the image.

[0009] As a preferred implementation of the present invention, a target detection algorithm is used to identify people in an image.

[0010] Step 2: Build an improved Resnet network, replace the normal convolution in stage 0 of the Resnet network with partial convolution, and introduce CBAM after the partial convolution; and use SAAN to improve the Bottleneck in stage 1 to stage 3;

[0011] As a preferred embodiment of the present invention, the improvement of Bottleneck includes improving the un-downsampled Bottleneck, and the improved un-downsampled Bottleneck includes: a 1×1 convolution layer, a 3×3 convolution layer, and then a 1×1 convolution layer, and a residual connection with the input feature map is performed and then input into SAAN, and finally passed through a Relu function.

[0012] As a preferred embodiment of the present invention, the improvement of Bottleneck also includes an improved downsampling Bottleneck, and the improved downsampling Bottleneck includes: the main branch sequentially undergoes 1×1 convolution, 3×3 convolution and 1×1 convolution, and 1 1×1 convolution is added to the shortcut connection, the main branch and the shortcut connection are added and then input into SAAN, and finally activated by ReLU.

[0013] Step 3: Use the training set to train the improved Resnet network and use the loss function to converge the model.

[0014] As a preferred implementation of the present invention, the loss function is TripleLoss.

[0015] As a preferred embodiment of the present invention, the hit rate and average precision are used to evaluate the improved Resnet network.

[0016] Beneficial effects of the present invention:

[0017] 1. Use cGAN network to synthesize data images, reducing the workload of collecting data images;

[0018] 2. Replace the standard convolution on the original backbone network ResNet50; add the CBAM attention mechanism and apply the residual block with attention mechanism in the network, which not only enables the model to better handle occluded targets, but also improves the model's extraction of key features and enhances the model's ability to capture long-distance dependencies by adding attention modules at different positions, thereby improving the tracking effect of the overall model;

[0019] 3. Two attention mechanisms are introduced into the Resnet network structure to enhance the focus on features and make the feature matching process more accurate. Triple loss is used to converge the model, aiming to improve the tracking accuracy of the model during the tracking process.

[0020] 4. The hit rate (rank@1) of the model of the present invention is improved from the original 81% to 84.6%, an increase of 3.6%, and the average accuracy is improved from 59.7% to 63.6%. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart of the target tracking method based on the improved resnet network of the present invention;

[0022] Figure 2 This is a schematic diagram of the cGCN network synthesized image;

[0023] Figure 3 It is to improve the resnet network structure diagram;

[0024] Figure 4 This is a schematic diagram of the improved un-downsampled Bottleneck and downsampled Bottleneck in the resnet network;

[0025] Figure 5 It is a tracking effect diagram of the present invention. DETAILED DESCRIPTION

[0026] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.

[0027] like Figure 1 As shown, a target tracking method based on an improved resnet network includes the following steps:

[0028] Step 1: Collect video data of the scene to be tested and perform frame processing to obtain several images;

[0029] Use the cGAN network to expand the image; for example, collect the original video sequence of a road; divide the video sequence into frames of image data, use the target detection algorithm, such as YOLOX, to identify the people in the image, and mark the people in the detection frame;

[0030] The cGAN network is used to synthesize existing person images to generate images of the same person from different perspectives or at different times, that is, to expand the person images; Figure 2 As shown in the figure, the expanded data set is divided into training set and test set in a ratio of 8:2.

[0031] Step 2: Build an improved Resnet network; including the following steps:

[0032] Step 21: Figure 3Taking Resnet50 as an example, the ordinary convolution in stage0 of the Resnet network is replaced with partial convolution, and CBAM is introduced after the partial convolution;

[0033] Partial convolution is used to replace the original standard convolution. Compared with standard convolution, partial conv can better handle occluded images, can automatically learn masks, and only update the weights of valid pixel areas; that is, when processing occluded areas, partial convolution will skip invalid pixels and only perform convolution operations on valid pixels, which can avoid information loss, thereby maintaining the integrity of information in the feature map, so that the output feature map contains more context information;

[0034] After the CBAM module is added to the partial convolution, the CBAM module is designed from two dimensions: channel and space. The channel attention mechanism can automatically learn and emphasize the key feature areas in the pedestrian image, such as people's faces, clothing styles, backpacks, etc. These areas are important for distinguishing different pedestrians; the spatial attention mechanism can emphasize the important spatial positions in the image, which is beneficial for the network structure to realize the positioning of pedestrians; therefore, CBAM can not only highlight the channel features that carry more information, but also emphasize the important spatial positions in the image, thereby helping the model to better focus on the key areas in the pedestrian image; it is crucial to improve the performance of the model when dealing with occlusion problems.

[0035] Step 22, improve the Bottleneck in stage 1 to stage 4;

[0036] The SAAN module is introduced after the residual connection of the un-downsampled Bottleneck and the downsampled Bottleneck; specifically, the improved Bottleneck is added after the main branch (the branch containing the convolutional layer) and the shortcut connection are added;

[0037] like Figure 4 The left figure is a schematic diagram of the improved non-downsampled Bottleneck. When the feature map passes through the non-downsampled Bottleneck, it is first reduced in dimension through a 1×1 convolution layer, and then the features are extracted through a 3×3 convolution layer; the feature map is then increased in dimension through a 1×1 convolution layer, and is residually connected and merged with the input feature map to obtain a fused feature map; the fused feature map is then input into SAAN for self-attention and adaptive normalization processing, and finally output through the Relu function.

[0038] This design enables the network to capture global information, making up for the deficiency that the convolution in the residual block only focuses on local information. In addition, the self-attention mechanism in the SAAN module can effectively enhance the expression of global features by calculating the attention weights of each position; SAAN adopts the Rethinking the competition between detection and ReID in Multi-Object Tracking structure in the paper.

[0039] In SAAN, the feature map generates attention weights through the self-attention mechanism, and the weights are used to perform weighted summation on each position in the feature map to obtain an attention-enhanced feature map. Subsequently, the attention-enhanced feature map is processed by adaptive normalization, and is adjusted using the normalization parameters dynamically generated according to the input feature map, and finally the processed feature map is output.

[0040] like Figure 4 The right figure is a schematic diagram of the improved downsampling Bottleneck. In the downsampling Bottleneck, the main branch first reduces the dimension through 1×1 convolution, then downsamples and extracts features through 3×3 convolution with a stride of 2, and then increases the dimension through 1×1 convolution; due to downsampling, the dimension of the feature map output by the main branch does not match the dimension of the input feature map, so a 1×1 convolution with a stride of 2 is added to the shortcut connection to downsample the input feature map to make it consistent with the dimension of the feature map output by the main branch; then, the outputs of the main branch and the shortcut connection are added to obtain the fused feature map, which enters the SAAN module, uses the self-attention mechanism to weight important features, enhances the global feature expression, and finally passes it to the next layer through ReLU activation; this design can effectively capture long-distance dependencies and is suitable for global feature matching across cameras.

[0041] Step 3: Use the training set to train the Resnet network. During the training process, use TripleLoss to converge the model. After the training, save the weight file and evaluate the performance of the model on the test set.

[0042] TripleLoss mainly minimizes the distance between A and P, while maximizing the distance between A and N, which must be greater than the distance between A and P plus a preset margin value.

[0043] AnchorA (input image), positive example P (image of the same person as the input image), negative example N (image of a different person than the input image), margin is set to 0.5.

[0044] Figure 5This is the effect diagram evaluated on the test set. Serial numbers 8 and 10 represent different people. The improved model in Table 2 improves the overall accuracy by 3.6% and the Rank@1 by 3.9%.

[0045] Table 2 Ablation experiment

[0046]

[0047] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A target tracking method based on an improved resnet network, characterized in that: The following steps are involved: Step 1: Collect video data of the scene to be tested and perform frame processing to obtain several images; Step 2: Build an improved Resnet network, replace the normal convolution in stage 0 of the Resnet network with partial convolution, and introduce CBAM after the partial convolution; and use SAAN to improve the Bottleneck in stage 1 to stage 4; Step 3: Use the training set to train the improved Resnet network and use the loss function to converge the model.

2. The target tracking method based on the improved resnet network according to claim 1 is characterized in that: The improvement of Bottleneck includes the improvement of un-downsampled Bottleneck. The improvement of un-downsampled Bottleneck includes: a 1×1 convolution layer, a 3×3 convolution layer, and then a 1×1 convolution layer, and a residual connection with the input feature map, and then input into SAAN, and finally through the Relu function.

3. The target tracking method based on the improved resnet network according to claim 1 is characterized in that: The improvements to Bottleneck also include the improved downsampling Bottleneck. The improved downsampling Bottleneck includes: the main branch passes through 1 1×1 convolution, 3×3 convolution and 1 1×1 convolution in sequence, and 1 1×1 convolution is added to the shortcut connection. The main branch and the shortcut connection are added and then input into SAAN, and finally activated by ReLU.

4. The target tracking method based on the improved resnet network according to claim 1 is characterized in that: Use cGAN network to expand the image.

5. The target tracking method based on the improved resnet network according to claim 1 is characterized in that: The loss function is TripleLoss.

6. The target tracking method based on the improved resnet network according to claim 1 is characterized in that: The hit rate and average precision are used to evaluate the improved Resnet network.

7. The target tracking method based on the improved resnet network according to claim 1 is characterized in that: Use target detection algorithms to identify people in images.

8. The target tracking system based on the improved resnet network is characterized by: include: a memory for storing instructions executable by a processor; A processor, configured to execute instructions to implement the target tracking method based on the improved ResNet network as described in any one of claims 1 to 7.

9. A computer readable medium storing a computer program code, characterized in that: When the computer program code is executed by a processor, the computer program code implements the target tracking method based on the improved ResNet network as described in any one of claims 1 to 7.

Images