Vehicle re-identification method based on multi-channel feature collaborative fusion

Abstract

The invention discloses a vehicle re-identification method based on multi-channel feature collaborative fusion, and relates to the technical field of image processing, and the method comprises the steps: firstly, adding a squeezing excitation network attention mechanism after the Conv2x and Conv3x stages of a backbone network, and improving the feature extraction quality of the backbone network; after the Conv4x stage, a multi-channel convolution collaboration module is used to extract different semantic features; the processed features in each stage are independently extracted and spliced according to channels, and finally, rich feature information is extracted by using a feature processing module to further improve the network performance; the method has a better effect on vehicle re-identification, richer feature information is integrated, the problems of intra-class similarity and intra-class difference are further solved, the robustness of the model is improved, and the quantitative index of the method exceeds that of a current advanced comparison method.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a vehicle re-identification method based on collaborative fusion of multi-channel features. Background Art

[0002] Since the 21st century, rapid urbanization has led to increasingly busy road traffic, necessitating the growing importance of vehicle monitoring and management. In the field of traffic management, real-time monitoring of vehicle conditions and the identification and tracking of suspicious or violating vehicles are crucial for preventing traffic accidents and combating crime. However, the volume of image and other data has skyrocketed, making traditional manual methods of locating vehicle information in complex surveillance scenarios inefficient and costly. Vehicle re-identification technology has emerged as a key enabler of this need. It can quickly and accurately identify target vehicles under different camera conditions, providing strong support for combating crime, improving traffic safety, strengthening public security, and empowering intelligent transportation systems.

[0003] Vehicle re-identification (VRI) is essentially an image retrieval problem. It aims to determine whether vehicle images captured by different cameras in a specific traffic monitoring scenario belong to the same vehicle by analyzing them. However, VRI faces numerous challenges. For one thing, factors such as fluctuating lighting conditions and low resolution can cause license plate pixels to blur and incomplete information, significantly reducing license plate recognition accuracy. Furthermore, in many traffic scenarios, license plates are often obscured, removed, or forged, making the search for the target vehicle even more challenging. Consequently, VRI primarily relies on vehicle appearance features for identity retrieval and matching.

[0004] In practical applications, the same vehicle can appear significantly different from different viewing angles due to the effects of lighting, viewing angle, and resolution. However, different vehicles may appear similar due to the same model or similar color. The appearance of the same vehicle changes from different viewing angles, while the similarity between different vehicles due to similar colors and models, creates two major challenges for vehicle re-ID: how to effectively distinguish different vehicles with similar appearances (i.e., the inter-class similarity problem); and how to accurately identify the same vehicle from different viewing angles (i.e., the intra-class difference problem). Summary of the Invention

[0005] In order to solve the above technical problems, the present invention provides a vehicle re-identification method based on multi-channel feature collaborative fusion, comprising the following steps:

[0006] S1. Construct a vehicle re-identification network architecture with a multi-channel feature collaborative fusion network. This network architecture includes a squeeze excitation module, a multi-channel convolution collaborative module, and a multi-stage feature fusion module. The squeeze excitation module uses full connectivity to adjust the number of channels. The multi-channel convolution collaborative module uses a parallelized multi-head grouping mechanism and uses convolutional layers to adjust the number of channels. The multi-stage feature fusion module is used to combine the output features of the squeeze excitation module with the features of the multi-channel convolution collaborative module in the channel dimension after convolution and pooling operations, and then adjust the number of channels again.

[0007] S2. Construct a loss function, including a classification branch loss function and a measurement branch loss function;

[0008] S3. Obtain the Veri-776 dataset and divide it into a training set and a test set. Train the vehicle re-identification network architecture of the multi-channel feature collaborative fusion network to obtain the optimal model.

[0009] S4. Input the image to be tested into the optimal model to obtain the prediction result of the vehicle identity.

[0010] The technical solution further defined in the present invention is:

[0011] Furthermore, in step S1, the backbone network of the vehicle re-identification network architecture of the multi-channel feature collaborative fusion network includes five stages, namely Conv1, Conv2_x, Conv3_x, Conv4_x and Conv5_x; there are two squeezing excitation modules, which are respectively arranged after the Conv2_x and Conv3_x stages of the backbone network; there is one multi-channel convolution collaborative module, which is arranged after the Conv4_x stage of the backbone network; the multi-stage feature fusion module extracts the output features of the squeezing excitation module in the Conv2_x and Conv3_x stages from the backbone network, and after convolution pooling operation, splices them together with the output features of the multi-channel collaborative module in the Conv4_x stage in the channel dimension, and inputs them into the Conv_5x stage for processing after spatial and channel scale adjustment.

[0012] As mentioned above, in a vehicle re-identification method based on the collaborative fusion of multi-channel features, the squeeze excitation module first performs a global average pooling operation on the features to compress the spatial dimension information of each channel into a scalar, which is expressed as:

[0013]

[0014] Among them, z c is the global average of the cth channel, F sq is the feature map u c Perform global average pooling function operation, u cIt is the feature map output by the convolutional layer, i and j represent the row and column coordinate values in the feature map, H and W are the height and width of the feature map, respectively; the fully connected layer is used to compress and restore the feature dimension.

[0015] As mentioned above, in a vehicle re-identification method based on collaborative fusion of multi-channel features, the input channels of the two squeeze excitation modules are set to 256 and 512, respectively, and the dimension reduction ratios are 8 and 16, respectively.

[0016] As mentioned above, a vehicle re-identification method based on multi-channel feature collaborative fusion is proposed. The multi-channel convolution collaborative module first uses the view operation to group the input feature channels into multiple subgroups, and each subgroup is independently global average pooled. Then, two convolutional layers with a kernel size of 1×1 are used to reduce and restore the dimension respectively. Finally, the attention weight is generated by the Sigmoid function, which is expressed as:

[0017]

[0018] Among them, z represents the weighted sum of the previous layer, and e represents a natural constant.

[0019] As mentioned above, a vehicle re-identification method based on multi-channel feature collaborative fusion is proposed. The number of channels, attention heads, and dimension reduction of the multi-channel convolution collaborative module are set to 1024, 8, and 32, respectively.

[0020] As mentioned above, a vehicle re-identification method based on multi-channel feature collaborative fusion, the multi-channel convolution collaborative module first groups the input features, that is, the feature map is divided into 8 groups according to the channel dimension; each group of features x i Use global average pooling independently to capture global features, use 1×1 convolution to reduce the dimension, then perform ReLU function nonlinear activation, and then restore the original dimension through 1×1 convolution expansion to obtain the semantic enhancement tensor a i , a i Input to the Sigmoid activation function, and then compare it with the feature x at each position in the original feature group i Multiply them together to spatially scale these features so that their values are normalized between 0 and 1, as shown in the following formula:

[0021] x′ i =x i ·σ(a i )

[0022] Among them, σ represents the Sigmoid function; finally, each set of feature tensors is concatenated by channel to restore the original number of channels as the input of the next stage.

[0023] As mentioned above, in the vehicle re-identification method based on the collaborative fusion of multi-channel features, in step S2, the classification branch loss function is expressed as:

[0024]

[0025] Among them, L cls represents the cross entropy loss value of the classification branch, represents the true label distribution of the i-th category, represents the model's predicted probability for the i-th category, and C is the number of categories in training.

[0026] As described above, in a vehicle re-identification method based on collaborative fusion of multi-channel features, in step S2, the metric branch loss function is expressed as:

[0027]

[0028] Where P represents the number of identities, K represents the number of samples for each identity, m represents the distance threshold, P(a) represents the positive sample set of each anchor point, N(a) represents the negative sample set of each anchor point, and x a Represented as anchor point, x p Denotes the extracted embedding of the positive sample, x n It represents the extracted embedding of negative samples and d() represents the Euclidean distance.

[0029] As described above, in a vehicle re-identification method based on the collaborative fusion of multi-channel features, in step S3, the Adam algorithm is used to optimize the loss function during training. The image block size, training batch size, and initial learning rate are set to 256×256, 48, and 0.0001, respectively. The number of training epochs is 160. When the epochs are 45, 90, and 135, respectively, the learning rate is divided by 0.1.

[0030] The beneficial effects of the present invention are:

[0031] In the present invention, an excitation squeezing module is used to enable the first two stages of the backbone network to more effectively capture the discriminative features of vehicles, and a multi-channel convolution collaborative module is used to more effectively process local feature information; a multi-stage feature fusion module fuses the features of the three stages, increasing the dimension and information content of the features, so that the model can learn richer feature representations; thereby, the present invention has better performance in vehicle re-identification and better accuracy in identifying vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0033] Figure 2 Schematic diagram of the vehicle re-identification network architecture of the multi-channel feature collaborative fusion network in an embodiment of the present invention;

[0034] Figure 3 Schematic diagram of the structure of the extrusion excitation module in an embodiment of the present invention;

[0035] Figure 4 Schematic diagram of the structure of a multi-channel convolution collaborative module in an embodiment of the present invention;

[0036] Figure 5 Schematic diagram of the structure of the multi-stage feature fusion module in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] This embodiment provides a vehicle re-identification method based on multi-channel feature collaborative fusion, such as Figure 1 As shown, the following steps are included:

[0038] S1. Construct a vehicle re-identification network architecture of a multi-channel feature collaborative fusion network (Multi-Channel Feature Collaborative Fusion Network, MFCF-Net). The network architecture includes a squeeze excitation module, a multi-channel convolutional cooperation module (Multi-channel Convolutional Cooperation, MCC) and a multi-stage feature fusion module; the backbone network of the vehicle re-identification network architecture of the multi-channel feature collaborative fusion network includes five stages, namely Conv1, Conv2_x, Conv3_x, Conv4_x and Conv5_x.

[0039] like Figure 2 As shown in the figure, the squeezing excitation modules are placed after the Conv2_x and Conv3_x stages of the backbone network respectively, to improve the backbone network's capture of vehicle image detail features, while suppressing the influence of cluttered background. After processing by the Conv4_x stage network, more semantically rich spatial features are generated. Then, through the multi-channel convolution collaborative module, more fine-grained features are captured by constructing a multi-branch channel grouping processing structure; then the multi-stage feature fusion module is introduced to fuse the Conv2_x, Conv3_x and Conv4_x stage networks together to form a spatial tensor with rich semantic information. After the convolution pooling operation, it enters the last layer of the backbone network and is processed by the loss function to obtain the vehicle identity prediction result.

[0040] There are two squeeze excitation modules, which are respectively set after the Conv2_x and Conv3_x stages of the backbone network; the squeeze excitation module uses full connection to adjust the number of channels; the use of the squeeze excitation module enhances the feature representation, enabling the model to more effectively capture the discriminative features of vehicles, thereby improving the differentiation ability.

[0041] The input channels of the two squeeze excitation modules are set to 256 and 512 respectively, and the dimension reduction ratios are 8 and 16 respectively; Figure 3 As shown in Figure 2, the squeeze excitation module first performs a global average pooling operation on the features to compress the spatial dimension information of each channel into a scalar, which is expressed as:

[0042]

[0043] Among them, z c is the global average of the cth channel, F sq is the feature map u c Perform global average pooling function operation, u c It is the feature map output by the convolutional layer, i and j represent the row and column coordinate values in the feature map, H and W are the height and width of the feature map, respectively; the fully connected layer is used to compress and restore the feature dimension.

[0044] The squeeze excitation module can highlight the subtle features of the vehicle, increasing the network's ability to capture the discriminative features of the vehicle without increasing the amount of computation. When using this module in the shallow network stage, it can better focus on the local details of the vehicle when extracting low-level and mid-level features, thereby improving the recognition of the vehicle and further improving the accuracy of vehicle re-identification. The weight of each channel is dynamically adjusted to capture the key features of the vehicle, so that the model pays more attention to the feature channels related to the task, thereby enhancing the feature expression ability and using the extracted features as the input of the next stage network.

[0045] A multi-channel convolution collaborative module and a parallel multi-head grouping mechanism are adopted to realize fine-grained inter-channel feature interaction, effectively enhancing the model's ability to represent different semantic features; the multi-channel convolution collaborative module is equipped with a, which is located after the backbone network Conv4_x stage; the multi-channel convolution collaborative module adopts a parallel multi-head grouping mechanism and uses a 1×1 convolution layer to adjust the number of channels.

[0046] The number of channels, attention heads, and dimension reduction of the multi-channel convolutional collaborative module are set to 1024, 8, and 32, respectively; Figure 4 As shown in the figure, the multi-channel convolution collaborative module first uses the view operation to group the input feature channels into multiple subgroups, and each subgroup performs global average pooling independently; convolution is used instead of the original fully connected layer to reduce and restore dimensions, where the kernel size of the first convolution layer is set to 1×1 to reduce the dimension, and the kernel size of the second convolution layer is set to 1×1 to restore the dimension. The convolution layer is used to act directly on the feature map, and weight calculation is applied more efficiently in the multi-channel feature map; the attention weight is then generated through the Sigmoid function, which is expressed as:

[0047]

[0048] Here, z represents the weighted sum of the previous layer, and e represents a natural constant, whose approximate value is approximately 2.71828.

[0049] The multi-channel convolution collaborative module can improve the processing of multiple semantic features without almost increasing the number of meals and computational complexity. Compared with the squeeze-excitation module, the parallelized multi-head grouping mechanism realizes fine-grained inter-channel feature interaction, effectively enhancing the model's ability to represent different semantic features. At the same time, the multi-head mechanism can also prevent the bias problem of single-channel attention.

[0050] Inspired by the capsule network, the multi-channel convolutional collaborative module first groups the input features, that is, the feature map is divided into 8 groups according to the channel dimension. Each group of features can capture a specific semantics during the learning process; each group of features x i Use global average pooling independently to capture global features, use 1×1 convolution to reduce the dimension, then perform ReLU function nonlinear activation, and then restore the original dimension through 1×1 convolution expansion to obtain the semantic enhancement tensor a i , a i Input to the Sigmoid activation function, and then compare it with the feature x at each position in the original feature group i Multiply them together to spatially scale these features so that their values are normalized between 0 and 1, as shown in the following formula:

[0051] x′ i =x i ·σ(a i )

[0052] Among them, σ represents the Sigmoid function; finally, each set of feature tensors is concatenated by channel to restore the original number of channels as the input of the next stage.

[0053] like Figure 5 As shown in the figure, the multi-stage feature fusion module extracts the output features of the squeeze excitation module in the Conv2_x and Conv3_x stages from the backbone network. After convolution and pooling operations, it is spliced together with the output features of the multi-channel collaborative module in the Conv4_x stage in the channel dimension. The number of channels is adjusted again, and after spatial and channel scale adjustments, it is input into the Conv_5x stage for processing.

[0054] S2. Construct loss functions, including classification branch loss functions and metric branch loss functions, to achieve the effect of extracting important features and improve the generalization ability of the model.

[0055] The classification branch loss function is expressed as:

[0056]

[0057] Among them, Lcls represents the cross entropy loss value of the classification branch, represents the true label distribution of the i-th category, represents the model's predicted probability for the i-th category, and C is the number of categories in training.

[0058] The metric branch loss function is expressed as:

[0059]

[0060] Where P represents the number of identities, K represents the number of samples for each identity, m represents the distance threshold, P(a) represents the positive sample set of each anchor point, N(a) represents the negative sample set of each anchor point, and x a Represented as anchor point, x p Denotes the extracted embedding of the positive sample, x n It represents the extracted embedding of negative samples and d() represents the Euclidean distance.

[0061] S3. Obtain the Veri-776 dataset and divide it into training and test sets. Train the vehicle re-identification network architecture based on the multi-channel feature collaborative fusion network. During training, the mAP, CMC1, and CMC5 indicators increase to a certain extent, and the network tends to be stable.

[0062] In this example, the experimental data selection includes 776 different vehicles for training and testing the proposed MCFCF-Net network model; the training dataset contains 576 identities and 37,778 images; the test dataset contains 200 identities and 11,579 images, of which 1,678 images are used as queries, and the remaining images are used as galleries for matching query images.

[0063] In this embodiment, the network model is implemented using the Pytorch framework and implemented on a workstation with an NVIDIA GTX 3070 GPU; during training, the Adam algorithm is used to optimize the loss function, and the image block size, training batch size, and initial learning rate are set to 256×256, 48, and 0.0001, respectively. The number of epochs for training is 160, and the learning rate is divided by 0.1 when the epochs are 45, 90, and 135, respectively.

[0064] S4. After model training, the prediction result of vehicle identity can be obtained through the output layer of the network.

[0065] As shown in Table 1 below, the vehicle re-identification evaluation results of different methods are displayed. Through comparative experiments, it can be found that compared with other methods disclosed in the prior art, the MCFCF-Net method of this embodiment performs best on the Veri-776 data.

[0066] Table 1

[0067]

[0068] The method of this embodiment uses an excitation squeezing module to enable the first two stages of the backbone network to more effectively capture the discriminative features of the vehicle, and adopts a multi-channel convolution collaborative module to more effectively process local feature information; the multi-stage feature fusion module fuses the features of the three stages, increasing the dimension and information content of the features, so that the model can learn richer feature representations; thereby, the present invention has better performance in vehicle re-identification and better accuracy in identifying vehicles.

[0069] In addition to the above embodiments, the present invention may also have other implementations. Any technical solution formed by equivalent replacement or equivalent transformation falls within the scope of protection required by the present invention.

Claims

1. A vehicle re-identification method based on collaborative fusion of multi-channel features, characterized by: The following steps are involved: S1. Construct a vehicle re-identification network architecture with a multi-channel feature collaborative fusion network, which includes a squeeze excitation module, a multi-channel convolution collaborative module, and a multi-stage feature fusion module; The squeeze excitation module uses full connectivity to adjust the number of channels; the multi-channel convolutional collaborative module adopts a parallel multi-head grouping mechanism and uses convolutional layers to adjust the number of channels; the multi-stage feature fusion module is used to combine the output features of the squeeze excitation module with the features of the multi-channel convolutional collaborative module in the channel dimension after convolution and pooling operations, and then adjust the number of channels again; S2. Construct a loss function, including a classification branch loss function and a measurement branch loss function; S3. Obtain the Veri-776 dataset and divide it into a training set and a test set. Train the vehicle re-identification network architecture of the multi-channel feature collaborative fusion network to obtain the optimal model. S4. Input the image to be tested into the optimal model to obtain the prediction result of the vehicle identity.

2. The vehicle re-identification method based on multi-channel feature collaborative fusion according to claim 1, characterized in that: In the step S1, the backbone network of the vehicle re-identification network architecture of the multi-channel feature collaborative fusion network includes five stages, namely Conv1, Conv2_x, Conv3_x, Conv4_x and Conv5_x; there are two squeezing excitation modules, which are respectively arranged after the Conv2_x and Conv3_x stages of the backbone network; there is one multi-channel convolution collaborative module, which is arranged after the Conv4_x stage of the backbone network; the multi-stage feature fusion module extracts the output features of the squeezing excitation module in the Conv2_x and Conv3_x stages from the backbone network, and after the convolution pooling operation, splices them together with the output features of the multi-channel collaborative module in the Conv4_x stage in the channel dimension, and inputs them into the Conv_5x stage for processing after spatial and channel scale adjustment.

3. The vehicle re-identification method based on multi-channel feature collaborative fusion according to claim 2, characterized in that: The squeeze excitation module first performs a global average pooling operation on the features to compress the spatial dimension information of each channel into a scalar, which is expressed as: Among them, z c is the global average of the cth channel, F sq is the feature map u c Perform global average pooling function operation, u c It is the feature map output by the convolutional layer, i and j represent the row and column coordinate values in the feature map, H and W are the height and width of the feature map, respectively; the fully connected layer is used to compress and restore the feature dimension.

4. The vehicle re-identification method based on multi-channel feature collaborative fusion according to claim 3, characterized in that: The input channels of the two squeeze excitation modules are set to 256 and 512 respectively, and the dimension reduction ratios are 8 and 16 respectively.

5. The vehicle re-identification method based on multi-channel feature collaborative fusion according to claim 2, characterized in that: The multi-channel convolutional collaborative module first uses the view operation to group the input feature channels into multiple subgroups, and each subgroup is independently global average pooled; then it is sequentially passed through two convolutional layers with a kernel size of 1×1 to reduce and restore the dimension respectively; and then the attention weight is generated by the Sigmoid function, which is expressed as: Among them, z represents the weighted sum of the previous layer, and e represents a natural constant.

6. The vehicle re-identification method based on multi-channel feature collaborative fusion according to claim 5, characterized in that: The number of channels, attention heads, and dimensionality reduction of the multi-channel convolutional collaborative module are set to 1024, 8, and 32, respectively.

7. The vehicle re-identification method based on multi-channel feature collaborative fusion according to claim 5, characterized in that: The multi-channel convolution collaborative module first groups the input features, that is, the feature map is divided into 8 groups according to the channel dimension; each group of features x i Use global average pooling independently to capture global features, use 1×1 convolution to reduce the dimension, then perform ReLU function nonlinear activation, and then restore the original dimension through 1×1 convolution expansion to obtain the semantic enhancement tensor a i , a i Input to the Sigmoid activation function, and then compare it with the feature x at each position in the original feature group i Multiply them together to spatially scale these features so that their values are normalized between 0 and 1, as shown in the following formula: x′ i =x i ·σ(a i ) Among them, σ represents the Sigmoid function; finally, each set of feature tensors is concatenated by channel to restore the original number of channels as the input of the next stage.

8. The vehicle re-identification method based on multi-channel feature collaborative fusion according to claim 1, characterized in that: In step S2, the classification branch loss function is expressed as: Among them, L cls represents the cross entropy loss value of the classification branch, represents the true label distribution of the i-th category, represents the model's predicted probability for the i-th category, and C is the number of categories in training.

9. The vehicle re-identification method based on multi-channel feature collaborative fusion according to claim 1, characterized in that: In step S2, the metric branch loss function is expressed as: Where P represents the number of identities, K represents the number of samples for each identity, m represents the distance threshold, P(a) represents the positive sample set of each anchor point, N(a) represents the negative sample set of each anchor point, and x a Represented as anchor point, x p Denotes the extracted embedding of the positive sample, x n It represents the extracted embedding of negative samples and d() represents the Euclidean distance.

10. The vehicle re-identification method based on multi-channel feature collaborative fusion according to claim 1, characterized in that: In step S3, the Adam algorithm is used to optimize the loss function during training, and the image block size, training batch size, and initial learning rate are set to 256×256, 48, and 0.0001, respectively. The number of epochs for training is 160, and the learning rate is divided by 0.1 when the epochs are 45, 90, and 135, respectively.

Images