Vehicle retrieval method and system using diverse feature information representation

Abstract

The present invention proposes a vehicle retrieval method and system using diverse feature information representation, including using a multi-head attention mechanism network Vision Transformer as the backbone network to extract global features of the received image; using ResNet50 as the backbone network to extract several local features; subjecting the global features and some of the local features to L2 norm constraints; aggregating the global features with some of the local features, encoding them, and performing global average pooling; and utilizing diverse feature constraint losses to allow the diverse features to complement each other and obtain more comprehensive features. The present invention aims to obtain diverse and robust features to achieve vehicle matching with the same ID. The diverse loss is used to remove the correlation between the two features, ensuring that the same vehicle can be identified and that they are not highly correlated, and promoting the two networks to learn vehicle features from different spaces to complement each other and achieve robust feature expression.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle retrieval and re-identification, and in particular to a vehicle retrieval method and system using diverse feature information representation. Background Art

[0002] With the rapid development of intelligent security systems and smart transportation, cameras are deployed on various main roads in the city, and vehicle detection, tracking, and retrieval technologies have emerged. Among them, vehicle retrieval is a key technology in the smart transportation system. The purpose of vehicle retrieval and re-identification is to identify vehicles with the same ID under non-overlapping camera perspectives, realize cross-lens vehicle recognition, and make up for the shortcomings of license plate recognition. Driven by the smart transportation system, vehicle retrieval technology has made great progress, but the development and improvement of vehicle retrieval technology is still a very challenging task.

[0003] In existing technologies, vehicle retrieval often uses convolutional neural networks as the backbone network, utilizing convolution and downsampling operations to capture image features. Due to the inherent convolution and downsampling operations of convolutional neural networks, they focus more on local details such as tires and rearview mirrors, while being unable to learn global information within the image, such as the positional relationships between different vehicle parts. This significantly limits vehicle retrieval performance. Unlike convolutional neural networks, multi-head attention networks remove convolution and downsampling operations and utilize a self-attention mechanism to construct global information within the image, making them widely applicable to computer vision tasks.

[0004] In summary, how to design a framework that can couple global features and local clues is the key to improving retrieval performance. The present invention builds a vehicle retrieval algorithm based on a convolutional neural network and a multi-head attention mechanism network. The convolutional neural network is used to capture local clues, and the multi-head attention mechanism network is used to mine global features. In order to prevent the network from converging quickly and making the distribution of the two features gradually identical, the present invention also proposes a diverse feature constraint loss to remove the correlation between the two features, so that the two features can identify the same vehicle, and the identified vehicles are not highly correlated. It can promote the two networks to learn vehicle features from different spaces, so that global features and local features complement each other, thereby achieving robust feature expression. Summary of the Invention

[0005] The present invention aims to solve one or more technical deficiencies in the above-mentioned prior art and proposes the following technical solutions.

[0006] A vehicle retrieval method using diverse feature information representation, the method comprising:

[0007] S1: The Vision Transformer multi-head attention mechanism network is used as the backbone network for extracting global features. The received image is cut into multiple small blocks and sent to the Vision Transformer multi-head attention mechanism network to extract global features, which is represented by the letter F1. B is the training batch size of the network model, which is a fixed parameter. C1 represents the feature dimension size of the global feature F1. N represents the number of image blocks.

[0008] S2: ResNet50 is used as the backbone network to extract local features, and the extracted local features are respectively R1 、F R2 、F R3 、F R4 Indicates that the extracted local features are all four-dimensional tensors, and the structure of the local features includes four residual convolution blocks, represented by Stage1, Stage2, Stage3 and Stage4 respectively;

[0009] S3: Combine the global feature F1 described in S1 with the first two local features F described in S2 R1 、F R2 Perform L2 norm constraint. The result of L2 norm constraint is expressed as follows:

[0010]

[0011]

[0012] Wherein, L1 represents the global feature F1 and the local feature F R1 The result of L2 norm constraint, L2 represents the global feature F1 and the local feature F R2 The result of L2 norm constraint;

[0013] S4: Combine the global feature F1 described in S1 and the last two local features F described in S2 R3 、F R4 Aggregate and perform feature encoding to obtain the local features of the final output of ResNet50, represented by the letter F2, where B is the training batch size of the network model, which is a fixed parameter. C2 represents the feature dimension size of the local feature F2.

[0014] S5: Diverse feature constraint loss, the last two local features F described in S2 R3 、F R4Perform global average pooling and use a fully connected layer to transform the channel dimension of the local feature into C1, and then calculate the cosine similarity with the global feature F1 in S1, and the results are represented by cos1 and cos2 respectively;

[0015] The overall loss function of the vehicle retrieval task is represented by L, which includes the triple loss function and the cross entropy loss function with label smoothing regularization. The calculation formula is as follows:

[0016]

[0017] L triplet is the triplet loss function, and the calculation formula is as follows:

[0018]

[0019] Among them, represents the number of P different vehicle IDs, K0 represents the selection of K0 different images for each vehicle ID, and for each training image x a , in and x a Among the images of the same identity, select an image x with the largest similarity distance p As a positive sample, in and x a Select an image x with the smallest similarity distance among images of different identities n As negative samples, is a threshold set artificially, d(·,·) is defined as the Euclidean distance, and batch represents the parameter size of batch processing;

[0020] is the cross entropy loss function of label smoothing regularization, and the calculation formula is as follows:

[0021]

[0022] Where ε∈[0,1], K is the number of vehicle categories, p(k) represents the probability of predicting that the vehicle belongs to category k, and p(y) represents the probability of predicting that the vehicle has label y.

[0023] S6: Use the overall loss function L of the vehicle retrieval task to train the network structure, and then use the global feature F1 and the local feature F2 to perform feature comparison to retrieve vehicle information.

[0024] Furthermore, the step S3 also includes the following steps before performing the L2 norm constraint calculation: using global average pooling to convert the local features from a four-dimensional tensor to a two-dimensional tensor, and using a fully connected layer to convert the feature dimension of the local features into C1.

[0025] Furthermore, the global feature F1 is transformed in size so that the size of the global feature F1 is the same as that of the local feature F R3 The size of the global feature F1 is the same as that of the local feature F1, and then the feature channel aggregation and 1*1 convolution operation are performed on the global feature F1 to make the feature dimension of the global feature F1 and the local feature F R3 The feature dimension of F1 is the same as that of F1, and the F1 is sent to the fourth convolution block stage4 in S2 for feature encoding;

[0026] The global feature F1 is transformed in size so that the size of the global feature F1 is the same as that of the local feature F R4 The size of the global feature F1 is the same as that of the local feature F2, and the feature channel aggregation and global average pooling are performed on the global feature F1 to obtain the local feature finally output by ResNet50, which is represented by the letter F2.

[0027] The present invention also proposes a vehicle retrieval system using diverse feature information representation, characterized in that the device comprises:

[0028] Feature extraction module: Use the multi-head attention mechanism network Vision Transformer as the backbone network for extracting global features, cut the received image into multiple small blocks, and send them to the multi-head attention mechanism network Vision Transformer to extract global features, represented by the letter F1, where B is the training batch size of the network model, which is a fixed parameter. C1 represents the feature dimension size of the global feature F1, and N represents the number of image blocks. At the same time, ResNet50 is used as the backbone network for extracting local features. The extracted local features are respectively expressed as F R1 、F R2 、F R3 、F R4 Indicates that the local features are all four-dimensional tensors, and the structure of the local features includes four residual convolution blocks, which are represented by Stage1, Stage2, Stage3 and Stage4 respectively;

[0029] L2 norm constraint module: The global feature F1 is respectively combined with the first two local features F R1 、F R2 Perform L2 norm constraint. The result of L2 norm constraint is expressed as follows:

[0030]

[0031]

[0032] Wherein, L1 represents the global feature F1 and the local feature F R1The result of L2 norm constraint, L2 represents the global feature F1 and the local feature F R2 The result of L2 norm constraint;

[0033] Aggregate feature encoding module: the global feature F1 and the last two local features F R3 、F R4 Aggregate and perform feature encoding to obtain the local features of the final output of ResNet50, represented by the letter F2, where B is the training batch size of the network model, which is a fixed parameter. C2 represents the feature dimension size of the local feature F2.

[0034] Diverse feature constraint loss module: The last two local features F of the feature extraction module are R3 、F R4 Perform global average pooling and use a fully connected layer to transform the feature channel dimension of the local feature into C1, and then calculate the cosine similarity with the global feature F1 respectively. The results are represented by cos1 and cos2 respectively;

[0035] The overall loss function of the vehicle retrieval task is represented by L, which includes the triple loss function and the cross entropy loss function with label smoothing regularization. The calculation formula is as follows:

[0036]

[0037] L triplet is the triplet loss function, and the calculation formula is as follows:

[0038]

[0039] Among them, represents the number of P different vehicle IDs, K0 represents the selection of K0 different images for each vehicle ID, and for each training image x a , in and x a Among the images of the same identity, select an image x with the largest similarity distance p As a positive sample, in and x a Select an image x with the smallest similarity distance among images of different identities n As negative samples, is a threshold set artificially, d(·,·) is defined as the Euclidean distance, and batch represents the parameter size of batch processing;

[0040] is the cross entropy loss function of label smoothing regularization, and the calculation formula is as follows:

[0041]

[0042] Where ε∈[0,1], K is the number of vehicle categories, p(k) represents the probability of predicting that the vehicle belongs to category k, and p(y) represents the probability of predicting that the vehicle has label y.

[0043] Vehicle information retrieval module: uses the overall loss function L of the vehicle retrieval task to train the network structure, and then uses the global feature F1 and the local feature F2 to perform feature comparison to retrieve vehicle information.

[0044] Furthermore, the L2 norm constraint module also includes the following steps before performing the L2 norm constraint calculation: using global average pooling to convert the local features from a four-dimensional tensor to a two-dimensional tensor, and using a fully connected layer to convert the feature dimension of the local features into C1.

[0045] Furthermore, the aggregated feature encoding is to transform the size of the global feature F1 so that the size of the global feature F1 is the same as the size of the local feature, and then perform feature channel aggregation and 1*1 convolution operation on the global feature F1 so that the feature dimension of the global feature F1 is the same as the feature dimension of the local feature, and send the global feature F1 to the fourth convolution block stage4 in S2 for feature encoding;

[0046] The global feature F1 is resized so that the size of the global feature F1 is the same as the size of the local feature. The global feature F1 is then subjected to feature channel aggregation and global average pooling to obtain the local feature finally output by ResNet50, which is represented by the letter F2.

[0047] It should be noted that the loss function is used to train the multi-head attention mechanism network VisionTransformer and the convolutional neural network ResNet50 for feature extraction, so that the two networks can extract features more accurately. At the same time, constraining the loss by extracting global features and local features can promote the complementarity between the two features.

[0048] It should be noted that the diverse feature constraint loss is designed to make all feature information and local feature information have as different feature distributions as possible under the constraints of the loss function, and to encode and extract information from more feature spaces, so that the features learned by the network model are more diverse.

[0049] The present invention further provides a computer-readable storage medium storing computer program instructions, wherein the computer program instructions implement any of the above methods when executed by a processor.

[0050] The technical effect of the present invention is that it combines the advantages of a multi-head attention mechanism network and a convolutional neural network, and at the same time utilizes the extracted diverse features to constrain the loss, promotes the complementarity between global features and local features, removes the correlation between the two features, and obtains diverse and robust features for comparison to achieve vehicle matching with the same ID, to ensure that the same vehicle can be retrieved and identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Other features, objects and advantages of the present application will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings.

[0052] Figure 1 A vehicle retrieval method using diverse feature information representation is provided according to an embodiment of the present invention.

[0053] Figure 2 The flowchart of a vehicle retrieval method using diverse feature information representation is provided according to an embodiment of the present invention.

[0054] Figure 3 2 is a schematic diagram of a vehicle retrieval system using diverse feature information representation provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0056] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0057] Figure 1 A vehicle retrieval method using diverse feature information representation is shown according to an embodiment of the present invention.

[0058] The method includes but is not limited to the following steps:

[0059] S101: extract global features;

[0060] The Vision Transformer multi-head attention mechanism network is used as the backbone network for extracting global features. The received image is cut into multiple small blocks and sent to the Vision Transformer multi-head attention mechanism network to extract global features, which are represented by the letter F1. B is the training batch size of the network model, which is a fixed parameter. C1 represents the feature dimension size of the global feature F1. N represents the number of image blocks.

[0061] S102: extract local features;

[0062] ResNet50 is used as the backbone network for extracting local features, and the extracted local features are respectively R1 、F R2 、F R3 、F R4 Indicates that the extracted local features are all four-dimensional tensors, and the structure of the local features includes four residual convolution blocks, represented by Stage1, Stage2, Stage3 and Stage4 respectively;

[0063] S103: L2 norm constraint;

[0064] The global feature F1 is respectively combined with the first two local features F R1 、F R2 Perform L2 norm constraint. The result of L2 norm constraint is expressed as follows:

[0065]

[0066]

[0067] Wherein, L1 represents the global feature F1 and the local feature F R1 The result of L2 norm constraint, L2 represents the global feature F1 and the local feature F R2 The result of L2 norm constraint;

[0068] S104: feature aggregation and feature encoding;

[0069] The global feature F1 in S101 and the last two local features F in S102 R3 、F R4 Aggregate and perform feature encoding to obtain the local features of the final output of ResNet50, represented by the letter F2, where B is the training batch size of the network model, which is a fixed parameter, and C2 represents the feature dimension size of the local feature F2.

[0070] It should be noted that the local features output by the ResNet50 are four-dimensional tensors, which need to be converted into two-dimensional tensors using global average pooling, and the feature dimensions need to be converted into C1 using a fully connected layer before L2 norm constraint calculation can be performed.

[0071] It should be noted that the specific method of feature aggregation and feature encoding is: the global feature F1 is transformed in size so that the size of the global feature F1 is the same as that of the local feature F R3 The size of the global feature F1 is the same as that of the local feature F1, and then the feature channel aggregation and 1*1 convolution operation are performed on the global feature F1 to make the feature dimension of the global feature F1 and the local feature F R3 The feature dimension of F1 is the same as that of F1, and the F1 is sent to the fourth convolution block stage4 in S102 for feature encoding;

[0072] The global feature F1 is transformed in size so that the size of the global feature F1 is the same as that of the local feature F R4 The size of the global feature F1 is the same as that of the local feature F2, and the feature channel aggregation and global average pooling are performed on the global feature F1 to obtain the local feature finally output by ResNet50, which is represented by the letter F2.

[0073] Figure 2 A flow chart of a vehicle retrieval method using diverse feature information representation provided according to an embodiment of the present invention is shown.

[0074] Including but not limited to the following steps:

[0075] S101: extract global features;

[0076] The Vision Transformer multi-head attention mechanism network is used as the backbone network for extracting global features. The received image is cut into multiple small blocks and sent to the Vision Transformer multi-head attention mechanism network to extract global features, which are represented by the letter F1. B is the training batch size of the network model, which is a fixed parameter. C1 represents the feature dimension size of the global feature F1. N represents the number of image blocks.

[0077] S102: extract local features;

[0078] ResNet50 is used as the backbone network for extracting local features, and the extracted local features are respectively R1 、F R2 、F R3 、F R4 Indicates that the extracted local features are all four-dimensional tensors, and the structure of the local features includes four residual convolution blocks, represented by Stage1, Stage2, Stage3 and Stage4 respectively;

[0079] S103: L2 norm constraint;

[0080] The global feature F1 is respectively combined with the first two local features FR1 、F R2 Perform L2 norm constraint. The result of L2 norm constraint is expressed as follows:

[0081]

[0082]

[0083] Wherein, L1 represents the global feature F1 and the local feature F R1 The result of L2 norm constraint, L2 represents the global feature F1 and the local feature F R2 The result of L2 norm constraint;

[0084] S104: feature aggregation and feature encoding;

[0085] The global feature F1 in S101 and the last two local features F in S102 R3 、F R4 Aggregate and perform feature encoding to obtain the local features of the final output of ResNet50, represented by the letter F2, where B is the training batch size of the network model, which is a fixed parameter. C2 represents the feature dimension size of the local feature F2.

[0086] S105: Diverse feature constraint loss;

[0087] The last two local features F described in S102 R3 、F R4 Perform global average pooling and use a fully connected layer to transform the channel dimension of the local feature into C1, and then calculate the cosine similarity with the global feature F1 in S1, and the results are represented by cos1 and cos2 respectively;

[0088] The overall loss function of the vehicle retrieval task is represented by L, which includes the triple loss function and the cross entropy loss function with label smoothing regularization. The calculation formula is as follows:

[0089]

[0090] L triplet is the triplet loss function, and the calculation formula is as follows:

[0091]

[0092] Among them, represents the number of P different vehicle IDs, K0 represents the selection of K0 different images for each vehicle ID, and for each training image x a , in and x a Among the images of the same identity, select an image x with the largest similarity distancep As a positive sample, in and x a Select an image x with the smallest similarity distance among images of different identities n As negative samples, is a threshold set artificially, d(·,·) is defined as the Euclidean distance, and batch represents the parameter size of batch processing;

[0093] is the cross entropy loss function of label smoothing regularization, and the calculation formula is as follows:

[0094]

[0095] Where ε∈[0,1], K is the number of vehicle categories, p(k) represents the probability of predicting that the vehicle belongs to category k, and p(y) represents the probability of predicting that the vehicle has label y.

[0096] S106: Use the overall loss function L of the vehicle retrieval task to train the network structure, and then use the global feature F1 and the local feature F2 to perform feature comparison to retrieve vehicle information.

[0097] It should be noted that the local features output by the ResNet50 are four-dimensional tensors, which need to be converted into two-dimensional tensors using global average pooling, and the feature dimensions need to be converted into C1 using a fully connected layer before L2 norm constraint calculation can be performed.

[0098] It should be noted that the specific method of feature aggregation and feature encoding is: the global feature F1 is transformed in size so that the size of the global feature F1 is the same as that of the local feature F R3 The size of the global feature F1 is the same as that of the local feature F1, and then the feature channel aggregation and 1*1 convolution operation are performed on the global feature F1 to make the feature dimension of the global feature F1 and the local feature F R3 The feature dimension of F1 is the same as that of F1, and the F1 is sent to the fourth convolution block stage4 in S102 for feature encoding;

[0099] The global feature F1 is transformed in size so that the size of the global feature F1 is the same as that of the local feature F R4 The size of the global feature F1 is the same as that of the local feature F2, and the feature channel aggregation and global average pooling are performed on the global feature F1 to obtain the local feature finally output by ResNet50, which is represented by the letter F2.

[0100] It should be noted that the loss function is used to train the multi-head attention mechanism network VisionTransformer and the convolutional neural network ResNet50 for feature extraction, so that the two networks can extract features more accurately. At the same time, constraining the loss by extracting global features and local features can promote the complementarity between the two features.

[0101] It should be noted that the diverse feature constraint loss is designed to make all feature information and local feature information have as different feature distributions as possible under the constraints of the loss function, and to encode and extract information from more feature spaces, so that the features learned by the network model are more diverse.

[0102] Figure 3 A schematic diagram of a vehicle retrieval system using diverse feature information representation is shown according to an embodiment of the present invention.

[0103] Including but not limited to the following modules:

[0104] Feature extraction module: Use the multi-head attention mechanism network Vision Transformer as the backbone network for extracting global features, cut the received image into multiple small blocks, and send them to the multi-head attention mechanism network Vision Transformer to extract global features, represented by the letter F1, where B is the training batch size of the network model, which is a fixed parameter. C1 represents the feature dimension size of the global feature F1, and N represents the number of image blocks. At the same time, ResNet50 is used as the backbone network for extracting local features. The extracted local features are respectively expressed as F R1 、F R2 、F R3 、F R4 Indicates that the local features are all four-dimensional tensors, and the structure of the local features includes four residual convolution blocks, which are represented by Stage1, Stage2, Stage3 and Stage4 respectively;

[0105] L2 norm constraint module: The global feature F1 is respectively combined with the first two local features F R1 、F R2 Perform L2 norm constraint. The result of L2 norm constraint is expressed as follows:

[0106]

[0107]

[0108] Wherein, L1 represents the global feature F1 and the local feature F R1The result of L2 norm constraint, L2 represents the global feature F1 and the local feature F R2 The result of L2 norm constraint;

[0109] Aggregate feature encoding module: the global feature F1 and the last two local features F R3 、F R4 Aggregate and perform feature encoding to obtain the local features of the final output of ResNet50, represented by the letter F2, where B is the training batch size of the network model, which is a fixed parameter. C2 represents the feature dimension size of the local feature F2.

[0110] Diverse feature constraint loss module: The last two local features F of the feature extraction module are R3 、F R4 Perform global average pooling and use a fully connected layer to transform the feature channel dimension of the local feature into C1, and then calculate the cosine similarity with the global feature F1 respectively. The results are represented by cos1 and cos2 respectively;

[0111] The overall loss function of the vehicle retrieval task is represented by L, which includes the triple loss function and the cross entropy loss function with label smoothing regularization. The calculation formula is as follows:

[0112]

[0113] L triplet is the triplet loss function, and the calculation formula is as follows:

[0114]

[0115] Among them, represents the number of P different vehicle IDs, K0 represents the selection of K0 different images for each vehicle ID, and for each training image x a , in and x a Among the images of the same identity, select an image x with the largest similarity distance p As a positive sample, in and x a Select an image x with the smallest similarity distance among images of different identities n As negative samples, is a threshold set artificially, d(·,·) is defined as the Euclidean distance, and batch represents the parameter size of batch processing;

[0116] is the cross entropy loss function of label smoothing regularization, and the calculation formula is as follows:

[0117]

[0118] Where ε∈[0,1], K is the number of vehicle categories, p(k) represents the probability of predicting that it belongs to category k, and p(y) represents the probability of predicting that it is label y;

[0119] Vehicle information retrieval module: uses the overall loss function L of the vehicle retrieval task to train the network structure, and then uses the global feature F1 and the local feature F2 to perform feature comparison to retrieve vehicle information.

[0120] It should be noted that the L2 norm constraint module also includes the following steps before performing the L2 norm constraint calculation: using global average pooling to convert the local features from a four-dimensional tensor to a two-dimensional tensor, and using a fully connected layer to convert the feature dimension of the local features into C1.

[0121] It should be noted that the aggregated feature encoding is to transform the size of the global feature F1 so that the size of the global feature F1 is the same as the size of the local feature, and then perform feature channel aggregation and 1*1 convolution operations on the global feature F1 so that the feature dimension of the global feature F1 is the same as the feature dimension of the local feature, and send the global feature F1 to the fourth convolution block stage4 in S2 for feature encoding;

[0122] The global feature F1 is resized so that the size of the global feature F1 is the same as the size of the local feature. The global feature F1 is then subjected to feature channel aggregation and global average pooling to obtain the local feature finally output by ResNet50, which is represented by the letter F2.

[0123] It should be noted that the loss function is used to train the multi-head attention mechanism network Vision Transformer and the convolutional neural network ResNet50 for feature extraction, so that the two networks can extract features more accurately. At the same time, constraining the loss by extracting global features and local features can promote the complementarity between the two features.

[0124] It should be noted that the diverse feature constraint loss is designed to make all feature information and local feature information have as different feature distributions as possible under the constraints of the loss function, and to encode and extract information from more feature spaces, so that the features learned by the network model are more diverse.

[0125] In the present invention, in order to solve the problem that the convolutional neural network in the prior art focuses more on the local details of the image, the present invention adopts a multi-head attention mechanism to extract the global feature information of the vehicle and adopts a convolutional neural network to extract the local feature information of the vehicle. This is an important inventive concept of the present invention.

[0126] In order to prevent the network from converging quickly and make the distribution of the two features gradually the same, the present invention also proposes a diverse feature constraint loss to remove the correlation between the two features, so that the two features can identify the same vehicle. At the same time, the identified vehicles are not highly correlated, which can promote the two networks to learn vehicle features from different spaces, so that the global features and local features complement each other, thereby achieving robust feature expression. This is an important concept of the present invention.

[0127] Finally, it should be noted that the above embodiments are only intended to illustrate rather than limit the technical solutions of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the present invention can still be modified or replaced by equivalents. Any modification or partial replacement that does not depart from the spirit and scope of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A vehicle retrieval method using diverse feature information representation, characterized in that: include: S1: The Vision Transformer multi-head attention mechanism network is used as the backbone network for extracting global features. The received image is cut into multiple small blocks and sent to the Vision Transformer multi-head attention mechanism network to extract global features, which is represented by the letter F1. , B is the training batch size of the network model, which is a fixed parameter. represents the feature dimension size of the global feature F1, Represents the number of image blocks; S2: ResNet50 is used as the backbone network for extracting local features, and the extracted local features are used Indicates that the extracted local features are all four-dimensional tensors, and the structure of the local features includes four residual convolution blocks, represented by Stage1, Stage2, Stage3 and Stage4 respectively; S3: Combine the global feature F1 described in S1 with the first two local features of S2 Perform L2 norm constraint. The result of L2 norm constraint is expressed as follows: ， ， Wherein, L1 represents the global feature F1 and the local feature F R1 The result of L2 norm constraint, L2 represents the global feature F1 and the local feature F R2 The result of L2 norm constraint; S4: Combine the global feature F1 described in S1 and the two local features after S2 Aggregate and perform feature encoding to obtain the local features of the final output of ResNet50, represented by the letter F2, where , B is the training batch size of the network model, which is a fixed parameter, and C2 represents the feature dimension size of the local feature F2; S5: Diverse feature constraint loss, the last two local features in S2 Perform global average pooling and use the fully connected layer to transform the channel dimension of the local features into , and then perform cosine similarity calculation with the global feature F1 described in S1, and the results are represented by cos1 and cos2 respectively; The overall loss function of the vehicle retrieval task is represented by L, which includes the triple loss function and the cross entropy loss function with label smoothing regularization. The calculation formula is as follows: ， is the triplet loss function, and the calculation formula is as follows: ， Among them, representatives The number of different vehicle IDs, Represents each vehicle ID selection different images, for each training image , in and Among the images of the same identity, select the one with the largest similarity distance. As positive samples, Select an image with the smallest similarity distance among images of different identities As negative samples, It is an artificially set threshold. Defined as the Euclidean distance, Indicates the parameter size of the batch; is the cross entropy loss function of label smoothing regularization, and the calculation formula is as follows: ， in, , is the number of vehicle categories, and p(k) represents the predicted category The probability of , p(y) represents the probability of predicting label y; S6: Use the overall loss function L of the vehicle retrieval task to train the network structure, and then use the global feature F1 and the local feature F2 to perform feature comparison to retrieve vehicle information.

2. The vehicle retrieval method using diverse feature information representation according to claim 1, characterized in that: The step S3 also includes the following steps before performing the L2 norm constraint calculation: using global average pooling to transform the local features from a four-dimensional tensor to a two-dimensional tensor, and using a fully connected layer to transform the feature dimensions of the local features into .

3. The vehicle retrieval method using diverse feature information representation according to claim 1, characterized in that: In step S4, the global feature F1 is transformed in size so that the size of the global feature F1 is the same as that of the local feature The size of the global feature F1 is the same as that of the local feature, and then the feature channel aggregation and 1*1 convolution operation are performed on the global feature F1 to make the feature dimension of the global feature F1 and the local feature The feature dimension of F1 is the same as that of F1, and the F1 is sent to the fourth convolution block stage4 in S2 for feature encoding; The global feature F1 is transformed in size so that the size of the global feature F1 is the same as that of the local feature The size of the global feature F1 is the same as that of the local feature F2, and the feature channel aggregation and global average pooling are performed on the global feature F1 to obtain the local feature finally output by ResNet50, which is represented by the letter F2.

4. A vehicle retrieval system using diverse feature information representation, characterized in that: The system includes: Feature extraction module: Use the multi-head attention mechanism network Vision Transformer as the backbone network for extracting global features, cut the received image into multiple small blocks, and send them to the multi-head attention mechanism network Vision Transformer to extract global features, represented by the letter F1, where , B is the training batch size of the network model, which is a fixed parameter. represents the feature dimension size of the global feature F1, Represents the number of image blocks; ResNet50 is used as the backbone network for extracting local features, and the extracted local features are respectively used Indicates that the local features are all four-dimensional tensors, and the structure of the local features includes four residual convolution blocks, which are represented by Stage1, Stage2, Stage3 and Stage4 respectively; L2 norm constraint module: The global feature F1 is combined with the first two local features F R1 、F R2 Perform L2 norm constraint. The result of L2 norm constraint is expressed as follows: ， ， Wherein, L1 represents the global feature F1 and the local feature F R1 The result of L2 norm constraint, L2 represents the global feature F1 and the local feature F R2 The result of L2 norm constraint; Aggregate feature encoding module: The global feature F1 and the last two local features described in the feature extraction module are aggregated. Aggregate and perform feature encoding to obtain the local features of the final output of ResNet50, represented by the letter F2, where , B is the training batch size of the network model, which is a fixed parameter, and C2 represents the feature dimension size of the local feature F2; Diverse feature constraint loss module: the last two local features of the feature extraction module are Perform global average pooling and use the fully connected layer to transform the feature channel dimension of the local features into , and then perform cosine similarity calculation with the global feature F1, and the results are represented by cos1 and cos2 respectively; The overall loss function of the vehicle retrieval task is represented by L, which includes the triple loss function and the cross entropy loss function with label smoothing regularization. The calculation formula is as follows: ， is the triplet loss function, and the calculation formula is as follows: ， Among them, representatives The number of different vehicle IDs, Represents each vehicle ID selection different images, for each training image , in and Among the images of the same identity, select the one with the largest similarity distance. As positive samples, Select an image with the smallest similarity distance among images of different identities As negative samples, It is an artificially set threshold. Defined as the Euclidean distance, Indicates the parameter size of the batch; is the cross entropy loss function of label smoothing regularization, and the calculation formula is as follows: ， in, , is the number of vehicle categories, and p(k) represents the predicted category The probability of , p(y) represents the probability of predicting label y; Vehicle information retrieval module: uses the overall loss function L of the vehicle retrieval task to train the network structure, and then uses the global feature F1 and the local feature F2 to perform feature comparison to retrieve vehicle information.

5. A vehicle retrieval system using diverse feature information representation according to claim 4, characterized in that: The L2 norm constraint module also includes the following steps before performing the L2 norm constraint calculation: using global average pooling to transform the local features from a four-dimensional tensor to a two-dimensional tensor, and using a fully connected layer to transform the feature dimensions of the local features into .

6. A vehicle retrieval system using diverse feature information representation according to claim 4, characterized in that: The aggregated feature encoding is to transform the size of the global feature F1 so that the size of the global feature F1 is the same as the size of the local feature, then perform feature channel aggregation and 1*1 convolution operation on the global feature F1 so that the feature dimension of the global feature F1 is the same as the feature dimension of the local feature, and send the global feature F1 to the fourth convolution block stage4 in S2 for feature encoding; The global feature F1 is resized so that the size of the global feature F1 is the same as the size of the local feature. The global feature F1 is then subjected to feature channel aggregation and global average pooling to obtain the local feature finally output by ResNet50, which is represented by the letter F2.

7. A computer-readable storage medium storing computer program instructions, characterized in that: The computer program instructions implement the method according to any one of claims 1 to 3 when executed by a processor.

Images