Vehicle re-identification method and system with local-global adaptive guidance enhancement

Abstract

The present invention discloses a vehicle re-identification method and system with local-global adaptive guidance enhancement. The method comprises: performing image preprocessing on training images and constructing a training set; training a vehicle re-identification collaborative representation network with local-global adaptive guidance enhancement based on the training set; the vehicle re-identification collaborative representation network with local-global adaptive guidance enhancement comprises a backbone network module based on VisionTransformer and an adaptively optimized feature encoding module based on local attention guidance; obtaining a query image and a gallery set, performing search and matching on the query image in the gallery set, and obtaining a matching result. The system comprises: a preprocessing unit, a network training unit, and a search and matching unit. By using the present invention, the accuracy of vehicle re-identification can be improved. The present invention can be widely applied in the field of vehicle re-identification.

Description

Technical Field

[0001] The present invention relates to the field of vehicle re-identification, and in particular to a vehicle re-identification method and system with local-global adaptive guidance enhancement. Background Art

[0002] Early vehicle re-identification methods focused on extracting robust global features from the entire vehicle image. However, in recent years, an increasing number of vehicles have significant inter-class similarity in appearance, and due to the different angles at which cameras capture vehicle images, the same vehicle can exhibit significant intra-class variability. Currently, research on mining local, fine-grained features for vehicle re-identification has garnered widespread attention. Deep learning methods that consider combining local features for vehicle re-identification can be summarized into two categories: one category relies on additional information to supervise the training process, including vehicle key point positioning, vehicle key area detection, vehicle partial analysis, and vehicle orientation-based methods. However, the above methods are highly dependent on manual information labeling or other pre-trained networks (such as target detection networks, semantic segmentation networks, etc.) to extract prior knowledge, resulting in high time consumption and complexity of the overall algorithm; the other category relies solely on vehicle ID labels to design network structures in a weakly supervised learning manner to extract robust feature representations, including dividing input images or dividing feature maps for local feature extraction, and using graph convolutional neural networks to enhance spatial feature structures. However, the above methods are currently affected by background information in the image, and do not highlight significant regional features or distinguishable regional features, and the network structure is highly complex. Summary of the Invention

[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a local-global adaptive guided enhanced vehicle re-identification collaborative representation method and system, which fully considers local fine-grained features, focuses on the problems of high similarity between different vehicle classes and large differences within the same vehicle class, and improves the accuracy of vehicle re-identification.

[0004] The first technical solution adopted by the present invention is: a vehicle re-identification method with local-global adaptive guidance enhancement, comprising the following steps:

[0005] Perform image preprocessing on training images and construct training sets;

[0006] Based on the training set, the local-global adaptive guided enhanced vehicle re-identification collaborative representation network is trained to obtain a fully trained local-global adaptive guided enhanced vehicle re-identification collaborative representation network;

[0007] The local-global adaptive guided enhanced vehicle re-identification collaborative representation network includes a backbone network module based on VisionTransformer and an adaptive optimized feature encoding module based on local attention guidance;

[0008] The query image and gallery set are obtained, and the query image is retrieved and matched in the gallery set based on the well-trained local-global adaptive guided enhanced vehicle re-identification collaborative representation network to obtain the matching result.

[0009] Furthermore, the step of performing image preprocessing on the training images and constructing a training set specifically includes:

[0010] Acquire a vehicle image for training to obtain a training image;

[0011] Resize the training image to obtain an adjusted training image;

[0012] Performing data augmentation on the adjusted training image to obtain an enhanced training image;

[0013] The training set is constructed by combining the augmented training images and their corresponding true labels.

[0014] Furthermore, the step of training the vehicle re-ID collaborative representation network enhanced by local-global adaptive guidance based on the training set to obtain a fully trained vehicle re-ID collaborative representation network enhanced by local-global adaptive guidance specifically includes:

[0015] The training set is input into the vehicle re-identification collaborative representation network enhanced by local-global adaptive guidance;

[0016] Based on the Vision Transformer backbone network module, the input image is divided into small local area blocks and global and local features are extracted.

[0017] The adaptive optimization feature encoding module based on local attention guidance selects candidate local features according to the attention weights corresponding to the local small area blocks and optimizes them to obtain the optimized local area features;

[0018] The global features and key local area features are spliced ​​and fused after batch normalization to obtain a complete feature representation;

[0019] Perform retrieval and matching based on the complete feature representation to obtain matching results;

[0020] The network parameters are adjusted according to the matching results and the true labels, and a well-trained local-global adaptive guided enhanced vehicle re-identification collaborative representation network is obtained.

[0021] Furthermore, the Vision Transformer-based backbone network module includes an input encoding submodule and a feature extraction submodule. The step of dividing the input image into gridded local small area blocks and extracting global features and local features specifically includes:

[0022] The input encoding submodule divides the input image into gridded small local area blocks to form a one-dimensional vector sequence input;

[0023] The feature extraction submodule uses multiple Transformer encoding layers to perform global feature extraction on the one-dimensional vector sequence input and local feature extraction on small local blocks.

[0024] Furthermore, the adaptive optimization feature encoding module based on local attention guidance selects candidate local features and optimizes them according to the attention weights corresponding to the local small area blocks to obtain the optimized local area features. This step specifically includes:

[0025] The region selection submodule selects key areas according to the weight of the coding layer, obtains the index and local features of the corresponding small area blocks, and obtains candidate local features;

[0026] The local guided feature learning submodule combines the candidate local features with the global features separately and then optimizes the internal encoding of the candidate local features;

[0027] According to the attention weights of the candidate local features, the softmax normalization function is used to adaptively generate the external weights corresponding to the candidate local regions, and the weights of the external features of the candidate local regions are optimized to obtain the optimized local region features.

[0028] Furthermore, the loss function calculation formula for adjusting network parameters based on the matching results and true labels is as follows:

[0029]

[0030] In the above formula, w1 and w2 represent the weights of global features and local features, h represents the number of selected local feature blocks, and f g represents the global feature, f p Represents the enhanced local features, L ID represents the ID loss function, L Tri Represents the triplet loss function; in the triplet loss function, given a set of sample instances (a, p, n), a represents the anchor sample, p represents the sample with the same true label as the anchor sample, n represents the sample with a different true label from the anchor sample, and α represents the hyperparameter used to control the network backpropagation process.

[0031] The second technical solution adopted by the present invention is: a local-global adaptive guided enhanced vehicle re-identification collaborative representation system, comprising:

[0032] A preprocessing unit, which performs image preprocessing on the training images and constructs a training set;

[0033] A network training unit trains a local-global adaptive guided enhanced vehicle re-identification collaborative representation network based on a training set to obtain a fully trained local-global adaptive guided enhanced vehicle re-identification collaborative representation network, wherein the local-global adaptive guided enhanced vehicle re-identification collaborative representation network includes a backbone network module based on Vision Transformer and an adaptive optimization feature encoding module based on local attention guidance;

[0034] The retrieval and matching unit obtains the query image and the gallery set, and retrieves and matches the query image in the gallery set based on the well-trained local-global adaptive guided enhanced vehicle re-identification collaborative representation network to obtain the matching result.

[0035] The beneficial effects of the method and system of the present invention are as follows: the local-global adaptive guided enhanced vehicle re-identification method and system proposed in the present invention is an end-to-end vehicle re-identification method. The efficient VisionTransformer is used as the backbone network to extract global features and local features, and the candidate local key focus area features are screened and obtained according to the attention weights calculated by the network for each area of ​​the image; the local fine-grained features are enhanced by independently combining each local area with the global features and individually optimizing the feature encoding; and the adaptive weight calculation is performed based on the attention weight of the selected area, and the feature encoding of each local area feature is optimized again, and finally the complete feature representation is enhanced. The method of further combining local salient features to collaboratively explore key distinguishable features improves the accuracy of vehicle re-identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flowchart of the steps of a vehicle re-identification method with local-global adaptive guidance enhancement according to the present invention;

[0037] Figure 2 This is a structural block diagram of a collaborative representation network for vehicle re-identification enhanced by local-global adaptive guidance according to a specific embodiment of the present invention;

[0038] Figure 3 This is a structural block diagram of the internal structure of the Transformer network coding layer in a specific embodiment of the present invention;

[0039] Figure 4 It is a structural block diagram of the local-global adaptive guided enhanced vehicle re-identification collaborative representation system of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are provided for ease of description only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted based on the understanding of those skilled in the art.

[0041] like Figure 1 As shown, the present invention provides a vehicle re-identification method with local-global adaptive guidance enhancement. The vehicle re-identification collaborative representation network proposed in the present invention is a fast and efficient vehicle re-identification method that can be well applied in applications or development processes in different scenarios. The method includes the following steps:

[0042] S1. Perform image preprocessing on training images and construct a training set;

[0043] S1.1. Obtain vehicle images for training and obtain training images

[0044] Specifically, H and W represent the width and height of the vehicle image, respectively, and C represents the number of vehicle image channels.

[0045] S1.2. resize the training image to obtain an adjusted training image;

[0046] S1.3. Perform data enhancement on the adjusted training image to obtain an enhanced training image

[0047] Specifically, data augmentation methods include random horizontal flipping, padding, random cropping, and random erasing.

[0048] S1.4. Construct a training set by combining the enhanced training images and their corresponding true labels.

[0049] Specifically, is the true value label corresponding to X, T represents the total number of training samples, where y i ∈{1,2,...,C}, C represents the number of vehicle ID ground truth labels.

[0050] S2. Based on the training set, the local-global adaptive guided enhanced vehicle re-identification collaborative representation network is trained to obtain a fully trained local-global adaptive guided enhanced vehicle re-identification collaborative representation network; the local-global adaptive guided enhanced vehicle re-identification collaborative representation network includes a backbone network module based on Vision Transformer and an adaptive optimization feature encoding module based on local attention guidance. The network structure diagram is shown in FIG. Figure 2 ;

[0051] S2.1. Input the training set into the local-global adaptive guided enhanced vehicle re-identification collaborative representation network;

[0052] S2.2, the backbone network module based on Vision Transformer, which divides the input image into small local area blocks and extracts global and local features;

[0053] S2.2.1. The input encoding submodule divides the input image into gridded small local blocks to form a one-dimensional vector sequence input;

[0054] Specifically, the input encoding submodule includes a 2D convolutional layer that performs gridded region division on the input training image with a convolution kernel size of 16×16 and a stride of 16, and maps the image feature dimension to 768 dimensions, resulting in a training block of size 768×16×16. The training block is then flattened to 768×256 and a learnable global block is added. To learn global features, and then add a one-dimensional learnable position encoding information feature The training block is the final one-dimensional vector sequence input. The final input is in the form of:

[0055]

[0056] S2.2.2. The feature extraction submodule uses multiple Transformer encoding layers to perform global feature extraction on the one-dimensional vector sequence input and local feature extraction on small local area blocks.

[0057] Specifically, the feature extraction submodule includes 11 encoding layers of the Transformer network, which extracts global features and local features corresponding to all local small area blocks.

[0058] The Transformer network was first proposed for application in natural language processing tasks. Recently, it has been migrated to the field of computer vision and achieved good results. The Transformer network is based on the self-attention mechanism. There is no downsampling operation in the overall network structure. The internal full connection method can retain more feature information. In addition, it can build relationships between long-distance inputs, so it has a strong feature extraction capability. The present invention uses 11 encoding layers of the Transformer network. Specifically, the internal structural block diagram of the Transformer network encoding layer is shown in Figure 1. Figure 3 A coding layer consists of a multi-head self-attention module and a fully connected module, which are connected in a residual way. Specifically, given the one-dimensional vector input processed by the above input coding module First, a layer normalization (LN) operation is performed to summarize the statistical distribution of the unified samples, and then the multi-head self-attention module is used to extract network features. Specifically, the single-head self-attention mechanism is implemented through a set of weight matrices [W Q ,W K ,W V ] Convert the input X into the Query (Q), Keys (K), and Values ​​(V) vectors required to calculate the attention value:

[0059] Q=X×W Q ,K=X×W K ,V=X×W V

[0060] The attention value is obtained by calculating the self-attention mechanism formula.

[0061]

[0062] In the above formula, the softmax function normalizes the attention weight value.

[0063] The multi-head self-attention mechanism adopts multiple sets of weight matrices [W Q ,W K ,W V ] obtain multiple sets of different Q, K, V vectors, which expands the model's ability to focus on different input position information in parallel, so that more complete feature information can be obtained during the training process. After obtaining the attention value, a weight W is used O The attention values ​​calculated in all heads are fused according to the weights in the corresponding dimensions.

[0064] MHSAttention(Q,K,V)=concat(head1,head2,…,head h )W O

[0065] head i =Attention(QW i Q ,KW i K ,VW i V ),i∈1,2,…,h

[0066] In the above formula, h represents the number of heads. The present invention adopts a self-attention mechanism module with 12 heads.

[0067] After obtaining the attention values ​​calculated above, layer normalization is performed again. This is then fed into a multi-layer perceptron (MLP) network to produce feature outputs with the same dimensions as the input. Specifically, this network includes two fully connected layers and uses the GELU activation function to avoid vanishing gradients during training.

[0068] After calculations in 11 Transformer encoding layers, the global features and the local features of each small area block divided by the corresponding one-dimensional vector sequence input can be obtained:

[0069]

[0070] In the above formula, F class Represents global features, [F0,F1,…,F 256 ] represents all local features.

[0071] S2.3, an adaptive optimization feature encoding module based on local attention guidance, which selects candidate local features according to the attention weights corresponding to the local small area blocks and optimizes them to obtain the optimized local area features;

[0072] S2.3.1. The region selection submodule selects key regions based on the weights of the coding layer, obtains the index and local features of the corresponding small region blocks, and obtains candidate local features;

[0073] Specifically, the input vehicle image is divided into gridded small area blocks. Not all small area blocks contain the target vehicle image, nor do all small area blocks have independent distinguishable features. In order to avoid the interference of background information in the image and obtain the most distinguishable features, the present invention uses the focus area selection module to select candidate local areas. In the process of ViT feature extraction, the attention weight map generated by each attention head of each coding layer will be obtained, and the corresponding input gridded local small area block,

[0074]

[0075] In the above formula, L represents the number of Transformer encoding layers used, h represents the number of attention heads used in each encoding layer, and N represents the number of gridded local small area blocks.

[0076] Taking into account the changes in the transmission of feature information weights in all coding layers, the lower network layers pay more attention to image texture features, while the increase in the number of layers gradually pays attention to high-level semantic information features. In order to enable the network learning to select a more complete and reasonable local attention area, the weights of all layers of each attention head are matrix multiplied.

[0077]

[0078] For a final The final calculation results of the attention weight of each head are sorted, and the small local area block with the largest attention in each head is selected as the candidate local attention area, and the corresponding index is obtained as follows:

[0079] index=[A1,A2,…,A h ]

[0080] The corresponding candidate local features can be obtained based on the obtained local small area block index matrix.

[0081]

[0082] S2.3.2. The local guided feature learning submodule combines the candidate local features with the global features separately and then optimizes the internal encoding of the candidate local features.

[0083] Specifically, the feature information focused on by the selected local small area blocks in different attention heads is different. In order to maximize the unique features of each local salient area and enhance the relationship with the global features, the features of the selected local small area blocks and the global features are separately combined, and a Transformer encoding layer is used to construct the relationship between the global features and the local features and perform feature encoding optimization. The representation of each local feature in the global feature is enhanced by internal feature optimization, and finally the features that highlight the key local areas are obtained.

[0084] S2.3.3. Based on the attention weights of the candidate local features, the softmax normalization function is used to adaptively generate the external weights corresponding to the candidate local regions, and the weights of the external features of the candidate local regions are optimized to obtain the optimized local region features.

[0085] Specifically, the focus region selection module obtains local focus regions based on the attention weights of all encoding layers of ViT. To further balance the local region features in the final feature representation, the softmax normalization function is used to perform weight normalization based on the accumulated weights of all layers and all heads of the selected region. The weight matrix W occupied by each selected local region is adaptively learned, and the weights of different local regions in the final feature representation are adaptively adjusted and enhanced by optimizing the outer feature encoding to maximize the prominence of the distinguishable local features.

[0086] W=[w1,w2,…,w h ]

[0087] =softmax([a A1 ,a A2 ,…,a Ah ])

[0088] S2.4. The global features and key local area features are concatenated and fused after batch normalization to obtain a complete feature representation;

[0089] S2.5. Perform search and matching based on the complete feature representation to obtain matching results;

[0090] Specifically, all locally enhanced features are obtained, and the global features and locally enhanced features are respectively passed through a batch normalization (BN) layer, and are spliced ​​and fused at the dimensional level to obtain the final complete feature representation for retrieval and matching to obtain the matching results.

[0091] S2.6. Adjust the network parameters according to the matching results and the true labels to obtain a well-trained local-global adaptive guided enhanced vehicle re-identification collaborative representation network.

[0092] Specifically, the loss function for adjusting network parameters is The calculation formula is as follows:

[0093]

[0094] In the above formula, w1 and w2 represent the weights of global features and local features, which are set to 1 and 1 in the present invention. H represents the number of selected local feature blocks f g represents the global feature, f p represents the global feature, L ID represents the ID loss function, L Tri The triplet loss function shows good performance in reducing intra-class variability and increasing inter-class variability. In the triplet loss function, given a set of sample instances (a, p, n), a represents the anchor sample, p represents the sample with the same true label as the anchor sample, and n represents the sample with a different true label from the anchor sample. α is a hyperparameter used to control the network backpropagation process and is set to 0.3 in this paper.

[0095] S3. Obtain the query image and the gallery set, and perform retrieval and matching on the query image in the gallery set based on the well-trained local-global adaptive guided enhanced vehicle re-identification collaborative representation network to obtain the matching result.

[0096] like Figure 4 As shown, a vehicle re-identification system with local-global adaptive guidance enhancement includes:

[0097] A preprocessing unit, which performs image preprocessing on the training images and constructs a training set;

[0098] A network training unit trains a local-global adaptive guided enhanced vehicle re-identification collaborative representation network based on a training set to obtain a fully trained local-global adaptive guided enhanced vehicle re-identification collaborative representation network, wherein the local-global adaptive guided enhanced vehicle re-identification collaborative representation network includes a backbone network module based on Vision Transformer and an adaptive optimization feature encoding module based on local attention guidance;

[0099] The retrieval and matching unit obtains the query image and the gallery set, and retrieves and matches the query image in the gallery set based on the well-trained local-global adaptive guided enhanced vehicle re-identification collaborative representation network to obtain the matching result.

[0100] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0101] A vehicle re-identification device with local-global adaptive guidance enhancement:

[0102] at least one processor;

[0103] at least one memory for storing at least one program;

[0104] When the at least one program is executed by the at least one processor, the at least one processor implements the vehicle re-identification method with enhanced local-global adaptive guidance as described above.

[0105] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0106] A storage medium stores processor-executable instructions, wherein the processor-executable instructions are used to implement the above-mentioned vehicle re-identification method with local-global adaptive guidance enhancement when executed by the processor.

[0107] The contents of the above method embodiments are all applicable to the present storage medium embodiment. The functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0108] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A vehicle re-identification method with local-global adaptive guidance enhancement, characterized by: The following steps are involved: Perform image preprocessing on training images and construct training sets; The training set is input into the vehicle re-identification collaborative representation network enhanced by local-global adaptive guidance; Based on the Vision Transformer backbone network module, the input image is divided into small local area blocks and global and local features are extracted. The region selection submodule selects key areas according to the weight of the coding layer, obtains the index and local features of the corresponding small area blocks, and obtains candidate local features; The local guided feature learning submodule combines the candidate local features with the global features separately and then optimizes the internal encoding of the candidate local features; According to the attention weight of the candidate local feature, the softmax normalization function is used to adaptively generate the external weight corresponding to the candidate local area, and the weight of the external feature of the candidate local area is optimized to obtain the optimized local area feature; The global features and key local area features are spliced ​​and fused after batch normalization to obtain a complete feature representation; Perform retrieval and matching based on the complete feature representation to obtain matching results; The network parameters are adjusted according to the matching results and the true labels, and a well-trained local-global adaptive guided enhanced vehicle re-ID collaborative representation network is obtained; The local-global adaptive guided enhanced vehicle re-identification collaborative representation network includes a backbone network module based on VisionTransformer and an adaptive optimized feature encoding module based on local attention guidance; Obtain the query image and the gallery set, and perform retrieval and matching on the query image in the gallery set based on the well-trained local-global adaptive guided enhanced vehicle re-identification collaborative representation network to obtain the matching result; The Vision Transformer-based backbone network module includes an input encoding submodule and a feature extraction submodule. The step of dividing the input image into gridded small local area blocks and extracting global features and local features specifically includes: The input encoding submodule divides the input image into gridded small local area blocks to form a one-dimensional vector sequence input; The feature extraction submodule uses multiple Transformer encoding layers to perform global feature extraction and local feature extraction of small local blocks on the one-dimensional vector sequence input; The loss function calculation formula for adjusting network parameters based on matching results and true labels is as follows: In the above formula, w1 and w2 represent the weights of global features and local features, h represents the number of selected local feature blocks, and f g represents the global feature, f p Represents the enhanced local features, L ID represents the ID loss function, L Tri Represents the triplet loss function; in the triplet loss function, given a set of sample instances (a, p, n), a represents the anchor sample, p represents the sample with the same true label as the anchor sample, and n represents the sample with a different true label from the anchor sample.

2. The vehicle re-identification method with local-global adaptive guidance enhancement according to claim 1, characterized in that: The step of performing image preprocessing on the training images and constructing a training set specifically includes: Acquire a vehicle image for training to obtain a training image; Resize the training image to obtain an adjusted training image; Performing data augmentation on the adjusted training image to obtain an enhanced training image; The training set is constructed by combining the augmented training images and their corresponding true labels.

3. Local-global adaptive guidance enhanced vehicle re-identification system, characterized by: The vehicle re-identification method for performing the local-global adaptive guidance enhancement as claimed in claim 1 comprises: A preprocessing unit, which performs image preprocessing on the training images and constructs a training set; A network training unit trains a local-global adaptive guided enhanced vehicle re-identification collaborative representation network based on a training set to obtain a fully trained local-global adaptive guided enhanced vehicle re-identification collaborative representation network, wherein the local-global adaptive guided enhanced vehicle re-identification collaborative representation network includes a backbone network module based on Vision Transformer and an adaptive optimization feature encoding module based on local attention guidance; The retrieval and matching unit obtains the query image and the gallery set, and retrieves and matches the query image in the gallery set based on the well-trained local-global adaptive guided enhanced vehicle re-identification collaborative representation network to obtain the matching result.

Images