Vehicle re-identification method and device based on hidden space feature distance optimization and double-branch self-attention enhancement
By introducing latent space feature distance optimization and a dual-branch self-attention enhancement module into the ResNet network, the problems of intra-class differences and inter-class similarities in vehicle re-identification are solved, and the recognition accuracy and cross-camera tracking accuracy are improved.
Patent Information
- Application Number
- CN202510692906.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-27
- Publication Date
- 2025-09-23
AI Technical Summary
Existing deep learning-based vehicle re-identification methods fail to effectively solve the problems of intra-class differences and inter-class similarities in vehicle re-identification tasks, resulting in insufficient recognition accuracy and inability to meet practical application needs.
A latent space feature distance optimization module and a dual-branch self-attention enhancement module are designed in the ResNet network structure. By aggregating the features of the same vehicle in the latent space and enhancing the feature differences, the L2 distance and cosine distance are used to measure the similarity. The dual-branch self-attention enhancement module is combined to prevent redundant information learning.
It improves the performance of vehicle re-identification, enhances the discriminability and recognition accuracy of features, and improves the accuracy of cross-camera vehicle tracking.
Smart Images
Figure CN120689822A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer vision image processing, and specifically relates to a vehicle re-identification method and device based on latent space feature distance optimization and dual-branch self-attention enhancement. Background Art
[0002] Vehicle re-identification (ReID) is a key subtask of cross-camera vehicle tracking and is crucial for the development of smart urban transportation systems. With the increasing number of vehicles and the widespread deployment of intelligent surveillance equipment, the amount of vehicle video surveillance data has surged. Traditional manual recognition is not only slow but also lacks re-ID accuracy, making it unable to meet the needs of practical applications. Existing deep learning-based methods mostly ignore the common intra-class differences and inter-class similarities in vehicle re-ID tasks, resulting in limited performance. Summary of the Invention
[0003] The purpose of the present invention is to provide a vehicle re-identification method and device based on latent space feature distance optimization and dual-branch self-attention enhancement. A latent space feature distance optimization module is designed in the original ResNet network structure, and a dual-branch self-attention enhancement module is adopted in the fourth layer to expand the difference of dual-branch features and enhance the discriminability of features.
[0004] To achieve the above object, the technical solution adopted by the present invention is:
[0005] A vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement includes the following steps:
[0006] S1. Acquire a vehicle image and perform preprocessing on the vehicle image, including image resizing and image enhancement;
[0007] S2. Input the preprocessed vehicle image into the deep learning model to obtain features used to describe the vehicle image;
[0008] S3. Calculate the feature distance between the query image and the gallery image using a similarity metric. The smaller the distance, the more likely they are the same vehicle.
[0009] The deep learning model includes: a ResNet backbone network, a latent space feature distance optimization module, and a dual-branch self-attention enhancement module, wherein:
[0010] The ResNet backbone network receives the preprocessed vehicle image I and outputs feature maps F1, F2, F3, and F4 of four sizes through a four-layer structure;
[0011] The latent space feature distance optimization module first maps features into the latent space, clusters visually similar features of the same vehicle, and then maps the features into the final feature space. This method implements an adaptive distance constraint mechanism that allows for more structured aggregation of features in the latent space. Identical vehicles with similar visual features are clustered together, while identical vehicles with significant visual differences may be grouped into different clusters to avoid forcibly bringing features closer together, leading to overfitting and affecting feature discriminability.
[0012] The dual-branch self-attention enhancement module first copies the fourth-layer residual block as two branches. The two branches are independent of each other and do not share weights. Then, the convolution in the residual block is replaced with grouped convolution. Each group performs convolution operations independently without interfering with each other. The two branches respectively introduce spatial attention and channel attention for feature enhancement to prevent the two branches from learning redundant information and enhance feature differences.
[0013] The similarity metric uses both L2 distance and cosine distance to characterize the true distance between vehicle features to measure vehicle similarity. For each query image, library images of the same vehicle taken by the same camera must be removed to achieve cross-camera vehicle tracking.
[0014] Several optional methods are also provided below, but they are not intended to be additional limitations on the above-mentioned overall solution. They are merely further supplements or optimizations. Under the premise that there are no technical or logical contradictions, each optional method can be combined separately for the above-mentioned overall solution, or multiple optional methods can be combined.
[0015] Preferably, the preprocessing described in step S1 includes scaling the vehicle image to an image size of 256×256; the four-layer ResNet structure is defined as the first layer structure, the second layer structure, the third layer structure and the fourth layer structure according to the data flow direction; the first layer structure includes a Conv2d convolution layer, a batch normalization layer, a ReLU activation function, a Maxpool2d layer and a Bottleneck block connected in sequence from the data input side to the output side, and each layer structure in the second layer structure, the third layer structure and the fourth layer structure includes a Bottleneck block from the data input side to the output side; the stride of the first three layers is 2, and the width and height of the image are reduced by half after each layer. In order to retain details, the stride of the fourth layer is set to 1. The first layer structure outputs a feature map F1∈R 256×64×64 , the second layer structure outputs the feature map F2∈R 512×32×32 , the third layer structure outputs the feature map F3∈R 1024×16×16 , the fourth layer structure outputs the feature map F4∈R 2048×16×16 ;
[0016] Preferably, the latent space feature distance optimization module defines several prototypes, each of which contains shared semantic information in the feature space. In the latent space, the L2 distance between the vehicle feature and each prototype is calculated. Based on the distance, the probability of the vehicle feature belonging to each prototype is calculated. The vehicle feature is matched with the prototype based on the probability, and the feature is guided to move closer to its corresponding prototype.
[0017] As a preference, the spatial attention branch outputs the first three layers of residual blocks It is generated by three 1×1 convolutions and reshaped into The three matrices are connected with the original feature residual after the self-attention operation to obtain the spatial attention enhanced feature The channel attention branch is generated by three depth-wise separable convolutions and reshaped into Matrix, after self-attention operation, is connected with the original feature residual to obtain channel attention enhanced feature
[0018] Preferably, the fourth layer of the ResNet backbone network adopts group convolution with a group number of 2, wherein the first half channel features are used to calculate the triplet loss:
[0019]
[0020] Where: m, Δ, P(a), N(a) are the batch size, margin distance, positive sample set and negative sample set of anchors in the same batch respectively. are the features of the anchor, positive, and negative samples of the i-th branch respectively. The second half channel features are used to calculate the label smoothed cross entropy loss:
[0021]
[0022] Where: is the predicted probability of softmax output, where D is the number of vehicle IDs. is the label of the i-th image after label smoothing:
[0023]
[0024] ∈ is the smoothing parameter;
[0025] As a preference, for the i-th branch, the first half channel of the fourth layer structure output is taken to calculate the L2 distance Take the output of the back half channel through the fully connected layer to calculate the cos distance And add the distances of the two branches to get the final distance Where Q and G represent the query image and gallery image respectively. Re-ID is achieved by sorting the query image and all gallery images based on their final distance.
[0026] A second aspect of the present invention relates to a vehicle re-identification device based on latent space feature distance optimization and dual-branch self-attention enhancement, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement of the present invention.
[0027] A third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement of the present invention.
[0028] The present invention designs a latent space feature distance optimization module in the original ResNet network structure, and adopts a dual-branch self-attention enhancement module in the fourth layer to expand the difference of dual-branch features, enhance the discriminability of features, and ultimately improve the re-identification performance.
[0029] The advantages of the present invention are: by constructing category-independent semantic prototypes, guiding feature adaptive aggregation in the latent space in a probabilistic matching manner, and designing a dual-branch structure based on channel attention and spatial attention to improve the diversity of feature representation, enhance the discriminability of features, and ultimately improve the re-identification performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 Flowchart of the vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement of the present invention;
[0031] Figure 2 Schematic diagram of the structure of the deep learning model of the present invention;
[0032] Figure 3 It is a schematic diagram of the device of the present invention. DETAILED DESCRIPTION
[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.
[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.
[0035] Example 1
[0036] This embodiment provides a vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement to improve vehicle re-identification performance. Figure 1 As shown, the method includes the following steps:
[0037] S1. Acquire a vehicle image and preprocess the vehicle image.
[0038] To meet the neural network input requirements, this embodiment requires scaling the vehicle image to an image size of 256 × 256. The preprocessing in this embodiment focuses on image size scaling. In other embodiments, other image preprocessing, such as random horizontal flipping, may be performed to improve image quality.
[0039] S2. Input the preprocessed vehicle image into the deep learning model to obtain features used to describe the vehicle image.
[0040] like Figure 2 As shown in FIG, the deep learning model includes: a ResNet backbone network, a latent space feature distance optimization module, and a dual-branch self-attention enhancement module. The following describes the deep learning model used in this embodiment by introducing each module in detail.
[0041] 1) ResNet backbone network: receives the pre-processed vehicle image I and outputs feature maps F1, F2, F3, and F4 of four sizes through a four-layer structure. First, the input image is obtained through the first three layers of the structure. C3, H, and W are the number of channels, height, and width of the output features of the third layer, respectively. This feature is passed through two independent branches of the fourth layer. and After global average pooling (GAP), two features are obtained and This method replaces the convolution in the fourth layer structure with group convolution, with the number of groups being 2. This allows the features to be divided into 2 groups by channel, with the triplet loss calculated for the first and second half groups respectively. and cross entropy loss This avoids the mutual interference of the two losses. The final loss function can be written as:
[0042]
[0043] Where α, β and γ are the weight coefficients of cross entropy loss, triple loss and loss of training latent space feature distance optimization module respectively.
[0044] 2) Hidden space feature distance optimization module: First, randomly initialize K prototype vectors The space composed of the feature F3 output by the first three layers is defined as the latent space. Then, the L2 distance between the feature F3 and each prototype is calculated:
[0045] d k =‖F3-V k ‖2
[0046] where d k represents the L2 distance from feature F3 to the k-th prototype. Then, the probability of the vehicle feature belonging to each prototype is calculated based on the distance. In order to satisfy the properties of probability, the probability of matching to the k-th prototype is calculated as follows:
[0047]
[0048] Here, λ is a hyperparameter that controls the hardness of the probability assignment. The larger λ is, the greater the probability of assigning to the prototype with the smallest distance. Prototypes are then matched according to probability. The matching method is to randomly sample prototypes based on the probability and match the sampled prototypes to the features. Assuming that the kth prototype is matched, the L2 loss function is used to close the distance between the feature and the prototype:
[0049]
[0050] Finally, in order to avoid excessive concentration of prototypes during learning, it is necessary to impose a limit on the minimum distance between prototypes. Therefore, the distance from each prototype to its closest prototype is calculated and this distance is tried to be increased:
[0051]
[0052] Intuitively Pull the output features closer to the corresponding prototype, Pull the prototypes further apart. Finally, add these two losses together to form the loss function for training this module:
[0053]
[0054] 3) Two-branch self-attention enhancement module: For the spatial attention branch, first use three independent 1×1 convolution f Q ,f K ,f V Map feature F3 and then adjust the feature to:
[0055] Q s =f reshape (f Q (F3), HW×C3)
[0056] K s =f reshape (f K(F3), HW×C3)
[0057] V s =f reshape (f V (F3), HW×C3)
[0058] where Q s , K s , V s is the final result of mapping and expansion, f reshape The (·,HW×C3) operation represents the dimension Adjust to Then use the self-attention mechanism to implement spatial attention:
[0059]
[0060] Among them F s It is the result of self-attention. Finally, the residual mechanism is introduced to obtain the output of the spatial attention branch
[0061]
[0062] where f reshape The (·, C3×H×W) operation means adjusting the feature dimension back to
[0063] For the channel attention branch, similarly, three independent depth-wise separable convolutions are first used to obtain features. Then use the self-attention mechanism to achieve channel attention:
[0064]
[0065] Finally, the residual mechanism is used to obtain the output of the channel attention branch
[0066]
[0067] S3. Calculate the feature distance between the query image and the gallery image according to a certain similarity metric.
[0068] Specifically, taking the first branch as an example, suppose that after passing the fourth layer The features of the query set and gallery set obtained after global average pooling are and Where N and M are the number of images in the query set and gallery set respectively, and C4 is the feature dimension. Since the fourth layer uses a group convolution with a group number of 2, the feature can be divided into two groups by channel, which are recorded as and The number of channels in each group is For the features used to calculate the metric learning loss and Directly calculate their L2 distance:
[0069]
[0070] For the features used to calculate the classification loss and Use the predicted confidence (output of the fully connected layer) to calculate the cosine distance as the distance between the query set and the gallery set:
[0071]
[0072] in and Represent the prediction confidence of the query set and the gallery set respectively. Similarly, use the same method to calculate the distance of the second branch, and then sum up all the distances to get the final distance:
[0073]
[0074] Finally, the final distance between each query image and all the gallery images is sorted to achieve re-identification.
[0075] Example 2
[0076] like Figure 3 This embodiment relates to a vehicle re-identification device based on latent space feature distance optimization and dual-branch self-attention enhancement, including a memory and one or more processors. The memory stores executable code. When the one or more processors execute the executable code, they are used to implement the vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement of Example 1.
[0077] Example 3
[0078] This embodiment relates to a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement of Example 1 is implemented.
[0079] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0080] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person skilled in the art would be able to make numerous modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.
Claims
1. A vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement, characterized in that: The following steps are involved: S1. Acquire a vehicle image and preprocess the vehicle image; S2. Input the preprocessed vehicle image into the deep learning model to obtain features used to describe the vehicle image; S3. Calculate the feature distance between the query image and the gallery image according to a certain similarity metric; The deep learning model includes: a ResNet backbone network, a latent space feature distance optimization module, and a dual-branch self-attention enhancement module, wherein: The ResNet backbone network receives the preprocessed vehicle image I and outputs feature maps F1, F2, F3, and F4 of four sizes through a four-layer structure; The latent space feature distance optimization module first maps features into the latent space, clusters visually similar features of the same vehicle, and then maps the features into the final feature space. This method implements an adaptive distance constraint mechanism that allows for more structured aggregation of features in the latent space. Identical vehicles with similar visual features are clustered together, while identical vehicles with significant visual differences may be grouped into different clusters to avoid forcibly bringing features closer together, leading to overfitting and affecting feature discriminability. The dual-branch self-attention enhancement module first copies the fourth-layer residual block as two branches. The two branches are independent of each other and do not share weights. Then, the convolution in the residual block is replaced with grouped convolution. Each group performs convolution operations independently without interfering with each other. The two branches respectively introduce spatial attention and channel attention for feature enhancement to prevent the two branches from learning redundant information and enhance feature differences. The similarity metric uses both L2 distance and cosine distance to characterize the true distance between vehicle features to measure vehicle similarity. For each query image, library images of the same vehicle taken by the same camera must be removed to achieve cross-camera vehicle tracking.
2. The vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement as claimed in claim 1, characterized in that: The preprocessing described in step S1 includes scaling the vehicle image to an image size of 256×256; the four-layer ResNet structure is defined as the first layer structure, the second layer structure, the third layer structure, and the fourth layer structure according to the data flow direction; the first layer structure consists of a Conv2d convolution layer, a batch normalization layer, a ReLU activation function, a Maxpool2d layer, and a Bottleneck block connected in sequence from the data input side to the output side, and each layer structure in the second layer structure, the third layer structure, and the fourth layer structure includes a Bottleneck block from the data input side to the output side; the stride of the first three layers is 2, and the width and height of the image are reduced by half after each layer. To retain details, the stride of the fourth layer is set to 1. The first layer structure outputs a feature map F1∈R 256×64×64 , the second layer structure outputs the feature map F2∈R 512×32×32 , the third layer structure outputs the feature map F3∈R 1024×16×16 , the fourth layer structure outputs the feature map F4∈R 2048×16×16 .
3. The vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement as claimed in claim 1, characterized in that: The latent space feature distance optimization module defines several prototypes, each of which contains shared semantic information in the feature space. In the latent space, the L2 distance between the vehicle feature and each prototype is calculated. Based on the distance, the probability of the vehicle feature belonging to each prototype is calculated. The vehicle feature is then matched with the prototype using this probabilistic approach, guiding the feature toward its corresponding prototype.
4. The vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement as claimed in claim 1, characterized in that: The specific implementation of the self-attention enhancement module includes: the spatial attention branch outputs the first three layers of residual blocks Generated by three 1×1 convolutions and reshaped into Q s , K s 、 The three matrices are connected with the original feature residual after the self-attention operation to obtain the spatial attention enhanced feature The channel attention branch is generated by three depth-wise separable convolutions and reshaped into Q c , K c 、 Matrix, after self-attention operation, is connected with the original feature residual to obtain channel attention enhanced feature 5. The vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement as claimed in claim 1, characterized in that: The fourth layer of the ResNet backbone network uses a group convolution with a group number of 2, where the first half channel features are used to calculate the triplet loss: Where: m, Δ, P(a), N(a) are the batch size, margin distance, positive sample set and negative sample set of anchors in the same batch respectively. are the features of the anchor, positive, and negative samples of the i-th branch respectively. The second half channel features are used to calculate the label smoothed cross entropy loss: Where: is the predicted probability of softmax output, where D is the number of vehicle IDs. is the label of the i-th image after label smoothing: ∈ is the smoothing parameter.
6. The vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement as claimed in claim 5, characterized in that: For the i-th branch, take the first half channel of the fourth layer structure output to calculate the L2 distance Take the output of the back half channel through the fully connected layer to calculate the cos distance And add the distances of the two branches to get the final distance Where Q and G represent the query image and gallery image respectively. Re-identification is achieved by sorting each query image and all gallery images based on their final distance.
7. A vehicle re-identification device based on latent space feature distance optimization and dual-branch self-attention enhancement is characterized by: The invention comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement according to any one of claims 1 to 6.
8. A computer-readable storage medium, characterized in that A program is stored thereon, and when the program is executed by a processor, the vehicle re-identification method based on latent space feature distance optimization and dual-branch self-attention enhancement according to any one of claims 1 to 6 is implemented.