Vehicle re-identification method and device based on vehicle global attribute perception and local discriminant feature learning

Abstract

This application discloses a vehicle re-identification method and device based on vehicle global attribute perception and local discriminant feature learning. The method includes: obtaining a vehicle re-identification dataset and inputting it into a backbone network to obtain vehicle features; inputting the vehicle features into a global attribute perception module based on a Transformer mechanism, a vehicle local region perception module based on an attention mechanism, and a pyramid-shaped local fine-grained feature extraction module to obtain the vehicle's global attribute features, local region features, and local fine-grained features, and respectively calculating a global attribute loss function, a local region loss function, and a local fine-grained loss function; training the backbone network based on the above loss functions; inputting the vehicle re-identification dataset into the trained backbone network to obtain multiple vehicle image features, calculating the similarity of the multiple vehicle image features, and arranging them according to the similarity to obtain the vehicle re-identification results. This application can improve the recognition accuracy of vehicle re-identification.

Description

Technical Field

[0001] The present application relates to the field of vehicle re-identification technology, and in particular to a vehicle re-identification method and device based on vehicle global attribute perception and local discriminant feature learning. Background Art

[0002] With the rapid development of modern transportation systems and the continuous expansion of urban areas, vehicles have become a crucial element in urban operations. Accurately identifying and tracking vehicles is crucial in numerous fields, including intelligent traffic management, security monitoring, and smart parking. Against this backdrop, vehicle re-identification (VRI) technology has emerged. The goal of VRI is to retrieve and match the target vehicle from images captured by non-overlapping cameras, without relying on information such as the license plate. Traditional vehicle recognition relies primarily on license plate recognition (LPR). However, in real-world applications, license plates can be obscured (e.g., blurred due to long-term use), poor lighting conditions (strong direct sunlight, backlight, or low light), and limited camera angles, causing the LPR system to fail or significantly reduce its accuracy. Inspired by deep learning and other methods, existing VRI models begin by integrating the vehicle's overall visual image, including multi-dimensional visual information such as body color, model, and texture, and mining its inherent feature patterns. This provides a powerful solution to the challenges of vehicle tracking and management in scenarios where LPR is limited.

[0003] However, existing deep learning-based vehicle re-identification methods still face two major challenges: first, images taken from different cameras cause huge changes in vehicle and camera perspectives, resulting in huge differences within the class; second, different vehicles may have the same color, model and other attributes, resulting in extreme similarities between classes.

[0004] To address the above issues, existing methods can be divided into three main categories: First, some methods use deep convolutional neural networks and introduce additional annotated vehicle attribute information, such as vehicle color and model, to jointly learn vehicle identity feature representation. Although this global attribute information is not easily affected by changes in camera and vehicle perspectives and is rich in vehicle identity-related information, using only the global attribute information of the vehicle cannot capture the key visual differences to address the inter-class similarity challenge; Second, some methods attempt to learn key discriminative features from key local regions of the vehicle to compensate for the shortcomings of the vehicle's global attribute features. Although these methods can effectively address the inter-class similarity challenge by using the differences in the local regions of the vehicle, the performance of such methods depends largely on the detection accuracy of the local regions. Moreover, not all local regions can provide key discriminative information for vehicle identity feature representation, and the importance of different regions needs to be evaluated; Third, in response to the advantages and disadvantages of the first two categories of methods, some methods introduce multi-branch structures to learn global and local features of the vehicle from different scales. However, this type of method only relies on the vehicle's identity ID information to constrain the model at the macro level. Since the vehicle's identity ID information has no direct correlation with the vehicle's global appearance and local area features, it is difficult to learn global and local discriminant features. Summary of the Invention

[0005] The embodiments of the present application provide a vehicle re-identification method and device based on vehicle global attribute perception and local discriminant feature learning, which can improve the recognition accuracy of vehicle re-identification.

[0006] The present embodiment provides a vehicle re-identification method based on global vehicle attribute perception and local discriminant feature learning, including:

[0007] Get the vehicle re-identification dataset;

[0008] Inputting vehicle images in the vehicle re-identification dataset into the backbone network to obtain vehicle features;

[0009] Inputting the vehicle features into a global attribute perception module based on the Transformer mechanism to obtain the vehicle's global attribute features, and calculating a global attribute loss function based on the global attribute features;

[0010] Inputting the vehicle features into a vehicle local area perception module based on an attention mechanism to obtain local area features of the vehicle, and calculating a local area loss function based on the local area features;

[0011] Inputting the vehicle features into a pyramid-shaped local fine-grained feature extraction module to obtain local fine-grained features of the vehicle, and calculating a local fine-grained loss function based on the local fine-grained features;

[0012] Training the backbone network based on the global attribute loss function, the local area loss function, and the local fine-grained loss function to obtain a trained backbone network;

[0013] The vehicle re-identification dataset is input into the trained backbone network to obtain multiple vehicle image features, the similarity of the multiple vehicle image features is calculated, and the features are arranged according to the similarity to obtain a vehicle re-identification result.

[0014] Furthermore, in the vehicle re-identification method based on global vehicle attribute perception and local discriminant feature learning, the vehicle features are input into a global attribute perception module based on a Transformer mechanism to obtain the vehicle's global attribute features, including:

[0015] Extracting attribute features from the vehicle features through a convolution block, and extracting attribute feature vectors corresponding to the attribute features through a global average pooling operation;

[0016] Reshaping the attribute features to obtain a spatial information sequence;

[0017] The attribute feature vector and the spatial information sequence are passed through the cross attention in the global attribute perception module and then input into the feedforward network to obtain the global attribute features of the vehicle.

[0018] Furthermore, in the vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning, the step of calculating a global attribute loss function based on the global attribute features includes:

[0019] Optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on vehicle attribute label information, and calculating the first edge loss function of the global attribute perception module based on the Transformer mechanism;

[0020] Calculating a first cross entropy loss function based on the vehicle attribute label and the global attribute feature;

[0021] A global attribute loss function is obtained based on the first edge loss function and the first cross entropy loss function.

[0022] Furthermore, in the vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning, the vehicle features are input into a vehicle local area perception module based on an attention mechanism to obtain the vehicle local area features, including:

[0023] Extracting local area original features from the image mask information and the vehicle features;

[0024] The importance features of different key areas in the original features of the local area are learned through a mask-based attention mechanism to obtain the key area features of the vehicle;

[0025] The key area features of the vehicle are separated and fused through the area relationship interaction module to obtain the local area features of the vehicle.

[0026] Furthermore, in the vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning, the calculation of the local area loss function based on the local area features includes:

[0027] Optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on vehicle identity ID tag information, and calculating a second edge loss function of the vehicle local area perception module based on the attention mechanism;

[0028] Calculating a second cross entropy loss function based on the vehicle identity ID tag information and the local area features;

[0029] A local area loss function is obtained based on the second edge loss function and the second cross entropy loss function.

[0030] Furthermore, in the vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning, the pyramid-shaped local fine-grained feature extraction module includes three stages. Inputting the vehicle features into the pyramid-shaped local fine-grained feature extraction module to obtain the vehicle's local fine-grained features includes:

[0031] Carry out three stages of learning respectively to obtain high-level features and average-level features in each stage;

[0032] The high-level features and the average level features are used as local fine-grained features of the corresponding stage.

[0033] Furthermore, in the vehicle re-identification method based on global vehicle attribute perception and local discriminant feature learning, the calculation of the local fine-grained loss function based on the local fine-grained features includes:

[0034] Optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on vehicle identity ID tag information, and calculating the third edge loss function of the pyramid-shaped local fine-grained feature extraction module;

[0035] Calculating a third cross entropy loss function based on the vehicle identity ID tag information and the local fine-grained features;

[0036] A local fine-grained loss function is obtained based on the third edge loss function and the third cross entropy loss function.

[0037] Furthermore, the vehicle re-identification method based on global vehicle attribute perception and local discriminant feature learning further includes:

[0038] Before training the backbone network, performing a data enhancement strategy on the vehicle re-identification dataset;

[0039] During the training of the backbone network, the learning rate The adjustment strategy can be expressed as follows:

[0040]

[0041] in, is the maximum learning rate, is the minimum learning rate, Indicates the number of iterations of the current training, Indicates the maximum number of iterations for model training.

[0042] Furthermore, the above-mentioned vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning further includes: obtaining the image mask information;

[0043] The acquiring of the image mask information comprises:

[0044] Inputting the vehicle re-identification dataset into a vehicle key part detector to generate a boundary detection box;

[0045] Obtaining coordinates of key parts of the vehicle based on the boundary detection frame;

[0046] A vehicle component enclosing area is obtained based on the vehicle component position, and the image mask information is defined by the range of the vehicle component enclosing area.

[0047] The present application also provides a vehicle re-identification device based on global vehicle attribute perception and local discriminant feature learning, including:

[0048] Acquisition module, used to obtain vehicle re-identification dataset;

[0049] A first recognition module is used to input vehicle images in the vehicle re-identification dataset into a backbone network to obtain vehicle features;

[0050] A calculation module is configured to input the vehicle features into a global attribute perception module based on a Transformer mechanism to obtain global attribute features of the vehicle, and calculate a global attribute loss function based on the global attribute features; input the vehicle features into a vehicle local area perception module based on an attention mechanism to obtain local area features of the vehicle, and calculate a local area loss function based on the local area features; and input the vehicle features into a pyramid-shaped local fine-grained feature extraction module to obtain local fine-grained features of the vehicle, and calculate a local fine-grained loss function based on the local fine-grained features;

[0051] A training module, configured to train the backbone network based on the global attribute loss function, the local area loss function, and the local fine-grained loss function to obtain a trained backbone network;

[0052] The second recognition module is used to input the vehicle re-identification dataset into the trained backbone network to obtain multiple vehicle image features, calculate the similarity of the multiple vehicle image features, and arrange them according to the similarity to obtain a vehicle re-identification result.

[0053] An embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store multiple instructions and data, wherein the instructions are suitable for being loaded by a processor to execute any of the above-mentioned vehicle re-identification methods based on vehicle global attribute perception and local discriminant feature learning, and the data includes data input by any of the above-mentioned vehicle re-identification methods based on vehicle global attribute perception and local discriminant feature learning.

[0054] The present application provides a vehicle re-identification method and device based on vehicle global attribute perception and local discriminant feature learning. The present application fully extracts vehicle features by setting a global attribute perception module, a vehicle local area perception module, and a fine-grained feature extraction module, and trains the backbone module through these three modules, which can achieve the following effects: (1) Strong domain adaptation ability; the invention can show excellent re-identification results on datasets in different domains such as those with drastic changes in perspective, low image pixels, uneven vehicle attributes, and day and night cross-domains. (2) Capture of coarse-grained and fine-grained information; the invention makes full use of the vehicle global attribute information, key local information, and local subtle discriminant features contained in the vehicle image, and realizes the accurate representation of vehicle identity features from coarse to fine. (3) High vehicle re-identification accuracy; the invention realizes the coordinated representation of vehicle identity by accurately extracting vehicle global attribute information and local discriminant information, effectively addresses the challenges of inter-class similarity and intra-class difference in vehicle re-identification, and greatly improves the accuracy of vehicle re-identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The following detailed description of the specific embodiments of the present application in conjunction with the accompanying drawings will make the technical solutions and other beneficial effects of the present application apparent.

[0056] Figure 1 Flowchart of a vehicle re-identification method based on global vehicle attribute perception and local discriminant feature learning provided in an embodiment of the present application.

[0057] Figure 2 Schematic diagram of the vehicle re-identification method provided in an embodiment of the present application.

[0058] Figure 3 This is an information display diagram of the vehicle re-identification dataset provided in an embodiment of the present application.

[0059] Figure 4 A flowchart of local fine-grained feature extraction provided in an embodiment of the present application.

[0060] Figure 5 A comparison chart of mAP, CMC@1, and CMC@5 indicators on the VeRi-776 dataset provided in the embodiments of this application.

[0061] Figure 6 A comparison chart of the CMC@1 and CMC@5 indicators on the VehicleID dataset provided in the embodiment of this application.

[0062] Figure 7 This is a comparison chart of mAP, CMC@1, and CMC@5 indicators on the VeRi-Wild dataset provided in the examples of this application.

[0063] Figure 8 Schematic diagram of the results of the ablation experiments on each submodule provided in the embodiment of the present application.

[0064] Figure 9 Schematic diagram of the results of the L-PAM module neutron module ablation experiment provided in the embodiment of the present application.

[0065] Figure 10 The vehicle re-identification performance index provided in the embodiment of the present application is affected by the boundary loss function parameter and , and AAT module importance parameters Schematic diagram of the impact.

[0066] Figure 11 Schematic diagram showing how vehicle re-identification performance indicators are affected by different loss functions in an embodiment of the present application.

[0067] Figure 12 A schematic diagram of the structure of a vehicle re-identification device based on global vehicle attribute perception and local discriminant feature learning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0069] The present invention provides a vehicle re-identification method and apparatus based on global vehicle attribute perception and local discriminant feature learning. The present invention provides a vehicle re-identification apparatus based on global vehicle attribute perception and local discriminant feature learning, which can be integrated into a terminal, server, or other device. The terminal may include a tablet computer, laptop computer, personal computer (PC), microprocessor box, or other device.

[0070] The following first explains and illustrates the technical terms of the present invention:

[0071] Image mask: An image mask is a tool used to control the processing of image pixels. It is typically an image or data structure of the same size (or dimensions) as the original image. The value of each pixel in the image determines whether the corresponding pixel in the original image is processed, how it is processed, or whether it is visible, helping the model better focus on the specified area.

[0072] Average Precision (AP): In person and vehicle re-identification tasks, AP is a metric that measures a model's ranking performance on a single category. For a specific category, the model first ranks samples from that category based on their similarity scores with samples from other categories. The area under the Precision-Recall curve is the AP.

[0073] Mean Average Precision (mAP): mAP is the average of the APs across multiple categories. In real-world person and vehicle re-ID tasks, there are often multiple different pedestrian (or vehicle) categories. mAP comprehensively considers the performance across all categories, providing a more comprehensive assessment of model performance.

[0074] Cumulative Matching Characteristic (CMC): The CMC curve is primarily used to measure the probability of finding a correct match at different ranking positions. In person or vehicle re-ID tasks, for a query sample (such as a pedestrian or a vehicle), the model ranks all samples in the database by similarity. The CMC curve's horizontal axis represents the ranking position (e.g., [a person's name]), and the vertical axis represents the probability of finding a correct match within the ranking.

[0075] Euclidean distance: is a method to measure the distance between two points or two vectors in multidimensional space. When measuring the feature similarity of two vehicle images, the model can obtain the vector representation of the identity features of the two images. and , then the Euclidean distance can be expressed as .

[0076] See also Figure 1 and Figure 2 , Figure 1 This is a flow chart of a vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning provided in an embodiment of the present application. Figure 2 This is a schematic diagram of a vehicle re-identification method provided in an embodiment of the present application, which is applied to an electronic device. The vehicle re-identification method based on global vehicle attribute perception and local discriminant feature learning includes the following steps:

[0077] S1, obtain the vehicle re-identification dataset.

[0078] Among them, the vehicle re-identification dataset includes vehicle image data and annotation data.

[0079] Figure 3 This is an information display diagram of the vehicle re-identification dataset provided in the embodiment of this application. Figure 3 Three public datasets, VeRi-776, VehicleID, and VeRi-Wild, used to train the backbone network model are presented. Each dataset contains a different number of vehicle instances and images, and is divided into a training set and a test set. The present invention utilizes the attribute information of the vehicle for model training. When a dataset lacks a certain attribute, the attribute will not be considered to constrain the model to avoid overfitting the model. Specifically, in the VeRi-776 dataset, two attributes, vehicle color and model, are used; in the VehicleID dataset, two attributes, vehicle color and manufacturer, are used; and in the VeRi-Wild dataset, three attributes, vehicle color, model, and manufacturer, are used.

[0080] In one embodiment, step S1 includes the following steps:

[0081] S11, obtain the original image data in the vehicle re-identification dataset .

[0082] In the vehicle re-identification task, each image data in the dataset It is the raw data from which the model extracts vehicle identity features.

[0083] S12, obtain vehicle ID information .

[0084] S13, obtain the Vehicle global attribute (color, model, manufacturer) labels .

[0085] in, Indicates the number of vehicle attribute annotation information possessed by the current training dataset. Different datasets have different numbers of attribute information and the categories contained in each attribute information.

[0086] S14, obtain the image mask information corresponding to the key local areas (headlights, windows, and vehicle logo) of the vehicle image .

[0087] Specifically, step S14 includes:

[0088] S141: Input the vehicle re-identification dataset into the vehicle key part detector to generate a boundary detection box.

[0089] Use the YOLO target detection model to pre-train detectors for key vehicle components (headlights, windows, and vehicle logos) and generate bounding boxes for key components in the vehicle re-identification dataset.

[0090] S142, obtaining the coordinates of key parts of the vehicle based on the boundary detection frame.

[0091] Get the Coordinate representation of the position of vehicle components :

[0092]

[0093] For vehicle images that are not correctly detected due to image blur or occlusion, the average position of the parts in other images of the vehicle instance is calculated. , as the position of the vehicle parts in the image, is calculated as follows:

[0094] =

[0095] S143 , obtaining a vehicle component enclosing area based on the vehicle component position, and defining image mask information based on the range of the vehicle component enclosing area.

[0096] No. The area enclosed by the coordinates of the vehicle components is defined as , then Image masks corresponding to vehicle parts It can be defined as:

[0097]

[0098] Image masks of vehicle parts It can be expressed as , Represents the number of predefined key local areas of a vehicle.

[0099] S15, finally get the input of the backbone network :

[0100]

[0101] S2, inputs the vehicle images in the vehicle re-identification dataset into the backbone network to obtain vehicle features.

[0102] Specifically, ResNet-50 pre-trained on ImageNet is used as the backbone network to build a benchmark framework for extracting image features from the vehicle re-identification dataset.

[0103] The backbone network allows vehicles with the same identity from different cameras to learn consistent feature representations. , the output features can be obtained through the backbone network .

[0104] For the backbone network parameters, use the loss function It is constrained by a combination of cross-entropy loss and soft-margin loss.

[0105] For features , we use a convolutional block ( ), which consists of a The convolutional layer ( ), a batch normalization layer ( ), an activation layer ( ) and a global average pooling layer ( ) to obtain a vector Used to calculate the loss function , It can be expressed as: .

[0106] The vehicle re-identification task is regarded as a classification task, and the cross entropy loss is used for constraint. Considering the imbalance of sample distribution, the label-smoothing technology is integrated to prevent the model from over-trusting positive samples. The vehicle re-identification task is regarded as a metric learning task, and the soft margin loss function is used to allow the distance between positive and negative samples to be maintained within a certain interval. The loss function It can be expressed as:

[0107]

[0108]

[0109] in, represents the number of vehicle instances in the vehicle re-identification dataset, Indicates the identity ID corresponding to the current vehicle image, is a fully connected layer used to predict body classification, and They represent the Euclidean distance between the reference sample and the positive sample and the Euclidean distance between the reference sample and the negative sample, respectively. is a small constraint constant in the label smoothing technique.

[0110] S3, inputs the vehicle features into the global attribute perception module based on the Transformer mechanism to obtain the global attribute features of the vehicle, and calculates the global attribute loss function based on the global attribute features.

[0111] A Transformer-based global attribute awareness module (AAT) is pre-built. First, the attribute information contained in the current training dataset is clearly defined. For each attribute, cross-attention is used as the basis of the Transformer mechanism.

[0112] In one embodiment, step S3 includes the following steps:

[0113] S31, extract attribute features from vehicle features through convolution blocks, and extract attribute feature vectors corresponding to the attribute features through global average pooling operations.

[0114] Using different convolutional blocks ( ) From vehicle characteristics Get the first Attribute characteristics , use the global average pooling operation (GAP) to obtain the attribute feature vector :

[0115]

[0116]

[0117] S32, reshape the dimension of attribute features to obtain spatial information sequence .

[0118] Define the sinusoidal position embedding sequence :

[0119]

[0120] in, express The channel dimension index of , express The spatial position index of express spatial dimension.

[0121] S33, after the attribute feature vector and the spatial information sequence pass through the cross attention in the global attribute perception module, they are input into the feedforward network to obtain the global attribute features of the vehicle.

[0122] The outputs Q, K, V of the cross attention structure in the AAT module can be defined as:

[0123]

[0124]

[0125]

[0126] After the cross attention structure, the feed-forward network (FFN) is used to obtain the global attribute features of the vehicle. :

[0127]

[0128] S34, optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on the vehicle attribute label information, and calculating the first edge loss function of the global attribute perception module based on the Transformer mechanism.

[0129] Specifically, calculate the marginal loss function based on the global attributes of the vehicle ,The feature distance between positive and negative samples of vehicle images is optimized based on the vehicle attribute label information.

[0130] Step S34 specifically includes:

[0131] S341, unlike the backbone network which uses soft-margin loss, the AAT module uses margin loss to further reduce the distance between positive samples based on global attribute features and increase the difference between instances. It can be expressed as:

[0132]

[0133]

[0134] in, Indicates the samples in a batch and The Euclidean distance between and Indicates the upper and lower boundary thresholds.

[0135] S342, first edge loss based on vehicle attributes in AAT module It can be expressed as:

[0136]

[0137] S35, further, pre-construct a post-processing module (Post-process Block) for separating the classification task feature space and the measurement task feature space. In the AAT module, the learning process is used express.

[0138] A post-processing block is pre-built to separate the feature space. When constraining vehicle attribute features, if both classification loss and metric loss constraints are applied to the same feature, it is easy to cause inconsistent gradient directions and fluctuations. Therefore, a post-processing block is added between the metric loss and classification loss to separate the feature space.

[0139] Among them, the Post-process Block contains a The convolutional layer ( ), a batch normalization layer ( ), an activation layer ( ) and a global average pooling layer ( ).use Indicates this embedding operation.

[0140] S36, calculating a first cross entropy loss function based on the vehicle attribute label and the global attribute features.

[0141] Calculate the first cross entropy loss function based on the vehicle's global attributes , constrain the AAT network parameters based on the vehicle attribute label information;

[0142] Vehicle attribute features are processed through the post-processing module to calculate the attribute-based classification loss. :

[0143]

[0144] in, represents the number of vehicle attributes, Indicates the attribute tags, represents the fully connected layer, Indicates the post-processing module in the AAT module.

[0145] S37: Obtain a global attribute loss function based on the first edge loss function and the first cross entropy loss function.

[0146] Calculate the global attribute loss function :

[0147]

[0148] S4, inputs the vehicle features into the vehicle local area perception module based on the attention mechanism to obtain the local area features of the vehicle, and calculates the local area loss function based on the local area features.

[0149] Among them, a vehicle local area perception module (L-PAM) based on the attention mechanism is pre-built.

[0150] In one embodiment, step S4 includes the following steps:

[0151] S41, extracting local area original features from the image mask information and vehicle features.

[0152] Utilize image mask information of key areas of the vehicle Output features from the backbone network Extract local area original features :

[0153]

[0154] in, Represents an adaptive average pooling operation for alignment and dimension.

[0155] S42, learns the importance features of different key areas in the original features of the local area through the mask-based attention mechanism to obtain the key area features of the vehicle.

[0156] Step S42 specifically includes:

[0157] S421, activate the mask-based average pooling operation The specific operation can be expressed as:

[0158]

[0159] S422, learning the importance scores of different key areas based on attention mechanism :

[0160]

[0161]

[0162] in, Represents a series of network learning processes, Represents a learnable parameter.

[0163] S423, obtaining weighted vehicle key area features :

[0164]

[0165] S43, separating and fusing the key area features of the vehicle through the area relationship interaction module to obtain the local area features of the vehicle.

[0166] Specifically, step S43 includes:

[0167] S431, using mask From the weighted vehicle key area features Separate the features of each region :

[0168]

[0169] in, Indicates flipping the mask elements.

[0170] S432, for Characteristics of the region , established with other regional characteristics The relationship between the two, and the fused features are obtained :

[0171]

[0172] S44, optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on the vehicle identity ID tag information, and calculating the second edge loss function of the vehicle local area perception module based on the attention mechanism.

[0173] Specifically, the second edge loss function based on the key local area of the vehicle is calculated ,The feature distance between positive and negative samples of vehicle images is optimized based on the vehicle identity ID tag information.

[0174] Second edge loss in L-PAM modules It can be expressed as:

[0175]

[0176] Among them for In the calculation of, in the L-PAM module, The definition can be expressed as:

[0177]

[0178] Furthermore, the post-processing block is used to separate the classification task feature space and the measurement task feature space. In the L-PAM module, the learning process is performed using express.

[0179] S45, calculating a second cross entropy loss function based on the vehicle identity ID tag information and the local area features.

[0180] Calculate the second cross entropy loss function based on the key local area of the vehicle , constrain the L-PAM module network parameters based on the vehicle identity ID tag information.

[0181] The key local area features of the vehicle are processed through the post-processing module to calculate the second cross entropy loss based on the key local area of the vehicle :

[0182]

[0183] in, represents the number of predefined vehicle critical areas, Represents the vehicle identity ID label in the dataset, represents the fully connected layer, Represents the learning process of the post-processing module in the L-PAM module.

[0184] S46, obtaining a local area loss function based on the second edge loss function and the second cross entropy loss function.

[0185] Calculate the local area loss function :

[0186]

[0187] S5: Input the vehicle features into the pyramid-shaped local fine-grained feature extraction module to obtain the local fine-grained features of the vehicle, and calculate the local fine-grained loss function based on the local fine-grained features.

[0188] Among them, a pyramid-shaped local fine-grained feature extraction module (FGFE) is pre-built. The FGFE module consists of a series of fine-grained feature extraction blocks (Fine-Grained Lock, FGB). The input of the FGB module can obtain two outputs through the upper and lower branches. Figure 4The following is a flowchart of the local fine-grained feature extraction provided in this embodiment of the application. Figure 4 .

[0189] In one embodiment, step S5 includes the following steps:

[0190] S51, conduct three stages of learning respectively to obtain high-level features and average-level features in each stage.

[0191] Step S51 specifically includes:

[0192] S511, for FGB module, input , the upper branch uses adaptive maximum pooling to extract high-level features as the input of the next layer FGB module , the process can be expressed as:

[0193]

[0194] in, Indicates that the current FGB block is in the first position of the FGFE pyramid network. stage, Indicates that the current FGB block is in the first position of the FGFE pyramid network. layer.

[0195] S512, for FGB module, input , the lower branch first obtains the activation features along the channel direction , the process can be expressed as:

[0196]

[0197] S513, for activation features , calculate the high level eigenvalue along the horizontal direction :

[0198]

[0199] S514, will Middle The row height horizontal feature value is assigned to 0:

[0200]

[0201] S515, After removing high-level features, we get , and uses adaptive average pooling operation to extract the overall average level features :

[0202]

[0203] S516, FGB block lower branch output Will be the first Part of the input to the FGB block in the subsequent stage of the layer.

[0204] S517, FGFE network structure is composed of a series of FGB blocks in a pyramid structure, including stages, each stage contains different levels , and The corresponding relationship can be described as:

[0205]

[0206] S518, for stage The learning process can be described as:

[0207]

[0208] in, Indicates the stage of learning process, Indicates the The original input of the stage, the input of each stage It can be expressed as:

[0209]

[0210]

[0211]

[0212] S52, the high-level features and the average-level features are used as the local fine-grained features of the corresponding stage.

[0213] Specifically, the high-level features of the last layer of each stage are and average level characteristics As the local fine-grained features of this stage , used for subsequent fine-grained feature loss function calculation.

[0214] S53, optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on the vehicle identity ID tag information, and calculating the third edge loss function of the pyramid-shaped local fine-grained feature extraction module.

[0215] Specifically, the third edge loss function based on the local fine-grained features of the vehicle is calculated ,The feature distance between positive and negative samples of vehicle images is optimized based on the vehicle identity ID tag information.

[0216] Third edge loss in FGFE module It can be expressed as:

[0217]

[0218] Among them for In the calculation of, in the FGFE module, The definition can be expressed as:

[0219]

[0220] Furthermore, the post-processing block is used to separate the classification task feature space and the measurement task feature space. In the FGFE module, the learning process is performed using express.

[0221] S54, calculating a third cross entropy loss function based on the vehicle identity ID tag information and the local fine-grained features.

[0222] Specifically, calculate the third cross entropy loss function , constrain the FGFE module network parameters based on the vehicle identity ID tag information.

[0223] The local fine-grained features of the vehicle are processed through the post-processing module to calculate the third cross entropy loss :

[0224]

[0225] in, represents the number of predefined vehicle critical areas, represents the vehicle identity in the dataset, Represents the learning process of the post-processing module in the FGFE module.

[0226] S55: Obtain a local fine-grained loss function based on the third edge loss function and the third cross entropy loss function.

[0227] Calculate local fine-grained loss function :

[0228]

[0229] S6, train the backbone network based on the global attribute loss function, the local area loss function and the local fine-grained loss function to obtain a trained backbone network.

[0230] The multi-scale loss function is calculated based on the global attribute loss function, the local area loss function and the local fine-grained loss function. Consistency constraints are imposed on the model parameters of the backbone network:

[0231]

[0232] in, It is a hyperparameter used to balance the importance of the AAT module for the entire model to learn the vehicle identity feature representation. In this invention, based on the experimental results, we set This is because although the vehicle attribute information can effectively cope with changes in vehicle perspective and solve the challenge of intra-class differences, it also causes inter-class similarity challenges because different vehicles may have similar attributes.

[0233] In one embodiment, the vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning further includes the following steps:

[0234] A1. Before training the backbone network, data augmentation strategies are performed on the vehicle re-identification dataset, including boundary padding, random erasing, and random flipping.

[0235] A2, the learning rate during backbone network training The adjustment strategy can be expressed as follows:

[0236]

[0237] in, is the maximum learning rate, is the minimum learning rate, Indicates the number of iterations of the current training, Indicates the maximum number of iterations for model training.

[0238] Figure 5 This is a comparison chart of mAP, CMC@1, and CMC@5 indicators on the VeRi-776 dataset provided in the embodiment of this application. Figure 6 This is a comparison chart of the CMC@1 and CMC@5 indicators on the VehicleID dataset provided in the embodiment of this application. Figure 7 The comparison chart of mAP, CMC@1 and CMC@5 indicators on the VeRi-Wild dataset provided in the embodiment of this application is as follows: Figure 5-Figure 7It can be seen that: (1) the performance indicators of the present invention on the three datasets are all better than all other comparison models, which proves the advancement of the present invention; (2) on the three datasets, due to the differences in the dataset conditions, the experimental performance on each dataset is different, but all of them can surpass the most advanced methods on the current dataset, which shows that the present invention has strong domain adaptation capabilities in different environments and different situations; (3) among all the comparison methods, the methods that combine global and local features have achieved better results. This is because these methods comprehensively consider the overall macro information of the vehicle and the local subtle difference information. However, compared with the methods used in the present invention, these methods only use the vehicle's identity ID information to constrain the learning of global features and local features, and it is difficult to extract effective vehicle identity-related information for vehicle re-identification; (4) among all the comparison models, the vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning achieved the best results. In the present invention, the learning of global features of the vehicle is constrained by the vehicle's attributes, and the learning of local features is divided into the learning of vehicle key area features and the learning of local fine-grained features. These features contain rich vehicle identity-related information, so they can achieve excellent performance in vehicle re-identification.

[0239] Furthermore, ablation experiments are conducted to demonstrate the effectiveness of each module. Figure 8 Schematic diagram of the results of the ablation experiment of each submodule provided in the embodiment of this application, Figure 9 This is a schematic diagram of the results of the L-PAM module neutron module ablation experiment provided in the embodiment of the present application. Figure 8 The experimental results of training different modules on the VeRi-776 dataset are shown. A multi-branch network structure is adopted in this invention. The experimental results show that the combination of different modules is better than using a single module, which shows that different modules can make up for each other's shortcomings. At the same time, when combining the AAT module, L-PAM module and FGFE module, the model achieved the best performance of 85.2% and 97.8% on mAP and CMC@1 respectively. Figure 9 The ablation experiment results of the MAM and PIM submodules in the L-PAM module are shown. It can be seen that when the MAM module or the PIM module is not used in the L-PAM module, the experimental results are almost the same as those without using the entire L-PAM module. This shows that there is a strong correlation between the MAM module and the PIM module and they are an indispensable part of the L-PAM module.

[0240] Figure 10 The vehicle re-identification performance index provided in the embodiment of the present application is affected by the boundary loss function parameter and , and AAT module importance parameters Schematic diagram of the impact, Figure 11Schematic diagram showing how vehicle re-identification performance indicators are affected by different loss functions in an embodiment of the present application. Figure 10 Shows the three hyperparameters of the model , and ,in To balance the importance of the AAT module for the entire model to learn the vehicle identity feature representation, in this invention, based on the experimental results, we set This is because although the vehicle attribute information can effectively cope with changes in vehicle perspective and solve the challenge of intra-class differences, it also causes inter-class similarity challenges because different vehicles may have similar attributes; and Experimentally, they are set to 1 and 0.3 respectively to constrain the distance between positive and negative samples.

[0241] S7: Input the vehicle re-identification dataset into the trained backbone network to obtain multiple vehicle image features, calculate the similarity of the multiple vehicle image features, and arrange them according to the similarity to obtain the vehicle re-identification result.

[0242] Specifically, the trained backbone network is used to extract the features of the vehicle image to be retrieved (query) in the vehicle re-identification dataset, and the trained backbone network is also used to extract the features of the vehicle image in the image gallery (gallery) in the vehicle re-identification dataset. The Euclidean distance is used to calculate the similarity between the features of the vehicle image to be retrieved and the features of the vehicle image in the image gallery. The gallery images are arranged according to the similarity to obtain the vehicle re-identification results.

[0243] This application fully extracts vehicle features by setting up a global attribute perception module, a vehicle local area perception module, and a fine-grained feature extraction module. By training the backbone module through these three modules, the following effects can be achieved:

[0244] (1) Strong domain adaptation capability: This invention can demonstrate excellent re-identification results on datasets in different domains, such as datasets with drastic changes in viewing angles, low image pixels, uneven vehicle attributes, and day and night cross-domains.

[0245] (2) Capturing coarse-grained and fine-grained information: This invention makes full use of the vehicle's global attribute information, key local information, and local subtle discriminant features contained in the vehicle image, and realizes the accurate representation of the vehicle's identity features from coarse to fine.

[0246] (3) High vehicle re-identification accuracy: This invention realizes the coordinated representation of vehicle identity by accurately extracting global attribute information and local discriminant information of the vehicle, effectively responding to the challenges of inter-class similarity and intra-class difference in vehicle re-identification, and greatly improving the accuracy of vehicle re-identification.

[0247] Based on the method described in the above embodiment, this embodiment will be further described from the perspective of a vehicle re-identification device based on vehicle global attribute perception and local discriminant feature learning. The vehicle re-identification device based on vehicle global attribute perception and local discriminant feature learning can be implemented as an independent entity or integrated into a terminal, server, or other device, wherein the terminal may include a tablet computer, a laptop computer, a personal computer (PC), a micro processing box, or other devices.

[0248] See also Figure 12 , Figure 12 The present invention specifically describes a vehicle re-identification device based on vehicle global attribute perception and local discriminant feature learning, which is applied to an electronic device. The vehicle re-identification device based on vehicle global attribute perception and local discriminant feature learning may include:

[0249] Acquisition module, used to obtain vehicle re-identification dataset;

[0250] A first recognition module is used to input vehicle images in the vehicle re-identification dataset into a backbone network to obtain vehicle features;

[0251] A calculation module is configured to input the vehicle features into a global attribute perception module based on a Transformer mechanism to obtain global attribute features of the vehicle, and calculate a global attribute loss function based on the global attribute features; input the vehicle features into a vehicle local area perception module based on an attention mechanism to obtain local area features of the vehicle, and calculate a local area loss function based on the local area features; and input the vehicle features into a pyramid-shaped local fine-grained feature extraction module to obtain local fine-grained features of the vehicle, and calculate a local fine-grained loss function based on the local fine-grained features;

[0252] A training module, configured to train the backbone network based on the global attribute loss function, the local area loss function, and the local fine-grained loss function to obtain a trained backbone network;

[0253] The second recognition module is used to input the vehicle re-identification dataset into the trained backbone network to obtain multiple vehicle image features, calculate the similarity of the multiple vehicle image features, and arrange them according to the similarity to obtain a vehicle re-identification result.

[0254] During specific implementation, the above modules and / or units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above modules and / or units can refer to the previous method embodiments. The specific beneficial effects that can be achieved can also be found in the beneficial effects in the previous method embodiments, which will not be repeated here.

[0255] The above is a detailed introduction to the vehicle re-identification method and device based on vehicle global attribute perception and local discriminant feature learning provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A vehicle re-identification method based on global vehicle attribute perception and local discriminant feature learning, characterized by: The method comprises: Get the vehicle re-identification dataset; Inputting the vehicle re-identification dataset into a backbone network to obtain vehicle features; Inputting the vehicle features into a global attribute perception module based on the Transformer mechanism to obtain the vehicle's global attribute features, and calculating a global attribute loss function based on the global attribute features; Inputting the vehicle features into a vehicle local area perception module based on an attention mechanism to obtain local area features of the vehicle, and calculating a local area loss function based on the local area features; Inputting the vehicle features into a pyramid-shaped local fine-grained feature extraction module to obtain local fine-grained features of the vehicle, and calculating a local fine-grained loss function based on the local fine-grained features; Training the backbone network based on the global attribute loss function, the local area loss function, and the local fine-grained loss function to obtain a trained backbone network; The vehicle re-identification dataset is input into the trained backbone network to obtain multiple vehicle image features, the similarity of the multiple vehicle image features is calculated, and the features are arranged according to the similarity to obtain a vehicle re-identification result.

2. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 1 is characterized in that: The vehicle characteristics are input into a global attribute perception module based on a Transformer mechanism to obtain the vehicle's global attribute characteristics, including: Extracting attribute features from the vehicle features through a convolution block, and extracting attribute feature vectors corresponding to the attribute features through a global average pooling operation; Reshaping the attribute features to obtain a spatial information sequence; The attribute feature vector and the spatial information sequence are passed through the cross attention in the global attribute perception module and then input into the feedforward network to obtain the global attribute features of the vehicle.

3. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 2 is characterized in that: The calculating of the global attribute loss function based on the global attribute feature includes: Optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on vehicle attribute label information, and calculating the first edge loss function of the global attribute perception module based on the Transformer mechanism; Calculating a first cross entropy loss function based on the vehicle attribute label and the global attribute feature; A global attribute loss function is obtained based on the first edge loss function and the first cross entropy loss function.

4. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 1 is characterized in that: Inputting the vehicle features into a vehicle local area perception module based on an attention mechanism to obtain the vehicle local area features includes: Extracting local area original features from the image mask information and the vehicle features; The importance features of different key areas in the original features of the local area are learned through a mask-based attention mechanism to obtain the key area features of the vehicle; The key area features of the vehicle are separated and fused through the area relationship interaction module to obtain the local area features of the vehicle.

5. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 4 is characterized in that: The calculating of the local area loss function based on the local area features includes: Optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on vehicle identity ID tag information, and calculating a second edge loss function of the vehicle local area perception module based on the attention mechanism; Calculating a second cross entropy loss function based on the vehicle identity ID tag information and the local area features; A local area loss function is obtained based on the second edge loss function and the second cross entropy loss function.

6. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 5 is characterized in that: The pyramid-shaped local fine-grained feature extraction module includes three stages. The vehicle features are input into the pyramid-shaped local fine-grained feature extraction module to obtain the local fine-grained features of the vehicle, including: Carry out three stages of learning respectively to obtain high-level features and average-level features in each stage; The high-level features and the average level features are used as local fine-grained features of the corresponding stage.

7. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 6 is characterized in that: The calculating of the local fine-grained loss function based on the local fine-grained features includes: Optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on vehicle identity ID tag information, and calculating the third edge loss function of the pyramid-shaped local fine-grained feature extraction module; Calculating a third cross entropy loss function based on the vehicle identity ID tag information and the local fine-grained features; A local fine-grained loss function is obtained based on the third edge loss function and the third cross entropy loss function.

8. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 1 is characterized in that: The method further comprises: Before training the backbone network, performing a data enhancement strategy on the vehicle re-identification dataset; During the training of the backbone network, the learning rate The adjustment strategy can be expressed as follows: in, is the maximum learning rate, is the minimum learning rate, Indicates the number of iterations of the current training, Indicates the maximum number of iterations for model training.

9. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 4 is characterized in that: The method further includes: acquiring the image mask information; The acquiring of the image mask information comprises: Inputting the vehicle re-identification dataset into a vehicle key part detector to generate a boundary detection box; Obtaining the coordinates of key parts of the vehicle based on the boundary detection frame; A vehicle component enclosing area is obtained based on the vehicle component position, and the image mask information is defined by the range of the vehicle component enclosing area.

10. A vehicle re-identification device based on vehicle global attribute perception and local discriminant feature learning, characterized in that: include: Acquisition module, used to obtain vehicle re-identification dataset; A first recognition module is used to input vehicle images in the vehicle re-identification dataset into a backbone network to obtain vehicle features; A calculation module is configured to input the vehicle features into a global attribute perception module based on a Transformer mechanism to obtain global attribute features of the vehicle, and calculate a global attribute loss function based on the global attribute features; input the vehicle features into a vehicle local area perception module based on an attention mechanism to obtain local area features of the vehicle, and calculate a local area loss function based on the local area features; and input the vehicle features into a pyramid-shaped local fine-grained feature extraction module to obtain local fine-grained features of the vehicle, and calculate a local fine-grained loss function based on the local fine-grained features; A training module, configured to train the backbone network based on the global attribute loss function, the local area loss function, and the local fine-grained loss function to obtain a trained backbone network; The second recognition module is used to input the vehicle re-identification dataset into the trained backbone network to obtain multiple vehicle image features, calculate the similarity of the multiple vehicle image features, and arrange them according to the similarity to obtain a vehicle re-identification result.

Images