Vehicle re-identification method and device based on vehicle global attribute perception and local discriminant feature learning

Abstract

The invention discloses a vehicle re-identification method and device based on vehicle global attribute perception and local discriminant feature learning. The method comprises the following steps: acquiring a vehicle re-identification data set, and inputting the vehicle re-identification data set into a backbone network to obtain vehicle features; inputting the vehicle features into a global attribute sensing module based on a Transform mechanism, a vehicle local area sensing module based on an attention mechanism and a pyramid-shaped local fine-grained feature extraction module to obtain global attribute features, local area features and local fine-grained features of the vehicle; calculating a global attribute loss function, a local area loss function and a local fine-grained loss function respectively; training the backbone network based on the loss function; and inputting the vehicle re-identification data set into the trained backbone network to obtain a plurality of vehicle picture features, calculating the similarity of the plurality of vehicle picture features, and arranging the vehicle picture features according to the similarity to obtain a vehicle re-identification result. According to the invention, the recognition precision of vehicle re-recognition can be improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle re-identification technology, and in particular to a vehicle re-identification method and device based on vehicle global attribute perception and local discriminant feature learning. Background Art

[0002] With the rapid development of modern transportation systems and the continuous expansion of urban scale, vehicles have become an extremely critical element in urban operations. In many fields such as intelligent traffic management, security monitoring, and intelligent parking, it is of great significance to accurately identify and track vehicles. In this context, vehicle re-identification technology has emerged. The goal of the vehicle re-identification task is to retrieve and match the target vehicle from the vehicle images taken by non-overlapping cameras without relying on information such as license plates. Traditional vehicle recognition mainly relies on license plate recognition technology. However, in actual application scenarios, license plates may be blocked (such as blurred due to long-term use), poor lighting conditions (strong light, backlight or low light environment), and limited shooting angles. Factors such as the failure of the license plate recognition system or a significant reduction in accuracy. Inspired by methods such as deep learning, the existing vehicle re-identification model starts with the overall visual image of the vehicle, such as multi-dimensional visual information such as body color, model, and texture, and mines its inherent feature patterns, providing a powerful solution to the problem of vehicle tracking and management under limited license plate recognition.

[0003] However, existing deep learning-based vehicle re-identification methods still face two major challenges: first, images taken from different cameras cause huge changes in vehicle and camera perspectives, resulting in huge differences within the class; second, different vehicles may have the same color, model and other attributes, resulting in extreme similarities between classes.

[0004] In order to deal with the above problems, existing methods are mainly divided into three categories: First, some methods use deep convolutional neural networks and introduce additional annotated vehicle attribute information, such as vehicle color, vehicle model, etc., to jointly learn vehicle identity feature representation. Although these global attribute information is not easily affected by changes in camera and vehicle perspectives, and is rich in vehicle identity-related information, using only the global attribute information of the vehicle cannot capture key visual differences to address the inter-class similarity challenge; second, some methods try to learn key discriminant features from key local areas of the vehicle to make up for the lack of global attribute features of the vehicle. Although these methods can effectively address the inter-class similarity challenge through the differences in local areas of the vehicle, the performance of such methods depends largely on the detection accuracy of the local areas, and not all local areas can provide key discernible information for vehicle identity feature representation, and the importance of different areas needs to be evaluated; third, in view of the advantages and disadvantages of the first two methods, some methods introduce multi-branch structures to learn global and local features of vehicles from different scales. However, this type of method only relies on the vehicle's identity ID information to constrain the model at the macro level. Since the vehicle's identity ID information is not directly related to the vehicle's global appearance and local area features, it is difficult to learn global and local discriminant features. Summary of the invention

[0005] The embodiments of the present application provide a vehicle re-identification method and device based on vehicle global attribute perception and local discriminant feature learning, which can improve the recognition accuracy of vehicle re-identification.

[0006] The embodiment of the present application provides a vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning, including: Get the vehicle re-identification dataset; Inputting vehicle images in the vehicle re-identification dataset into the backbone network to obtain vehicle features; Inputting the vehicle features into a global attribute perception module based on a Transformer mechanism to obtain global attribute features of the vehicle, and calculating a global attribute loss function based on the global attribute features; Inputting the vehicle features into a vehicle local area perception module based on an attention mechanism to obtain local area features of the vehicle, and calculating a local area loss function based on the local area features; Inputting the vehicle features into a pyramid-shaped local fine-grained feature extraction module to obtain local fine-grained features of the vehicle, and calculating a local fine-grained loss function based on the local fine-grained features; Training the backbone network based on the global attribute loss function, the local area loss function and the local fine-grained loss function to obtain a trained backbone network; The vehicle re-identification data set is input into the trained backbone network to obtain multiple vehicle image features, the similarity of the multiple vehicle image features is calculated, and they are arranged according to the similarity to obtain a vehicle re-identification result.

[0007] Furthermore, in the above-mentioned vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning, the vehicle features are input into a global attribute perception module based on a Transformer mechanism to obtain the global attribute features of the vehicle, including: Extracting attribute features from the vehicle features through a convolution block, and extracting attribute feature vectors corresponding to the attribute features through a global average pooling operation; Reshaping the dimension of the attribute features to obtain a spatial information sequence; The attribute feature vector and the spatial information sequence are input into a feedforward network after being cross-attended in the global attribute perception module to obtain the global attribute features of the vehicle.

[0008] Furthermore, in the above-mentioned vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning, the global attribute loss function is calculated based on the global attribute feature, including: Optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on the vehicle attribute label information, and calculating the first edge loss function of the global attribute perception module based on the Transformer mechanism; Calculate a first cross entropy loss function based on the vehicle attribute label and the global attribute feature; A global attribute loss function is obtained based on the first edge loss function and the first cross entropy loss function.

[0009] Furthermore, in the above-mentioned vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning, the step of inputting the vehicle features into a vehicle local area perception module based on an attention mechanism to obtain the vehicle local area features comprises: Extracting local area original features from the image mask information and the vehicle features; The importance features of different key areas in the original features of the local area are learned through a mask-based attention mechanism to obtain the key area features of the vehicle; The key area features of the vehicle are separated and fused through the area relationship interaction module to obtain the local area features of the vehicle.

[0010] Furthermore, in the above-mentioned vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning, the calculation of the local area loss function based on the local area features includes: Optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on vehicle identity ID tag information, and calculating the second edge loss function of the vehicle local area perception module based on the attention mechanism; Calculate a second cross entropy loss function based on the vehicle identity ID tag information and the local area features; A local area loss function is obtained based on the second edge loss function and the second cross entropy loss function.

[0011] Furthermore, in the above-mentioned vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning, the pyramid-shaped local fine-grained feature extraction module includes three stages, wherein the vehicle features are input into the pyramid-shaped local fine-grained feature extraction module to obtain the local fine-grained features of the vehicle, including: Carry out three stages of learning respectively to obtain high-level features and average-level features in each stage; The high-level features and the average level features are used as local fine-grained features of the corresponding stage.

[0012] Furthermore, in the above-mentioned vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning, the calculation of the local fine-grained loss function based on the local fine-grained features includes: Optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on vehicle identity ID tag information, and calculating the third edge loss function of the pyramid-shaped local fine-grained feature extraction module; Calculating a third cross entropy loss function based on the vehicle identity ID tag information and the local fine-grained features; A local fine-grained loss function is obtained based on the third edge loss function and the third cross entropy loss function.

[0013] Furthermore, the above vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning, wherein the method further comprises: Before training the backbone network, performing a data enhancement strategy on the vehicle re-identification dataset; During the training of the backbone network, the learning rate Adjustment strategy can be expressed as follows:

[0014] in, is the maximum learning rate, is the minimum learning rate, Indicates the number of iterations of the current training. Indicates the maximum number of iterations for model training.

[0015] Furthermore, the above-mentioned vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning, wherein the method further comprises: obtaining the image mask information; The acquiring the image mask information comprises: Inputting the vehicle re-identification data set into a vehicle key part detector to generate a boundary detection box; Obtaining the coordinates of key parts of the vehicle based on the boundary detection frame; A vehicle component enclosing area is obtained based on the vehicle component position, and the image mask information is defined by the range of the vehicle component enclosing area.

[0016] The embodiment of the present application also provides a vehicle re-identification device based on vehicle global attribute perception and local discriminant feature learning, including: An acquisition module is used to acquire a vehicle re-identification dataset; A first recognition module, used for inputting vehicle images in the vehicle re-identification dataset into a backbone network to obtain vehicle features; A calculation module, used for inputting the vehicle features into a global attribute perception module based on a Transformer mechanism to obtain global attribute features of the vehicle, and calculating a global attribute loss function based on the global attribute features; inputting the vehicle features into a vehicle local area perception module based on an attention mechanism to obtain local area features of the vehicle, and calculating a local area loss function based on the local area features; and inputting the vehicle features into a pyramid-shaped local fine-grained feature extraction module to obtain local fine-grained features of the vehicle, and calculating a local fine-grained loss function based on the local fine-grained features; A training module, used for training the backbone network based on the global attribute loss function, the local area loss function and the local fine-grained loss function to obtain a trained backbone network; The second recognition module is used to input the vehicle re-identification data set into the trained backbone network to obtain multiple vehicle image features, calculate the similarity of the multiple vehicle image features, and arrange them according to the similarity to obtain the vehicle re-identification result.

[0017] An embodiment of the present application also provides a computer operation storage device readable storage medium, wherein the computer operation storage device is used to store multiple instructions and data, wherein the instructions are suitable for being loaded by a processor to execute any of the above-mentioned vehicle re-identification methods based on vehicle global attribute perception and local discriminant feature learning, and the data includes data input by any of the above-mentioned vehicle re-identification methods based on vehicle global attribute perception and local discriminant feature learning.

[0018] The present application provides a vehicle re-identification method and device based on vehicle global attribute perception and local discriminant feature learning. The present application fully extracts vehicle features by setting a global attribute perception module, a vehicle local area perception module and a fine-grained feature extraction module, and trains the backbone module through these three modules, which can achieve the following effects: (1) Strong domain adaptation ability: The invention can show excellent re-identification results on data sets of different domains such as drastic changes in perspective, low image pixels, uneven vehicle attributes, day and night cross-domains, etc. (2) Capture of coarse-grained-fine-grained information: The invention makes full use of the vehicle global attribute information, vehicle key local information and vehicle local subtle discriminant features contained in the vehicle image, from coarse to fine, to achieve accurate representation of vehicle identity features. (3) High vehicle re-identification accuracy: The invention achieves coordinated representation of vehicle identity by accurately extracting vehicle global attribute information and local discriminant information, effectively responds to the challenges of inter-class similarity and intra-class difference in vehicle re-identification, and greatly improves the accuracy of vehicle re-identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The technical solution and other beneficial effects of the present application will be made apparent by describing in detail the specific implementation methods of the present application in conjunction with the accompanying drawings.

[0020] Figure 1 A flowchart of a vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning provided in an embodiment of the present application.

[0021] Figure 2 A schematic diagram of a vehicle re-identification method provided in an embodiment of the present application.

[0022] Figure 3 This is an information display diagram of the vehicle re-identification dataset provided in an embodiment of the present application.

[0023] Figure 4 A flowchart of local fine-grained feature extraction provided in an embodiment of the present application.

[0024] Figure 5 A comparison chart of mAP, CMC@1 and CMC@5 indicators on the VeRi-776 dataset provided in the embodiment of the present application.

[0025] Figure 6 A comparison chart of the CMC@1 and CMC@5 indicators on the VehicleID dataset provided in an embodiment of the present application.

[0026] Figure 7 This is a comparison chart of mAP, CMC@1 and CMC@5 indicators on the VeRi-Wild dataset provided in the embodiment of the present application.

[0027] Figure 8A schematic diagram of the results of ablation experiments on each submodule provided in an embodiment of the present application.

[0028] Fig. 9 Schematic diagram of the results of the submodule ablation experiment of the L-PAM module provided in the embodiment of the present application.

[0029] Fig.10 The vehicle re-identification performance index provided in the embodiment of the present application is affected by the boundary loss function parameter and , and AAT module importance parameters Schematic diagram of the impact.

[0030] Fig.11 A schematic diagram showing how the vehicle re-identification performance index provided in an embodiment of the present application is affected by different loss functions.

[0031] Fig.12 A schematic diagram of the structure of a vehicle re-identification device based on vehicle global attribute perception and local discriminant feature learning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0033] The embodiment of the present application provides a vehicle re-identification method and device based on vehicle global attribute perception and local discriminant feature learning. The embodiment of the present application provides a vehicle re-identification device based on vehicle global attribute perception and local discriminant feature learning, which can be integrated in a terminal, server and other devices, wherein the terminal can include a tablet computer, a laptop computer, a personal computer (PC, Personal Computer), a micro processing box, or other devices.

[0034] The following first explains and illustrates the technical terms of the present invention: Image mask: An image mask is a tool used to control the processing of image pixels. It is usually an image or data structure of the same size (or the same dimension) as the original image, in which the value of each pixel is used to determine whether the corresponding pixel in the original image is processed, how it is processed, or whether it is visible, helping the model to better focus on the specified area.

[0035] Average Precision (AP): In the tasks of person re-identification and vehicle re-identification, AP is an indicator to measure the ranking performance of the model on a single category. For a specific category, the model must first rank the samples of this category based on the similarity scores of the samples of other categories. The area under the Precision - Recall curve is the AP.

[0036] Mean Average Precision (mAP): mAP is the average of APs of multiple categories. In actual person re-identification and vehicle re-identification tasks, there are usually multiple different pedestrian (or vehicle) categories. mAP comprehensively considers the performance of all categories and can more comprehensively evaluate the performance of the model.

[0037] Cumulative Matching Characteristic (CMC): The CMC curve is mainly used to measure the probability of finding a correct match at different ranking positions. In pedestrian re-identification or vehicle re-identification tasks, for a query sample (such as a pedestrian or a car), the model will rank all samples in the database by similarity. The horizontal axis of the CMC curve is the ranking position (for example), and the vertical axis is the probability of finding a correct match within the ranking.

[0038] Euclidean distance: is a method to measure the distance between two points or two vectors in multidimensional space. When measuring the feature similarity of two vehicle images, the model can obtain the vector representation of the identity features of the two images. and , then the Euclidean distance can be expressed as .

[0039] See also Figure 1 and Figure 2 , Figure 1 A flowchart of a vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning provided in an embodiment of the present application, Figure 2 The schematic diagram of the vehicle re-identification method provided in the embodiment of the present application is applied in an electronic device. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning includes the following steps: S1, obtain the vehicle re-identification dataset.

[0040] Among them, the vehicle re-identification dataset includes vehicle image data and annotation data.

[0041] Figure 3 This is an information display diagram of the vehicle re-identification data set provided in the embodiment of the present application. Figure 3Three public datasets VeRi-776, VehicleID, and VeRi-Wild are presented for training backbone network models. Each dataset contains a different number of vehicle instances and images, and is divided into training and test sets. The present invention uses the attribute information of the vehicle for model training. When a dataset lacks a certain attribute, the attribute will not be considered for constraining the model to avoid overfitting the model. Specifically, in the VeRi-776 dataset, two attributes, vehicle color and model, are used; in the VehicleID dataset, two attributes, vehicle color and manufacturer, are used; and in the VeRi-Wild dataset, three attributes, vehicle color, model, and manufacturer, are used.

[0042] In one embodiment, step S1 includes the following steps: S11, obtain the original image data in the vehicle re-identification dataset .

[0043] In the vehicle re-identification task, each image data in the dataset It is the raw data from which the model extracts vehicle identity features.

[0044] S12, obtaining vehicle ID information .

[0045] S13, obtain the Vehicle global attributes (color, model, manufacturer) labels .

[0046] in, It indicates the number of vehicle attribute annotation information in the current training data set. Different data sets have different number of attribute information and the categories contained in each attribute information. S14, obtaining the image mask information corresponding to the key local areas (headlights, windows, and vehicle logo) of the vehicle image .

[0047] Specifically, step S14 includes: S141, inputting the vehicle re-identification data set into the vehicle key part detector to generate a boundary detection box.

[0048] Use the YOLO target detection model to pre-train the detectors for key vehicle components (headlights, windows, and vehicle logos) and generate boundary detection boxes for the key components of the vehicle re-identification dataset.

[0049] S142, obtaining the coordinates of key parts of the vehicle based on the boundary detection frame.

[0050] Get the Coordinate representation of the position of vehicle components :

[0051] For vehicle images that are not correctly detected due to image blur or occlusion, the average position of the parts in other images of the vehicle instance is calculated. , as the position of the vehicle parts in the image, is calculated as follows: =

[0052] S143, obtaining a vehicle component enclosing area based on the vehicle component position, and defining image mask information according to the range of the vehicle component enclosing area.

[0053] No. The coordinate enclosing area of ​​vehicle components is defined as , then Image masks corresponding to vehicle parts It can be defined as:

[0054] Image masks of vehicle parts It can be expressed as , Represents the number of predefined critical local areas of the vehicle.

[0055] S15, finally get the input of the backbone network :

[0056] S2, input the vehicle images in the vehicle re-identification dataset into the backbone network to obtain vehicle features.

[0057] Specifically, ResNet-50 pre-trained on ImageNet is used as the backbone network to build a benchmark framework for extracting image features of the vehicle re-identification dataset.

[0058] The backbone network allows vehicles with the same identity from different cameras to learn consistent feature representations. , the output features can be obtained through the backbone network .

[0059] For the backbone network parameters, use the loss function It is constrained by a combination of cross-entropy loss and soft-margin loss.

[0060] For features , we use a convolutional block ( ), which consists of a The convolutional layer ( ), a batch normalization layer ( ), an activation layer ( ) and a global average pooling layer ( ) to obtain a vector Used to calculate the loss function , It can be expressed as: .

[0061] The vehicle re-identification task is regarded as a classification task, and the cross entropy loss is used for constraint. Considering the imbalance of sample distribution, label-smooth technology is integrated to prevent the model from over-trusting positive samples. The vehicle re-identification task is regarded as a metric learning task, and the soft margin loss function is used to allow the distance between positive and negative samples to be maintained within a certain interval. The loss function It can be expressed as:

[0062]

[0063] in, represents the number of vehicle instances in the vehicle re-identification dataset, Indicates the identity ID corresponding to the current vehicle image. is a fully connected layer used to predict the body classification, and They represent the Euclidean distance between the reference sample and the positive sample and the Euclidean distance between the reference sample and the negative sample, respectively. is a small constraint constant in the label smoothing technique.

[0064] S3, inputs the vehicle features into the global attribute perception module based on the Transformer mechanism, obtains the global attribute features of the vehicle, and calculates the global attribute loss function based on the global attribute features.

[0065] Among them, a global attribute perception module (AAT) based on the Transformer mechanism is pre-built. First, the attribute information contained in the current training data set is clarified, and for each attribute, cross-attention is used as the basis of the Transformer mechanism.

[0066] In one embodiment, step S3 includes the following steps: S31, extracting attribute features from vehicle features through convolution blocks, and extracting attribute feature vectors corresponding to the attribute features through global average pooling operations.

[0067] Using different convolutional blocks ( ) From vehicle characteristics Get the Attributes , use the global average pooling operation (GAP) to obtain the attribute feature vector :

[0068]

[0069] S32, reshape the attribute features to obtain a spatial information sequence .

[0070] Define the sinusoidal position embedding sequence :

[0071] in, express The channel dimension index of express The spatial position index, express spatial dimension.

[0072] S33, after the attribute feature vector and the spatial information sequence are cross-attended in the global attribute perception module, they are input into the feedforward network to obtain the global attribute features of the vehicle.

[0073] The outputs Q, K, V of the cross-attention structure in the AAT module can be defined as:

[0074]

[0075]

[0076] After the cross-attention structure, the feed-forward network (FFN) is used to obtain the global attribute features of the vehicle. :

[0077] S34, optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on the vehicle attribute label information, and calculating the first edge loss function of the global attribute perception module based on the Transformer mechanism.

[0078] Specifically, the marginal loss function based on the global attributes of the vehicle is calculated , the feature distance between positive and negative samples of vehicle images is optimized based on the vehicle attribute label information.

[0079] Step S34 specifically includes: S341, unlike the soft-margin loss used in the backbone network, the margin loss is used in the AAT module to further reduce the distance between positive samples based on global attribute features and increase the difference between instances. It can be expressed as:

[0080]

[0081] in, Indicates the samples in a batch and The Euclidean distance between and Indicates the upper and lower boundary thresholds.

[0082] S342, first edge loss based on vehicle attributes in AAT module It can be expressed as:

[0083] S35, further, pre-constructing a post-processing module (Post-process Block) for separating the classification task feature space and the measurement task feature space. In the AAT module, the learning process is performed by express.

[0084] Pre-build a post-processing module (Post-process Block) to separate the feature space. When constraining vehicle attribute features, if the classification loss and metric loss constraints are applied to the same feature at the same time, it is easy to cause inconsistent gradient directions and cause fluctuations. Therefore, a post-processing module is added between the metric loss and the classification loss to separate the feature space; Among them, the Post-process Block contains a The convolutional layer ( ), a batch normalization layer ( ), an activation layer ( ) and a global average pooling layer ( ).use Indicates this embedding operation.

[0085] S36, calculating a first cross entropy loss function based on the vehicle attribute label and the global attribute features.

[0086] Calculate the first cross entropy loss function based on the vehicle's global attributes , constrain the AAT network parameters according to the vehicle attribute label information; Vehicle attribute features are processed through a post-processing module to calculate attribute-based classification losses. :

[0087] in, represents the number of vehicle attributes, Indicates attribute tags, represents the fully connected layer, Indicates the post-processing module in the AAT module.

[0088] S37, obtaining a global attribute loss function based on the first edge loss function and the first cross entropy loss function.

[0089] Calculate the global attribute loss function :

[0090] S4, inputting the vehicle features into the vehicle local area perception module based on the attention mechanism, obtaining the local area features of the vehicle, and calculating the local area loss function based on the local area features.

[0091] Among them, a vehicle local area perception module (L-PAM) based on the attention mechanism is pre-built. In one embodiment, step S4 includes the following steps: S41, extracting local area original features from the image mask information and vehicle features.

[0092] Utilize image mask information of key areas of the vehicle Output features from the backbone network Extract local area original features :

[0093] in, Represents an adaptive average pooling operation for alignment and Dimension.

[0094] S42, the importance features of different key areas in the original features of the local area are learned through the mask-based attention mechanism to obtain the key area features of the vehicle.

[0095] Step S42 specifically includes: S421, activate the masked average pooling operation The specific operation can be expressed as:

[0096] S422, learning the importance scores of different key areas based on the attention mechanism :

[0097]

[0098] in, Represents a series of network learning processes, Represents a learnable parameter.

[0099] S423, obtaining weighted vehicle key area features :

[0100] S43, separating and fusing the key area features of the vehicle through the area relationship interaction module to obtain the local area features of the vehicle.

[0101] Specifically, step S43 includes: S431, using mask From the weighted vehicle key area features Separate the features of each region :

[0102] in, Indicates flipping the mask elements.

[0103] S432, for Characteristics of the region , build on other regional characteristics The relationship between them is used to obtain the fused features :

[0104] S44, optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on the vehicle identity ID tag information, and calculating the second edge loss function of the vehicle local area perception module based on the attention mechanism.

[0105] Specifically, the second edge loss function based on the key local area of ​​the vehicle is calculated , the feature distance between positive and negative samples of vehicle images is optimized based on the vehicle identity ID tag information.

[0106] Second edge loss in L-PAM modules It can be expressed as:

[0107] Among them for In the calculation of, in the L-PAM module, The definition can be expressed as:

[0108] Furthermore, a post-processing block is used to separate the classification task feature space and the measurement task feature space. In the L-PAM module, the learning process is performed by express.

[0109] S45, calculating a second cross entropy loss function based on the vehicle identity ID tag information and the local area features.

[0110] Calculate the second cross entropy loss function based on the key local area of ​​the vehicle , constrain the L-PAM module network parameters based on the vehicle identity ID tag information.

[0111] The key local area features of the vehicle are processed by the post-processing module to calculate the second cross entropy loss based on the key local area of ​​the vehicle. :

[0112] in, represents the number of predefined vehicle critical areas, Represents the vehicle ID label in the dataset, represents the fully connected layer, Represents the learning process of the post-processing module in the L-PAM module.

[0113] S46, obtaining a local area loss function based on the second edge loss function and the second cross entropy loss function.

[0114] Calculate the local area loss function :

[0115] S5, inputting the vehicle features into a pyramid-shaped local fine-grained feature extraction module to obtain the local fine-grained features of the vehicle, and calculating the local fine-grained loss function based on the local fine-grained features.

[0116] Among them, a pyramid-shaped local fine-grained feature extraction module (FGFE) is pre-built. The FGFE module consists of a series of fine-grained feature extraction blocks (Fine-Grained Lock, FGB). The input of the FGB module can get two outputs through the upper and lower branches. Figure 4 The local fine-grained feature extraction flow chart provided in the embodiment of the present application, the subsequent steps can be referred to Figure 4 .

[0117] In one embodiment, step S5 comprises the following steps: S51, conduct three stages of learning respectively to obtain high-level features and average-level features in each stage. Step S51 specifically includes: S511, for FGB module, input The upper branch uses adaptive maximum pooling to extract high-level features as the input of the next layer FGB module , the process can be expressed as:

[0118] in, Indicates that the current FGB block is in the first position of the FGFE pyramid network. stage, Indicates that the current FGB block is in the first position of the FGFE pyramid network. layer.

[0119] S512, for FGB module, input , the lower branch first obtains the activation features along the channel direction , the process can be expressed as:

[0120] S513, for activation features , calculate the high level eigenvalue along the horizontal direction :

[0121] S514, will Middle The row height horizontal feature value is assigned a value of 0:

[0122] S515, After removing high-level features, we get , and an adaptive average pooling operation is used to extract the overall average level features :

[0123] S516, FGB block lower branch output Will be the first Part of the FGB block input to the subsequent stage of the layer.

[0124] S517, FGFE network structure is composed of a series of FGB blocks in a pyramid structure, including Each stage contains different levels , and The corresponding relationship can be described as:

[0125] S518, for stage The learning process can be described as:

[0126] in, Indicates The learning process of the stage Indicates The original input of each stage It can be expressed as:

[0127]

[0128]

[0129] S52, taking the high-level features and the average-level features as the local fine-grained features of the corresponding stage.

[0130] Specifically, the high-level features at the last layer of each stage are and average level characteristics As the local fine-grained features of this stage , used for subsequent fine-grained feature loss function calculation.

[0131] S53, optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on the vehicle identity ID tag information, and calculating the third edge loss function of the pyramid-shaped local fine-grained feature extraction module.

[0132] Specifically, the third edge loss function based on the local fine-grained features of the vehicle is calculated , the feature distance between positive and negative samples of vehicle images is optimized based on the vehicle identity ID tag information.

[0133] Third edge loss in FGFE module It can be expressed as:

[0134] Among them for In the calculation of, in the FGFE module, The definition can be expressed as:

[0135] Furthermore, a post-processing block is used to separate the classification task feature space and the measurement task feature space. In the FGFE module, the learning process is performed by express.

[0136] S54, calculating a third cross entropy loss function based on the vehicle identity ID tag information and the local fine-grained features.

[0137] Specifically, calculate the third cross entropy loss function , constrain the FGFE module network parameters based on the vehicle identity ID tag information.

[0138] The local fine-grained features of the vehicle are processed by the post-processing module to calculate the third cross entropy loss. :

[0139] in, represents the number of predefined vehicle critical areas, represents the vehicle identity in the dataset, Represents the learning process of the post-processing module in the FGFE module.

[0140] S55, obtaining a local fine-grained loss function based on the third edge loss function and the third cross entropy loss function.

[0141] Calculate local fine-grained loss function :

[0142] S6, training the backbone network based on the global attribute loss function, the local area loss function and the local fine-grained loss function to obtain a trained backbone network.

[0143] The multi-scale loss function is calculated based on the global attribute loss function, the local area loss function and the local fine-grained loss function. Consistency constraints are imposed on the model parameters of the backbone network:

[0144] in, is a hyperparameter used to balance the importance of the AAT module for the entire model to learn the vehicle identity feature representation. In this invention, based on the experimental results, we set This is because although the vehicle attribute information can well cope with the changes in vehicle perspective and solve the intra-class difference challenge, it also causes the inter-class similarity challenge because different vehicles may have similar attributes.

[0145] In one embodiment, the vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning further includes the following steps: A1, before backbone network training, data augmentation strategies are performed on the vehicle re-identification dataset, including boundary padding, random erasing, and random flipping strategies.

[0146] A2, learning rate during backbone network training Adjustment strategy can be expressed as follows:

[0147] in, is the maximum learning rate, is the minimum learning rate, Indicates the number of iterations of the current training. Indicates the maximum number of iterations for model training.

[0148] Figure 5 The comparison chart of mAP, CMC@1 and CMC@5 indicators on the VeRi-776 dataset provided in the embodiment of this application is as follows: Figure 6 A comparison chart of CMC@1 and CMC@5 indicators on the VehicleID dataset provided in the embodiment of this application, Figure 7 The comparison chart of mAP, CMC@1 and CMC@5 indicators on the VeRi-Wild dataset provided in the embodiment of this application is shown in Figure 2. Figure 5-Figure 7It can be seen that: (1) the performance indicators of the present invention on the three data sets are all better than all other comparison models, which proves the advancement of the present invention; (2) on the three data sets, due to the differences in the data set conditions, the experimental performance on each data set is different, but all can exceed the most advanced methods on the current data set, which shows that the present invention has strong domain adaptation capabilities in different environments and different situations; (3) among all the comparison methods, the methods that use the combination of global and local have achieved good results, because these methods comprehensively consider the overall macro information of the vehicle and the local subtle difference information. However, compared with the methods used in the present invention, these methods only use the vehicle's identity ID information to constrain the learning of global features and local features, and it is difficult to extract effective vehicle identity-related information for vehicle re-identification; (4) among all the comparison models, the vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning has achieved the best results. In the present invention, the learning of the global features of the vehicle is constrained by the vehicle's attributes, and the learning of local features is divided into the learning of vehicle key area features and the learning of local fine-grained features. These features contain rich vehicle identity-related information, so they can achieve excellent performance in vehicle re-identification.

[0149] Furthermore, ablation experiments are performed to demonstrate the effectiveness of each module. Figure 8 A schematic diagram of the results of the ablation experiments of each submodule provided in the embodiment of the present application, Fig. 9 A schematic diagram of the results of the L-PAM module neutron module ablation experiment provided in the embodiment of the present application, Figure 8 The experimental results of training different modules on the VeRi-776 dataset are shown. A multi-branch network structure is adopted in the present invention. It can be seen from the experimental results that the combination effect of different modules is better than using a certain module alone, which shows that different modules can make up for each other's shortcomings. At the same time, when the AAT module, L-PAM module and FGFE module are combined, the model achieves the best performance of 85.2% and 97.8% on mAP and CMC@1 respectively. Fig. 9 The ablation experimental results of the sub-modules MAM and PIM in the L-PAM module are shown. It can be seen that when the MAM module or the PIM module is not used in the L-PAM module, the experimental results are almost the same as the results of not using the entire L-PAM module, which shows that there is a strong correlation between the MAM module and the PIM module and they are an indispensable part of the L-PAM module.

[0150] Fig.10 The vehicle re-identification performance index provided in the embodiment of the present application is affected by the boundary loss function parameter and , and AAT module importance parameters Schematic diagram of the impact, Fig.11A schematic diagram showing how the vehicle re-identification performance index provided in an embodiment of the present application is affected by different loss functions. Fig.10 Shows the three hyperparameters of the model , and ,in To balance the importance of the AAT module for the whole model to learn the vehicle identity feature representation, in this invention, according to the experimental results, we set This is because although the vehicle attribute information can well cope with the changes in vehicle perspective and solve the intra-class difference challenge, it also causes the inter-class similarity challenge because different vehicles may have similar attributes; and Experimentally, they are set to 1 and 0.3 respectively to constrain the distance between positive and negative samples.

[0151] S7, inputting the vehicle re-identification data set into the trained backbone network to obtain multiple vehicle image features, calculating the similarity of the multiple vehicle image features, and arranging them according to the similarity to obtain the vehicle re-identification result.

[0152] Specifically, the trained backbone network is used to extract the features of the vehicle image to be retrieved (query) in the vehicle re-identification dataset, and the trained backbone network is used to extract the features of the vehicle image in the image gallery (gallery) in the vehicle re-identification dataset. The Euclidean distance is used to calculate the similarity between the features of the vehicle image to be retrieved and the features of the vehicle image in the image gallery. The gallery images are arranged according to the similarity to obtain the vehicle re-identification results.

[0153] This application fully extracts vehicle features by setting up a global attribute perception module, a vehicle local area perception module and a fine-grained feature extraction module, and trains the backbone module through these three modules, which can achieve the following effects: (1) Strong domain adaptation capability: The invention can demonstrate excellent re-identification results on datasets in different domains, such as datasets with drastic changes in viewing angles, low image pixels, uneven vehicle attributes, and day and night cross-domains.

[0154] (2) Capture of coarse-grained and fine-grained information: This invention makes full use of the vehicle's global attribute information, key local information, and local subtle distinguishing features contained in the vehicle image, from coarse to fine, to achieve accurate representation of the vehicle's identity features.

[0155] (3) High vehicle re-identification accuracy: This invention realizes the coordinated representation of vehicle identity by accurately extracting the global attribute information and local discrimination information of the vehicle, effectively coping with the challenges of inter-class similarity and intra-class difference in vehicle re-identification, and greatly improving the accuracy of vehicle re-identification.

[0156] According to the method described in the above embodiment, this embodiment will be further described from the perspective of a vehicle re-identification device based on vehicle global attribute perception and local discriminant feature learning. The vehicle re-identification device based on vehicle global attribute perception and local discriminant feature learning can be implemented as an independent entity or integrated in a terminal, server and other devices, wherein the terminal may include a tablet computer, a laptop computer, a personal computer (PC, Personal Computer), a micro processing box, or other devices.

[0157] See also Fig.12 , Fig.12 The vehicle re-identification device based on vehicle global attribute perception and local discriminant feature learning provided by the embodiment of the present application is specifically described, and is applied to an electronic device. The vehicle re-identification device based on vehicle global attribute perception and local discriminant feature learning may include: An acquisition module is used to acquire a vehicle re-identification dataset; A first recognition module, used for inputting vehicle images in the vehicle re-identification dataset into a backbone network to obtain vehicle features; A calculation module, used for inputting the vehicle features into a global attribute perception module based on a Transformer mechanism to obtain global attribute features of the vehicle, and calculating a global attribute loss function based on the global attribute features; inputting the vehicle features into a vehicle local area perception module based on an attention mechanism to obtain local area features of the vehicle, and calculating a local area loss function based on the local area features; and inputting the vehicle features into a pyramid-shaped local fine-grained feature extraction module to obtain local fine-grained features of the vehicle, and calculating a local fine-grained loss function based on the local fine-grained features; A training module, used for training the backbone network based on the global attribute loss function, the local area loss function and the local fine-grained loss function to obtain a trained backbone network; The second recognition module is used to input the vehicle re-identification data set into the trained backbone network to obtain multiple vehicle image features, calculate the similarity of the multiple vehicle image features, and arrange them according to the similarity to obtain the vehicle re-identification result.

[0158] During specific implementation, the above modules and / or units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above modules and / or units can refer to the previous method embodiments. The specific beneficial effects that can be achieved can also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0159] The above is a detailed introduction to a vehicle re-identification method and device based on vehicle global attribute perception and local discriminant feature learning provided in an embodiment of the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning, characterized in that: The method comprises: Get the vehicle re-identification dataset; Inputting the vehicle re-identification dataset into a backbone network to obtain vehicle features; Inputting the vehicle features into a global attribute perception module based on a Transformer mechanism to obtain global attribute features of the vehicle, and calculating a global attribute loss function based on the global attribute features; Inputting the vehicle features into a vehicle local area perception module based on an attention mechanism to obtain local area features of the vehicle, and calculating a local area loss function based on the local area features; Inputting the vehicle features into a pyramid-shaped local fine-grained feature extraction module to obtain local fine-grained features of the vehicle, and calculating a local fine-grained loss function based on the local fine-grained features; Training the backbone network based on the global attribute loss function, the local area loss function and the local fine-grained loss function to obtain a trained backbone network; The vehicle re-identification data set is input into the trained backbone network to obtain multiple vehicle image features, the similarity of the multiple vehicle image features is calculated, and they are arranged according to the similarity to obtain a vehicle re-identification result.

2. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 1 is characterized in that: The step of inputting the vehicle features into a global attribute perception module based on a Transformer mechanism to obtain global attribute features of the vehicle includes: Extracting attribute features from the vehicle features through a convolution block, and extracting attribute feature vectors corresponding to the attribute features through a global average pooling operation; Reshaping the dimension of the attribute features to obtain a spatial information sequence; The attribute feature vector and the spatial information sequence are input into a feedforward network after being cross-attended in the global attribute perception module to obtain the global attribute features of the vehicle.

3. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 2 is characterized in that: The calculating the global attribute loss function based on the global attribute feature comprises: Optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on the vehicle attribute label information, and calculating the first edge loss function of the global attribute perception module based on the Transformer mechanism; Calculate a first cross entropy loss function based on the vehicle attribute label and the global attribute feature; A global attribute loss function is obtained based on the first edge loss function and the first cross entropy loss function.

4. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 1 is characterized in that: The step of inputting the vehicle features into a vehicle local area perception module based on an attention mechanism to obtain the vehicle local area features includes: Extracting local area original features from the image mask information and the vehicle features; The importance features of different key areas in the original features of the local area are learned through a mask-based attention mechanism to obtain the key area features of the vehicle; The key area features of the vehicle are separated and fused through the area relationship interaction module to obtain the local area features of the vehicle.

5. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 4 is characterized in that: The calculating the local area loss function based on the local area feature includes: Optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on vehicle identity ID tag information, and calculating the second edge loss function of the vehicle local area perception module based on the attention mechanism; Calculate a second cross entropy loss function based on the vehicle identity ID tag information and the local area features; A local area loss function is obtained based on the second edge loss function and the second cross entropy loss function.

6. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 5 is characterized in that: The pyramid-shaped local fine-grained feature extraction module includes three stages. The vehicle feature is input into the pyramid-shaped local fine-grained feature extraction module to obtain the local fine-grained features of the vehicle, including: Carry out three stages of learning respectively to obtain high-level features and average-level features in each stage; The high-level features and the average level features are used as local fine-grained features of the corresponding stage.

7. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 6 is characterized in that: The calculating the local fine-grained loss function based on the local fine-grained features includes: Optimizing the distance features between positive and negative samples of vehicle images in the vehicle re-identification dataset based on vehicle identity ID tag information, and calculating the third edge loss function of the pyramid-shaped local fine-grained feature extraction module; Calculating a third cross entropy loss function based on the vehicle identity ID tag information and the local fine-grained features; A local fine-grained loss function is obtained based on the third edge loss function and the third cross entropy loss function.

8. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 1 is characterized in that: The method further comprises: Before training the backbone network, performing a data enhancement strategy on the vehicle re-identification dataset; During the training of the backbone network, the learning rate Adjustment strategy can be expressed as follows: in, is the maximum learning rate, is the minimum learning rate, Indicates the number of iterations of the current training. Indicates the maximum number of iterations for model training.

9. The vehicle re-identification method based on vehicle global attribute perception and local discriminant feature learning according to claim 4 is characterized in that: The method further comprises: acquiring the image mask information; The acquiring the image mask information comprises: Inputting the vehicle re-identification data set into a vehicle key part detector to generate a boundary detection box; Obtaining the coordinates of key parts of the vehicle based on the boundary detection frame; A vehicle component enclosing area is obtained based on the vehicle component position, and the image mask information is defined by the range of the vehicle component enclosing area.

10. A vehicle re-identification device based on vehicle global attribute perception and local discriminant feature learning, characterized in that: include: An acquisition module is used to acquire a vehicle re-identification dataset; A first recognition module, used for inputting vehicle images in the vehicle re-identification dataset into a backbone network to obtain vehicle features; A calculation module, used for inputting the vehicle features into a global attribute perception module based on a Transformer mechanism to obtain global attribute features of the vehicle, and calculating a global attribute loss function based on the global attribute features; inputting the vehicle features into a vehicle local area perception module based on an attention mechanism to obtain local area features of the vehicle, and calculating a local area loss function based on the local area features; and inputting the vehicle features into a pyramid-shaped local fine-grained feature extraction module to obtain local fine-grained features of the vehicle, and calculating a local fine-grained loss function based on the local fine-grained features; A training module, used for training the backbone network based on the global attribute loss function, the local area loss function and the local fine-grained loss function to obtain a trained backbone network; The second recognition module is used to input the vehicle re-identification data set into the trained backbone network to obtain multiple vehicle image features, calculate the similarity of the multiple vehicle image features, and arrange them according to the similarity to obtain the vehicle re-identification result.

Images