A vehicle re-identification method based on feature similarity measurement and deep learning

Abstract

This invention provides a vehicle re-identification method based on feature similarity measurement and deep learning, comprising two stages: the first stage is semi-supervised cross-domain joint pre-training, and the second stage is fine-tuning of the pre-trained model in the target domain. This invention proposes a novel feature similarity measurement method—Overlapping Feature Reordering (OFR)—and designs a new unsupervised vehicle re-identification framework. This framework employs an effective semi-supervised domain training method called Semi-Supervised Cross-Domain Joint Learning (JCLS). OFR cross-divides the extracted features to obtain feature partitions and sums the measurement results of all feature partitions as the basis for more accurate clustering. In JCLS, the previous single-domain training is abandoned; instead, samples with target domain style are introduced to jointly train the Re-ID model. This method can extract more representative features to participate in similarity measurement, improving the recognition rate.

Description

Technical Field

[0001] This invention relates to the field of vehicle re-identification technology, specifically a vehicle re-identification method based on feature similarity measurement and deep learning. Background Technology

[0002] Vehicle re-identification aims to identify target vehicles in multi-camera networks. It plays a crucial role in many practical applications, such as urban traffic surveillance and intelligent transportation. With the rapid development of deep learning in computer vision, vehicle re-identification tasks using CNNs (Convolutional Neural Networks) have become a research hotspot and have attracted widespread attention from researchers.

[0003] Feature similarity metrics can be widely applied in content-based unsupervised vehicle re-identification research. Generally, vehicle images are input into a pre-trained CNN model to extract color, shape, or texture features. The re-identification model uses the obtained feature vectors to measure the similarity between different vehicle samples, clusters unlabeled images based on the similarity metric results, and assigns pseudo-labels. The re-identification model iteratively fine-tunes using pseudo-labeled samples until it converges. Therefore, finding a more reasonable vehicle Re-ID similarity metric is of great significance. Currently, the most direct method for measuring the similarity between two features is to use a distance metric to calculate their distance, which is also one of the most popular methods. Euclidean distance and Jacquard distance are commonly used distance metrics to measure the similarity between vehicle features. However, these methods inevitably produce pseudo-label noise, and the model is at high risk of collapse when the initial pseudo-labels are noisy. Part of the reason for this problem is that these vehicle feature similarity measurement methods mainly focus on the distance measurement of features, while neglecting the processing of extracted features.

[0004] Traditional vehicle Re-ID frameworks directly apply pre-trained models from the source domain to the target domain for feature extraction. Due to domain biases between different datasets, these traditional frameworks make the model overly sensitive to differences in the target domain, affecting its feature extraction capabilities and misleading feature similarity measurement results. Summary of the Invention

[0005] Existing unsupervised vehicle feature similarity measurement methods ignore pseudo-label noise during training, and due to significant domain discrepancies in feature measurement, the model cannot extract accurate target sample features. To address this issue, this invention proposes a vehicle re-identification method based on feature similarity measurement and deep learning. It introduces a novel feature similarity measurement method—Overlapping Feature Reordering (OFR)—and designs a new unsupervised vehicle re-identification framework. This framework employs an efficient semi-supervised domain training method called Semi-Supervised Cross-Domain Joint Learning (JCLS). OFR cross-divides the extracted features to obtain feature partitions and sums the measurement results of all feature partitions as the basis for more accurate clustering. In JCLS, instead of the previous single-domain training, samples with target domain style are introduced to jointly train the Re-ID model. This method can extract more representative features for similarity measurement, improving the recognition rate.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a vehicle re-identification method based on feature similarity measurement and deep learning, the method comprising two stages: the first stage is cross-domain joint pre-training based on semi-supervised learning, and the second stage is fine-tuning of the pre-trained model in the target domain;

[0007] Step A: In the first stage, the model is pre-trained in a semi-supervised manner; the generated image is combined with the source domain image and fed into the pre-trained network; in this process, a pseudo-label reassignment strategy based on dual domain style fusion is proposed to study the target domain style;

[0008] Step B: In the second stage, a more reasonable feature similarity measurement method was designed to measure the extracted target sample features; the extracted target features were reasonably divided to obtain more feature partitions, and then distance measurements were performed on these feature partitions to obtain similarity results.

[0009] Furthermore, step A specifically includes:

[0010] ① Cross-domain data augmentation during pre-training

[0011] At the same time, the already labeled source domain data and unlabeled samples Input into the CNN model to be trained; and Let G(S) represent the i-th sample and its corresponding label; where the dataset G(S) is generated by a style transfer network.

[0012] To make full use of the labeled source domain data S and the newly generated dataset G(S), a semi-supervised learning approach is used to pre-train the Re-ID model; a suitable learner is trained using the labeled dataset S, and pseudo-labels are assigned to the unlabeled samples G(S); throughout the pre-training process, the images generated by the style transfer network are treated as unlabeled data.

[0013] Design a virtual label for the newly generated data sample, giving it information about both the source and target domains;

[0014] ② A pseudo-label redistribution strategy based on dual-domain style fusion

[0015] We select to assign certain weights to the target domain sample types and make the labels exist in the form of soft labels; that is, the new samples generated by the style transfer network will discard the original IDs and participate in semi-supervised pre-training as an unlabeled set G(S) and obtain the soft labels assigned by the system; at the same time, we select to allocate a small portion of the weights to the label types in the target domain; and we treat each target domain image as a separate class for weight distribution, which allows the Re-ID model to capture obvious vehicle representations; for each unknown image, its weight distribution is as shown in Equation (1).

[0016]

[0017] Where i represents the i-th unknown sample, δ represents the weight assigned to the original ID of the image, t represents the category of the target domain in which the weight is being calculated, and l represents the number of sample categories in the target dataset. The original label is denoted as , belonging to the i-th image; for any G(S) sample, the loss function for the corresponding pseudo-label redistribution process is given by equation (2).

[0018]

[0019] Where p(t) represents the probability that the image belongs to the t-th class of the target dataset.

[0020] The pseudo-label redistribution strategy is applied to the cross-domain data augmentation strategy in ①; the new sample abandons the hard label, which hinders the target domain style of the model, and is used as the unlabeled dataset G(S); G(S) and S are placed in the pre-trained network and trained in a semi-supervised manner; the unlabeled sample is given a pseudo-soft label with a pseudo-label redistribution strategy and is used to optimize the pre-trained model with the source sample G(S); based on the above analysis, the overall loss function in the training phase is expressed as formula (3):

[0021] L Pre =L EC +L PLR (3)

[0022] L EC The cross-entropy loss function is represented; during the entire pre-training phase, the training model is optimized using two types of loss functions; the overall loss function is specifically expressed as (4):

[0023]

[0024] Where u is the sample ID; u = 1 indicates that sample u is a real image, and vice versa. p(u) represents the probability that the image belongs to the t-th class of the target dataset.

[0025] Furthermore, step B specifically includes:

[0026] A method for reordering overlapping features based on distance similarity is designed. This represents the features extracted from the target dataset T via a CNN; given each image i, ( (representing the features extracted from the i-th image in dataset T) is divided into N partition features (ie, f) i 1 f i 2 …f i P ), calculate f i c The distance similarity between (the features of the c-th partition) and the corresponding partition of each image;

[0027] This method does not divide the features into completely independent partitions, but instead divides the features into N parts; that is, there are overlapping areas between adjacent partitions; through the above method, the Re-ID model can still retain the relationship between features to a greater extent when performing feature division, and discover more similarities between features; the similarity vectors corresponding to each partition are shown in Equation (5);

[0028]

[0029] Where s(f) i d f j d (d = 1, 2, ..., N) (j = 1, 2, ..., l) is represented as f i d (Features of the d-th partition of the i-th image) and f j d The calculated similarity value (features of the d-th partition of the j-th image), Let represent the similarity vector between the segment d from the i-th image and the segment d of other sample features; finally, the feature similarity of each part is fused to obtain the final feature similarity; at the same time, in order to ensure the accuracy of the pseudo-label, the similarity calculated by the unsegmented features is added to the final feature similarity; the total similarity vector is shown in Equation (6);

[0030] S total =S ori +S1+S2+…+S l (6)

[0031] S i S represents the image similarity result of fusing the similarity of P feature partitions; ori This represents the similarity calculated based on the unsegmented features of the i-th image; during the fine-tuning stage, the network is optimized using the commonly used cross-entropy loss function; therefore, the total loss function of the proposed framework is as shown in equation (7);

[0032] L = L Train +L Pre (7)

[0033] L Train This represents losses incurred during formal training.

[0034] Compared with the prior art, the beneficial effects of the present invention are:

[0035] 1. This invention proposes a vehicle Re-ID feature similarity measurement method—Overlapping Feature Reordering (OFR). OFR uses reasonable cross-partitioning to divide the obtained features into several partitions. The final distance measurement result combines the original distance similarity and the partition distance similarity. This reduces false label noise and improves measurement accuracy during the measurement process.

[0036] 2. This invention proposes a novel vehicle pre-training method—Semi-supervised Joint Cross-Domain Learning (JCLS). This method can reduce the negative impact of domain bias on similarity measurement results. In this module, generated samples and source samples are pre-trained together. This approach reduces the sensitivity of the model to differences between target data before training. Simultaneously, a pseudo-label reassignment strategy is designed to gain a more comprehensive understanding of the feature distribution of target domain samples. Attached Figure Description

[0037] Figure 1 Clustering results from different theories;

[0038] Figure 2 Theoretical differences between different vehicle datasets;

[0039] Figure 3 This is an overall framework diagram of the present invention;

[0040] Figure 4 This is a schematic diagram of a pseudo-label reassignment strategy;

[0041] Figure 5 A schematic diagram of reordering overlapping features. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. The specific embodiments described herein are only for explaining the technical solutions of this invention and are not intended to limit the invention.

[0043] Feature similarity metrics can be widely applied in content-based unsupervised vehicle re-identification research. Generally, vehicle images are input into a pre-trained CNN model to extract color, shape, or texture features. The re-identification model uses the obtained feature vectors to measure the similarity between different vehicle samples, clusters unlabeled images based on the similarity metrics, and assigns pseudo-labels. The re-identification model iteratively fine-tunes using pseudo-labeled samples until it converges. Therefore, finding a more reasonable vehicle Re-ID similarity metric is of great significance. Currently, the most direct method for measuring the similarity between two features is to use a distance metric to calculate their distance, which is also one of the most popular methods. Euclidean distance and Jacquard distance are commonly used distance metrics to measure the similarity between vehicle features. However, these methods inevitably produce pseudo-label noise, and the model is at high risk of collapse when the initial pseudo-labels are noisy. Part of the reason for this problem is that these vehicle feature similarity measurement methods mainly focus on the distance measurement of features, while neglecting the processing of extracted features. Therefore, we divide the entire feature measurement process into two stages and add a feature preprocessing process before feature distance measurement. This process obtains more accurate measurement results by reasonably dividing the features through cross-references, such as... Figure 1 As shown.

[0044] Furthermore, to reduce the interference of upstream tasks on feature similarity measurement results and improve the feature extraction capability of the Re-ID model in the target domain, we improved the traditional unsupervised vehicle Re-ID framework. The traditional vehicle Re-ID framework directly applies a pre-trained model from the source domain to the target domain for feature extraction. However, due to domain biases between different datasets, such as… Figure 2 As shown, traditional frameworks make the model overly sensitive to differences in the target domain, affecting its feature extraction ability in the target domain and misleading feature similarity measurement results. Therefore, we propose a novel pre-training paradigm. This method improves the pre-trained model's feature capture ability and generalization ability on the target dataset by simultaneously processing source domain data and generated data with target domain style.

[0045] In traditional unsupervised vehicle re-identification algorithms, a re-identification model pre-trained in the source domain is used to extract sample features from the target domain. This model then clusters samples based on similarity measurements of the extracted target features and assigns pseudo-labels to unlabeled samples. These pseudo-labeled target samples are subsequently fed into the model for iterative fine-tuning. During fine-tuning, pseudo-label noise is inevitably generated. Furthermore, due to domain bias, the pre-trained model experiences a significant performance degradation when applied to target samples.

[0046] The overall framework proposed in this invention is as follows: Figure 3 As shown, it aims to solve the two problems mentioned above. The framework mainly consists of two stages: the first stage is semi-supervised cross-domain joint pre-training, and the second stage is fine-tuning of the pre-trained model on the target domain. In the first stage, the model is pre-trained using a semi-supervised approach. The generated image is combined with the source domain image and fed into the pre-trained network. In this process, we propose a pseudo-label reassignment strategy to better study the target domain style. Subsequently, in the fine-tuning module, a more reasonable feature similarity measurement method is designed to measure the extracted target sample features, including two steps. First, the extracted target features are reasonably divided to obtain more feature partitions, and then distance measurements are performed on these feature partitions separately to obtain similarity results.

[0047] The first stage specifically includes:

[0048] ① Cross-domain data augmentation during pre-training

[0049] To mitigate the impact of domain dissimilarity on the accuracy of the vehicle re-identification model, target domain data is augmented through cross-domain data enhancement during the pre-training process. We also incorporate labeled source domain data... and unlabeled samples The input is fed into the CNN model to be trained. and Let G(S) represent the i-th sample and its corresponding label. The dataset G(S) is generated by a style transfer network and has a strong target domain style.

[0050] To fully utilize the labeled source domain data S and the newly generated dataset G(S), we employ a semi-supervised learning approach to pre-train the Re-ID model. A suitable learner is trained using the labeled dataset S, and pseudo-labels are assigned to the unlabeled samples G(S). Throughout the pre-training process, images generated by the style transfer network are treated as unlabeled data. Although the generated samples retain the previous ID information, the samples G(S) generated by the style transfer network have a target domain style. Traditional hard labels are no longer sufficient for the Re-ID model to learn the data distribution of generated samples. Our proposed data augmentation method does not adequately address the regional disparity problem as envisioned.

[0051] Therefore, we need to design a virtual label for the newly generated data sample so that it can have information about the source and target domains.

[0052] ② A pseudo-label redistribution strategy based on dual-domain style fusion

[0053] To fully utilize the annotation information of labeled samples and gain a preliminary understanding of the geometric feature distribution of the target domain data, we propose a pseudo-label reassignment strategy that combines target style learning. The weight distribution strategy is as follows: Figure 4 As shown.

[0054] This method abandons the traditional hard labels, instead assigning weights to the target domain sample types and allowing the labels to exist in the form of soft labels. That is, the new samples generated by the style transfer network will discard their original IDs, becoming an unlabeled set G(S) for semi-supervised pre-training and obtaining soft labels assigned by the system. Simultaneously, we choose to allocate a small portion of the weights to the label types in the target domain. Furthermore, we treat each target domain image as a separate class for weight distribution, which allows the Re-ID model to capture obvious vehicle representations. For each unknown image, its weight distribution is shown in Equation (1).

[0055]

[0056] Where i represents the i-th unknown sample, δ represents the weight assigned to the original ID of the image, and t represents the category of the target domain for which the weight is being calculated. Let be the original label, belonging to the i-th image, and l represent the number of sample categories in the target dataset. For any G(S) sample, the corresponding loss function for the pseudo-label reassignment process (i.e., the pseudo-label reassignment loss function) is as shown in equation (2).

[0057]

[0058] Where p(t) represents the probability that the image belongs to the t-th class of the target dataset.

[0059] We apply the pseudo-label redistribution strategy to the cross-domain data augmentation strategy described in ①. The new sample discards the hard label, which hinders the model's target domain style, and is used as the unlabeled dataset G(S). G(S) and S are placed in the pre-trained network and trained in a semi-supervised manner. The unlabeled sample is given a pseudo-soft label with the pseudo-label redistribution strategy and is used to optimize the pre-trained model with the source sample G(S). Based on the above analysis, the overall loss function in the training phase is expressed as Equation (3):

[0060] L Pre =L EC +L PLR (3)

[0061] L EC Let represent the cross-entropy loss function. During the entire pre-training phase, two types of loss functions are used to optimize the trained model. The overall loss function is specifically expressed as (4):

[0062]

[0063] Where u is the sample ID. u = 1 indicates that sample u is a real image, and vice versa. p(u) represents the probability that the image belongs to the t-th class of the target dataset.

[0064] By introducing target domain style data G(S), the model's sensitivity to the target dataset is reduced, while its generalization ability to different datasets is improved.

[0065] The second phase specifically includes:

[0066] Most recent unsupervised domain adaptations for vehicles involve pre-training a CNN model using labeled samples with rich annotation information. The pre-trained model then extracts features from unknown target samples, calculates image similarity using these features, and pseudo-labels the images. Subsequently, the pseudo-labeled data is iteratively fed into the training model for fine-tuning until optimal performance is achieved.

[0067] In UDA tasks, Re-ID models can fully utilize the original feature extraction capabilities of preprocessing networks to identify image classes in new datasets. Fine-tuning methods effectively improve the performance of UDA methods, but inevitably introduce pseudo-label noise. To reduce pseudo-label noise, we propose an overlapping feature re-ranking algorithm based on feature partitioning. The aim is to correct the performance and accuracy of the trained model in the target domain.

[0068] Existing unsupervised vehicle Re-ID methods based on feature similarity metrics inevitably introduce pseudo-label noise during the fine-tuning stage. To address this issue, we design an overlapping feature re-ranking method that calculates distance similarity. This aims to improve the confidence of predicted pseudo-labels and reduce the misleading nature of model classification results. First, This represents the features extracted from the target dataset T using a CNN. Given each image i, ... ( (representing the features extracted from the i-th image in dataset T) is divided into N partition features (ie, f) i 1 f i 2 …f i P ), calculate f i c The distance similarity between the features of the c-th partition and the corresponding partition of each image. The process of this method is as follows: Figure 5 As shown.

[0069] Compared to traditional feature segmentation methods, this method does not divide features into completely independent partitions, but rather cross-segments features into N parts. That is, there are overlapping regions between adjacent partitions. Through the above method, we hope that the Re-ID model can still preserve the relationship between features to a greater extent and explore more similarities between features when performing feature segmentation. The similarity vectors corresponding to each partition are shown in Equation (5).

[0070]

[0071] Where s(f) i d f j d (d = 1, 2, ..., N) (j = 1, 2, ..., l) is represented as f i d (Features of the d-th partition of the i-th image) and f j d The similarity value calculated from the features of the d-th partition of the j-th image. Let represent the similarity vector between the segment d from the i-th image and the segment d of other sample features. Finally, the feature similarities of each part are fused to obtain the final feature similarity. At the same time, in order to ensure the accuracy of the pseudo-label, we add the similarity calculated from the unsegmented features to the final feature similarity. The total similarity vector is shown in Equation (6).

[0072] S total =S ori +S1+S2+…+S l (6)

[0073] S i S represents the image similarity result of fusing the similarity of P feature partitions. ori This represents the similarity calculated based on the unsegmented features of the i-th image. During the fine-tuning phase, we optimize the network using the commonly used cross-entropy loss function. Therefore, the total loss function of the proposed framework is shown in Equation (7). Driven by the total loss function, the proposed model performs well in addressing domain differences in different datasets and reducing noise carried by images.

[0074] L = L Train +L Pre (7)

[0075] L rrain This represents losses incurred during formal training.

[0076] The above description merely illustrates preferred embodiments of the present invention, and while the description is relatively specific and detailed, it should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications, improvements, and substitutions without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A vehicle re-identification method based on feature similarity measurement and deep learning, characterized in that: The method consists of two stages: the first stage is semi-supervised cross-domain joint pre-training, and the second stage is fine-tuning of the pre-trained model in the target domain. Step A: In the first stage, the model is pre-trained in a semi-supervised manner; the generated image is combined with the source domain image and fed into the pre-trained network; In this process, a pseudo-label redistribution strategy based on dual-domain style fusion is proposed to study target domain style; including: ① Cross-domain data augmentation during pre-training At the same time, the already labeled source domain data and unlabeled samples Input into the CNN model to be trained; and Indicates the first Each sample and its corresponding label; where the dataset... Generated by a style transfer network; To make full use of the labeled source domain data and the newly generated dataset The Re-ID model is pre-trained using a semi-supervised learning approach with the sample data; a suitable learner is trained using the labeled dataset S, and pseudo-labels are assigned to the unlabeled samples G(S); throughout the pre-training process, the images generated by the style transfer network are treated as unlabeled data. Design a virtual label for the newly generated data sample, giving it information about both the source and target domains; ② A pseudo-label redistribution strategy based on dual-domain style fusion We select to assign certain weights to the target domain sample types and make the labels exist in the form of soft labels; that is, the new samples generated by the style transfer network will discard the original IDs and participate in semi-supervised pre-training as an unlabeled set G(S) and obtain the soft labels assigned by the system; at the same time, we select to allocate a small portion of the weights to the label types in the target domain; and we treat each target domain image as a separate class for weight distribution, which allows the Re-ID model to capture obvious vehicle representations; for each unknown image, its weight distribution is as shown in Equation (1). ； in, Indicates the first An unknown sample, This represents the weight assigned to the original ID of the image, and t represents the category of the target domain from which the weight is being calculated. This is the original tag, belonging to the first... One image; This represents the number of sample classes in the target dataset; for any G(S) sample, the loss function for the corresponding pseudo-label redistribution process is shown in equation (2). ； in This indicates that the image belongs to the first element in the target dataset. The probability of class samples; The pseudo-label redistribution strategy is applied to the cross-domain data augmentation strategy in ①; the new sample abandons the hard label, which hinders the target domain style of the model, and is used as the unlabeled dataset G(S); G(S) and S are placed in the pre-trained network and trained in a semi-supervised manner; the unlabeled sample is given a pseudo-soft label with a pseudo-label redistribution strategy and is used to optimize the pre-trained model with the source sample G(S); based on the above analysis, the overall loss function in the training phase is expressed as formula (3): (3) The cross-entropy loss function is represented; during the entire pre-training phase, the training model is optimized using two types of loss functions; the overall loss function is specifically expressed as (4): ； in For sample ID; Indicates sample It is a real image, and vice versa; This indicates that the image belongs to the first element in the target dataset. The probability of class samples; Step B: In the second stage, a more reasonable feature similarity measurement method was designed to measure the extracted target sample features; the extracted target features were reasonably divided to obtain more feature partitions, and then distance measurements were performed on these feature partitions to obtain similarity results.

2. The vehicle re-identification method based on feature similarity measurement and deep learning according to claim 1, characterized in that: Step B specifically includes: A method for reordering overlapping features based on distance similarity is designed. This represents the features extracted from the target dataset T using a CNN; given each image i , ,Will Divided into N Partial partition features (ie, ),calculate The distance similarity between (the features of the c-th partition) and the corresponding partition of each image; This method does not divide the features into completely independent partitions, but instead divides the features into N parts; that is, there are overlapping areas between adjacent partitions; through the above method, the Re-ID model can still retain the relationship between features to a greater extent when performing feature division, and discover more similarities between features; the similarity vectors corresponding to each partition are shown in Equation (5); (5); in (d=1,2 N) ( =1,2, ) represents and The calculated similarity value, Indicates the first The first picture Characteristics of each partition , Indicates from the first Dividing in a 100-page image Division with other sample features The similarity vector between them; finally, the feature similarity of each part is fused to obtain the final feature similarity; at the same time, in order to ensure the accuracy of the pseudo-label, the similarity calculated by the unsegmented features is added to the final feature similarity; the total similarity vector is shown in Equation (6); (6)； The first part represents the fusion of similarity among P feature partitions. Image similarity results; This indicates that it is based on the first The similarity is calculated from the unsegmented features of the images; in the fine-tuning stage, the network is optimized using the commonly used cross-entropy loss function; therefore, the total loss function of the proposed framework is as shown in Equation (7); (7)； in, This represents losses incurred during formal training.

Images