Unsupervised vehicle day and night cross-domain re-identification method based on comparative learning

Abstract

The invention relates to an unsupervised vehicle day and night cross-domain re-identification method based on comparative learning. The method comprises the following steps: obtaining original image data sets of a certain road section in the daytime and at night; respectively extracting features of the daytime image and the night image and a cluster corresponding to each modal image set by adopting two flow backbones; respectively using a clustering method on the two-modal feature space to generate a pseudo tag and calculate a cluster center, and then performing data enhancement to obtain a training sample set for training; and constructing a model loss function based on the cluster center, and training the model by using the loss function so as to obtain a finally trained cross-modal recognition model. According to the method provided by the invention, the accuracy and robustness of day and night vehicle re-identification can be remarkably improved under the condition of no manual annotation.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, and in particular to an unsupervised cross-domain re-identification method for vehicles based on contrastive learning. Background Technology

[0002] Vehicle re-identification refers to the identification and comparison of vehicle images from different cameras or from different perspectives. With the continuous increase in the global vehicle population, this trend has not only exacerbated traffic congestion but also placed enormous pressure on the environment. Against this backdrop, vehicle re-identification technology has become one of the key technologies in the construction of intelligent transportation systems and smart cities.

[0003] Despite significant progress in vehicle re-identification technology in recent years, most research still focuses on re-identification in the single-modal visible light domain. Research on cross-domain vehicle re-identification for day-night scene changes remains relatively scarce, and publicly available day-night cross-domain datasets are limited, especially in the area of ​​unsupervised day-night cross-domain vehicle re-identification, where related research is still in its early stages.

[0004] Unsupervised day-night cross-domain vehicle re-identification differs from ordinary vehicle re-identification in the following two main aspects:

[0005] (1) Traditional vehicle re-identification methods can typically only retrieve visible light (RGB) images captured under natural daylight conditions. However, cameras in real-world intelligent transportation systems need to continuously monitor both day and night. Image data under the two lighting conditions differ significantly in brightness, color distribution, and reflectivity, leading to a noticeable shift in data distribution and making traditional methods ineffective for cross-domain retrieval. The day-night cross-domain vehicle re-identification method addresses the inter-domain differences caused by changes in lighting conditions by designing an inter-modal feature alignment mechanism, thereby enabling effective matching and retrieval of vehicle images between daytime and nighttime modalities.

[0006] (2) Traditional vehicle re-identification methods and some cross-domain re-identification methods rely on supervised learning, which requires manual labeling of images. However, unsupervised day and night cross-domain vehicle re-identification methods do not require manual labeling and can be trained directly using unlabeled data, thereby significantly reducing labor costs and having stronger generalization ability and practical application value.

[0007] This technology plays a crucial role in various scenarios, including intelligent transportation systems and urban surveillance and security management, especially in complex traffic environments requiring continuous 24 / 7 monitoring. Traditional vehicle re-identification algorithms are mostly trained on data under visible daylight conditions. When the monitoring system operates at night or in low-light environments, the accuracy drops significantly. If unsupervised vehicle re-identification can be achieved across day and night lighting conditions, users only need to input unlabeled daytime and nighttime vehicle image data during the training phase. The model can then automatically learn the mapping relationship between different lighting modalities, achieving vehicle identification retrieval regardless of lighting conditions. In tasks such as vehicle tracking, violation detection, and trajectory analysis, this model can quickly locate images and related information of the same vehicle at different time periods. The implementation of the unsupervised day-night cross-domain vehicle re-identification method is not only applicable to all-weather monitoring systems such as urban roads, parking lots, and highways, but also significantly reduces the manual annotation costs required for traditional supervised training, improves the model's adaptability to real-world complex lighting environments, and is of great significance for promoting the automation and low-cost deployment of intelligent transportation systems.

[0008] Current unsupervised cross-domain vehicle re-identification under day and night conditions faces the following challenges: First, in cross-domain scenarios, lighting conditions and imaging modalities differ significantly. Daytime images typically contain rich color and texture details, while nighttime images are affected by factors such as headlights, reflections, shadows, and low-light noise, resulting in incomplete vehicle appearance information and a large shift in feature distribution between the two modalities. Second, under unsupervised conditions, model training relies on automatically generated pseudo-labels for feature clustering and optimization. The accuracy and stability of these pseudo-labels directly affect the model's convergence performance. Due to the significant differences in vehicle features across different modalities, pseudo-label generation is prone to incorrect clustering or identity confusion, leading to difficulties in cross-modal feature alignment. Third, existing cross-domain learning methods mostly only constrain features within a modality, lacking an effective mechanism for constructing high-confidence sample correspondences between daytime and nighttime modalities, making it difficult for models to achieve consistent cross-modal learning. These issues collectively result in insufficient cross-domain recognition generalization ability of the model under unsupervised daytime and nighttime lighting conditions. Summary of the Invention

[0009] In view of the above-mentioned problems in the existing technology, the technical problem to be solved by the present invention is: how to improve the vehicle recognition accuracy in unsupervised day and night cross-domain scenarios.

[0010] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0011] An unsupervised cross-domain vehicle re-identification method based on contrastive learning includes the following steps:

[0012] S1: Select the original image dataset D of daytime modal vehicles for a certain road section. Dayand the original image dataset of nighttime modal vehicles D Night ;

[0013] S2: Using a two-stream backbone network for D Day and D Night Perform independent modal feature extraction and output D. Day The corresponding set of feature vectors X Day and D Night The corresponding set of feature vectors X Night The two-stream backbone network includes a daytime modality feature extraction subnetwork and a nighttime modality feature extraction subnetwork. Both the daytime modality feature extraction subnetwork and the nighttime modality feature extraction subnetwork are constructed based on residual convolutional networks, including input convolutional layers, batch normalization layers, nonlinear activation layers, and max pooling layers.

[0014] X Day and X Night X is obtained by sequentially applying the Jaccard similarity reordering algorithm and the DBSCAN unsupervised clustering algorithm. Day and X Night Their respective cluster sets, X Day and X Night Each set of clusters contains several clusters, and each cluster corresponds to a pseudo-label; each cluster contains several original images, and the pseudo-label of the original images contained in each cluster is the same as the pseudo-label of its cluster.

[0015] D Day The daytime modality dataset consists of each original image and its corresponding pseudo-label. Night Each original image and its corresponding pseudo-label constitute the nighttime modality dataset; each original image in the daytime modality dataset or the nighttime modality dataset is combined with its pseudo-label as a single original sample.

[0016] Data augmentation operations were performed on both the daytime modality dataset and the nighttime modality dataset. The pseudo-labels of the augmented images obtained through the augmentation operations were the same as the pseudo-labels of the original samples used for augmentation. The augmented daytime modality dataset and the augmented nighttime modality dataset were combined to form a pseudo-label training set. The images in the pseudo-label training set and their corresponding pseudo-labels constituted a training sample.

[0017] S3: Construct a cross-domain re-identification model M based on ResNet50, and initialize M with ResNet50 weights pre-trained on the ImageNet dataset; M includes a shallow feature extraction module and a shared deep feature extraction module; the shallow feature extraction module includes several layers of convolutional networks at the front end of ResNet50, and the shared deep feature extraction module includes several layers of convolutional networks at the back end of ResNet50.

[0018] S4: Construct the loss function for M Using the pseudo-labeled training set as input to M, alternating unidirectional cross-modal contrastive learning is employed. The parameters of M are optimized by performing backpropagation and gradient update. Training stops when the maximum number of iterations is reached or the loss function is minimized. At this point, the trained M' is obtained.

[0019] S5: Select the daytime modal image y of the vehicle to be queried, input y into M', M' searches for several images similar to y in the nighttime modal candidate image library, calculates the feature similarity between several images and y, and sorts the obtained feature similarity in ascending order. The images corresponding to the top-K feature similarities are output as the cross-modal retrieval results of y.

[0020] Preferably, in step S2, the steps for obtaining the corresponding pseudo-label for each original image are as follows:

[0021] S21: From X Day Select the d-th feature vector x d The similarity of x is calculated using the Jaccard reordering method. d With X Day The distance between all other feature vectors in the vector is used to obtain x. d The set of distances X to all other feature vectors D ;

[0022] S22: Define the clustering threshold and use the DBSCAN unsupervised clustering algorithm to cluster X. D The calculation is performed, and other feature vectors whose distance is less than the clustering threshold are considered to be related to x. d Having similar characteristics, will be with x d Feature vectors with similar characteristics are grouped into a cluster, which is assigned a pseudo-label, and all original images contained in the cluster receive the same pseudo-label as the cluster.

[0023] S23: Traverse all X Day and X Night Repeat steps S21-S22 to obtain pseudo-labels for all original images.

[0024] Preferably, in step S4, the loss function of M is constructed. The steps are as follows:

[0025] S41: Construct the daytime modal memory and the nighttime modal memory:

[0026] Calculate the average of the feature vectors corresponding to all original images within each cluster, and use the average of each cluster as the cluster center feature c. kBy averaging the eigenvectors, X Day and X Night Each corresponding cluster set yields several cluster center features;

[0027] The obtained cluster center features are normalized and combined with the pseudo-labeled training set to obtain X respectively. Day The corresponding daytime modal memory and X Night The corresponding nighttime modal memory bank;

[0028] During M training, the daytime modal memory and the nighttime modal memory are adaptively updated;

[0029] S42: Using X Day The calculated cluster center features are used to construct a daytime modal cluster map, which is then used to... Night The calculated cluster center features are used to construct the nighttime modal cluster map, and then the similarity matrix between the daytime modal cluster map and the nighttime modal cluster map is calculated.

[0030] Finally, using the daytime modal cluster graph and the nighttime modal cluster graph, a cross-modal cluster correspondence table is constructed through a progressive strategy;

[0031] The steps of the progressive strategy are as follows:

[0032] First, a similarity matrix is ​​constructed using the central features of the two-modal clusters. The reciprocal operation of the similarity matrix is ​​then performed to convert it into a cost matrix. The cost matrix is ​​then filled with zeros to achieve the desired dimension. At this point, the cost matrix is ​​considered to meet the matrix requirements for using the optimal allocation operation.

[0033] Secondly, based on the cost matrix, the Hungarian algorithm is used to perform optimal allocation operations on the cost matrix, and the row index and column index of the matched matrix are output. The row index and column index of the matrix represent the correspondence between the two modal clusters; where the row index represents the number of the daytime modal cluster; the column index represents the number of the nighttime modal cluster to which the daytime modal cluster is assigned.

[0034] Next, for the remaining clusters that failed to match in the previous two steps, the submatrix corresponding to each remaining cluster is extracted from the cost matrix, and the optimal allocation operation is performed again using the Hungarian algorithm, thereby realizing the progressive matching of the remaining clusters and obtaining the complete correspondence between the two modal clusters.

[0035] Finally, a cross-modal cluster correspondence table is constructed using row and column indices that represent the correspondence between the two modal clusters.

[0036] S43: Input the pseudo-labeled training set into M, and obtain the feature vector matrices of the daytime modality and the nighttime modality respectively through the shallow feature extraction module. Then, concatenate the feature vector matrices of the daytime modality and the nighttime modality, and input the concatenated feature vector matrix into the shared deep feature extraction module to output the dual-modality joint feature map. Next, perform global pooling on the dual-modality joint feature map, and then normalize the dual-modality joint feature map after global pooling to finally obtain the normalized dual-modality joint feature vector map.

[0037] S44: Construct a loss function using the normalized bimodal joint feature vector map, cross-modal cluster correspondence table, daytime modality memory, and nighttime modality memory. The steps are as follows:

[0038] S44-1: From X Day and X Night Choose any feature vector x i Calculate x i With cluster center vector c k The cosine similarity is calculated using the following formula:

[0039]

[0040] Among them, s i,k Represents cosine similarity. x represents i transpose;

[0041] For s i,k Perform temperature scaling to obtain the scaled cosine similarity z. i,k The calculation formula is as follows:

[0042]

[0043] Where τ represents temperature;

[0044] Define the pseudo-label cluster of the i-th training sample as y. i Then the contrastive learning loss based on pseudo-labels is:

[0045]

[0046] Where k represents the k-th pseudo-label cluster, and K represents the total number of clusters in the modal memory;

[0047] when When, it represents the single-mode loss of the daytime mode; when At that time, it represents the single-mode loss of the night mode;

[0048] S44-2: Constructing a cross-modal loss function from night to day and cross-modal loss function from day to night A cross-modal correspondence table is provided to represent the cross-modal correspondence of image feature vectors. This cross-modal correspondence is then used to construct... and The calculation expression is as follows:

[0049]

[0050] Where, N night N represents the total number of nighttime images. day K represents the total number of daytime images. day [k′] represents the k′-th cluster center vector in the daytime modal memory, K night [k″] represents the k″-th cluster center vector in the nighttime modal memory. express The cross-modal corresponding cluster center vector, express The cross-modal corresponding cluster center vector, This represents the original image corresponding to the i-th training sample in the night modality training set. This represents the original image corresponding to the j-th training sample in the daytime modality training set. This represents the normalized feature vector of the image from the i-th night. Represents the normalized feature vector of the j-th daytime image;

[0051] S44-3: Constructing the loss function of M The calculation expression is as follows:

[0052]

[0053] Where λ represents the weight parameter, This represents the cross-modal loss function when the number of training batches is even. When the number of training batches is odd

[0054] Preferably, in step S41, the adaptive update process of the daytime modal memory and the nighttime modal memory is as follows:

[0055] When a new original image is assigned to a pseudo-label cluster, the feature vector corresponding to the original image will update the center representation of the cluster according to a certain momentum weight, so that the cluster center can gradually complete the continuous adaptive update process of the modality memory bank between different training iterations.

[0056] Preferably, in step S5, the calculation process for the feature similarity value is as follows:

[0057] S51: Input y into M', where y contains N. qN vehicle samples q Feature extraction is performed to obtain the corresponding query feature matrix F. q The formula is as follows:

[0058]

[0059] Where f(·) represents the normalized feature of the output; Indicates the Nth q One vehicle sample;

[0060] S52: Extract features from all images in the nighttime modality image library to be searched, and obtain the image library feature matrix F. g The formula is as follows:

[0061]

[0062] Where, N g This represents the total number of images in the night modal image library to be searched. Indicates the Nth g One image sample;

[0063] M' is based on F q With F g The cross-modal similarity matrix is ​​calculated by multiplying the vectors between them, as shown in the following formula:

[0064]

[0065] To satisfy the distance metric form for re-identification retrieval, M' takes the inverse of D(s,t) as the final distance matrix, as shown in the following expression:

[0066] dist(i,j) = -D(i,j)

[0067] This matrix measures the query image. Image library The distance across different lighting conditions is used as the feature similarity value.

[0068] Compared with the prior art, the present invention has at least the following advantages:

[0069] The purpose of this invention is to address the shortcomings of existing unsupervised vehicle recognition models in day-night cross-domain scenarios, including limited cross-illumination feature alignment capabilities and low reliability of pseudo-labels. This invention proposes an unsupervised day-night cross-domain vehicle re-identification framework based on contrastive learning. By designing methods for cross-illumination feature matching and bidirectional modal alignment optimization, this invention achieves high-confidence correspondence and modal consistency learning of daytime and nighttime vehicle features under unsupervised conditions, thereby significantly improving the model's recognition accuracy and generalization ability in day-night cross-domain scenarios.

[0070] 1. To address the significant differences in illumination conditions and imaging modalities in daytime and nighttime cross-domain scenarios, leading to substantial shifts in feature distribution between daytime and nighttime vehicle images, this invention proposes an illumination-robust feature extraction mechanism. This mechanism involves data augmentation of daytime and nighttime images separately, inputting them into independent modal feature extraction sub-networks to obtain primary convolutional features for each modality. Subsequently, this invention fuses the primary features of the two modalities along the channel dimension and inputs them into a shared deep convolutional backbone network for joint modeling. Leveraging the ResNet residual structure, this invention can simultaneously model the structural consistency between modalities and the global dependencies across illumination conditions during deep feature learning, effectively eliminating apparent differences under different illumination conditions, allowing daytime and nighttime vehicle features to be encoded in a unified representation space. This mechanism improves the robustness of vehicle features across illumination conditions, laying a stable feature foundation for subsequent pseudo-label generation and cross-modal alignment.

[0071] 2. To address the issue that unsupervised training relies on automatically generated pseudo-labels, which can lead to clustering errors and identity confusion due to feature differences between different modalities, this invention proposes a self-generated and dynamically updated pseudo-label mechanism. It constructs an accurate sample similarity metric by reordering the distance matrix and uses a density clustering algorithm to generate high-quality pseudo-labels. Furthermore, this invention employs a momentum update strategy based on a memory repository, enabling each cluster center to continuously absorb feature representations from new samples, thereby gradually approaching the true identity center and suppressing the influence of noisy samples on the clustering results. This strategy effectively improves the stability and accuracy of pseudo-labels, ensuring that the model can continuously obtain reliable supervision signals under unsupervised conditions. This significantly improves training convergence and cross-modal recognition performance, making the model more suitable for large-scale, unlabeled, and low-cost deployment in urban scenarios.

[0072] 3. To address the problem that existing methods often only perform feature constraints within a single modality and lack an effective mechanism for establishing high-confidence sample correspondences between daytime and nighttime modalities, leading to difficulties in cross-modal consistency learning, this invention proposes a cross-illumination feature matching module and a bidirectional modal alignment optimization strategy. This invention establishes a one-to-one correspondence between daytime and nighttime clusters through cross-modal matching of cluster centers using the cross-illumination feature matching module. Then, the bidirectional modal alignment optimization strategy, combined with alternating cross-modal comparative learning, enables cross-modal samples to receive bidirectional constraints. This mechanism not only enhances the feature correlation between cross-modalities but also effectively improves the consistency of vehicle identity representations under cross-illumination conditions, thereby significantly improving the model's cross-domain generalization ability in practical applications. Attached Figure Description

[0073] Figure 1 This is a schematic diagram of the overall framework of the present invention.

[0074] Figure 2This is a schematic diagram of the training process of the cross-domain re-identification model of the present invention. Detailed Implementation

[0075] The present invention will now be described in further detail.

[0076] This invention discloses an unsupervised day-night cross-domain vehicle re-identification method based on contrastive learning. This method improves the identity matching and retrieval of vehicles under different lighting conditions by constructing a cross-domain vehicle re-identification network based on illumination feature matching and pseudo-label adaptive optimization.

[0077] See Figures 1-2 An unsupervised cross-domain re-identification method for vehicles based on contrastive learning, comprising the following steps:

[0078] S1: Select the original image dataset D of daytime modal vehicles for a certain road section. Day and the original image dataset of nighttime modal vehicles D Night ;

[0079] S2: Using a two-stream backbone network for D Day and D Night Perform independent modal feature extraction and output D. Day The corresponding set of feature vectors X Day and D Night The corresponding set of feature vectors X Night The two-stream backbone network includes a daytime modality feature extraction subnetwork and a nighttime modality feature extraction subnetwork. Both the daytime modality feature extraction subnetwork and the nighttime modality feature extraction subnetwork are constructed based on residual convolutional networks, including input convolutional layers, batch normalization layers, nonlinear activation layers, and max pooling layers.

[0080] When extracting features, the original image and the horizontally flipped image are input into the ResNet50 network respectively. The output features are obtained in inference mode. The features of the "original image + flipped image" are averaged to reduce the model's sensitivity to vehicle orientation and improve the reliability of pseudo-label generation. The purpose of this step is to extract the feature representation of each sample under the current model and provide input data for subsequent steps.

[0081] X Day and X Night X is obtained by sequentially applying the Jaccard similarity reordering algorithm and the DBSCAN unsupervised clustering algorithm. Day and X Night Their respective cluster sets, X Day and X NightEach cluster set contains several clusters, and each cluster corresponds to a pseudo-label; each cluster contains several original images, and the pseudo-label of the original images contained in each cluster is the same as the pseudo-label of its cluster; Jaccard reordering similarity algorithm and DBSCAN unsupervised clustering algorithm are existing technologies.

[0082] D Day The daytime modality dataset consists of each original image and its corresponding pseudo-label. Night Each original image and its corresponding pseudo-label constitute the nighttime modality dataset; each original image in the daytime modality dataset or the nighttime modality dataset is combined with its pseudo-label as a single original sample.

[0083] Data augmentation operations were performed on both the daytime and nighttime modal datasets. The pseudo-labels of the augmented images obtained through the augmentation operations were the same as the pseudo-labels of the original samples used for augmentation. The augmented daytime and nighttime modal datasets were combined to form a pseudo-labeled training set. The images in the pseudo-labeled training set and their corresponding pseudo-labels constituted a training sample. The data augmentation included operations such as boundary padding, random cropping, horizontal flipping, normalization, and random occlusion. Data augmentation can improve the robustness of the model to changes in illumination, viewpoint shift, and local occlusion.

[0084] In step S2, the steps for obtaining the corresponding pseudo-label for each original image are as follows:

[0085] S21: From X Day Select the d-th feature vector x d The similarity of x is calculated using the Jaccard reordering method. d With X Day The distance between all other feature vectors in the vector is used to obtain x. d The set of distances X to all other feature vectors D ;

[0086] S22: Define the clustering threshold and use the DBSCAN unsupervised clustering algorithm to cluster X. D The calculation is performed, and other feature vectors whose distance is less than the clustering threshold are considered to be related to x. d Having similar characteristics, will be with x d Feature vectors with similar characteristics are grouped into a cluster, which is assigned a pseudo-label, and all original images contained in the cluster receive the same pseudo-label as the cluster.

[0087] S23: Traverse all X Day and X Night Repeat steps S21-S22 to obtain pseudo-labels for all original images.

[0088] S3: A cross-domain re-identification model M is constructed based on ResNet50, and the weights of ResNet50 pre-trained on the ImageNet dataset are used to initialize M; M includes a shallow feature extraction module and a shared deep feature extraction module; the shallow feature extraction module includes several layers of convolutional networks at the front end of ResNet50, and the shared deep feature extraction module includes several layers of convolutional networks at the back end of ResNet50; the ImageNet dataset, the ResNet50 model, and the convolutional networks are all existing technologies;

[0089] S4: Construct the loss function for M Using the pseudo-labeled training set as input to M, alternating unidirectional cross-modal contrastive learning is employed. The parameters of M are optimized by performing backpropagation and gradient update. Training stops when the maximum number of iterations is reached or the loss function is minimized. At this point, the trained M' is obtained.

[0090] In step S4, the loss function of M is constructed. The steps are as follows:

[0091] S41: Construct the daytime modal memory and the nighttime modal memory:

[0092] Calculate the average of the feature vectors corresponding to all original images within each cluster, and use the average of each cluster as the cluster center feature c. k By averaging the eigenvectors, X Day and X Night Each corresponding cluster set yields several cluster center features;

[0093] The obtained cluster center features are normalized and combined with the pseudo-labeled training set to obtain X respectively. Day The corresponding daytime modal memory and X Night The corresponding nighttime modality memory; the normalization operation can keep all cluster centers at a uniform scale in the feature space, and the memory contains all the cluster center feature vectors corresponding to its modality, with the pseudo-label cluster number as the index;

[0094] During M training, the daytime modal memory and the nighttime modal memory are adaptively updated;

[0095] In step S41, the adaptive update process of the daytime modal memory and the nighttime modal memory is as follows:

[0096] When a new original image is assigned to a pseudo-label cluster, the feature vector corresponding to the original image will update the center representation of the cluster according to a certain momentum weight, so that the cluster center can gradually complete the continuous adaptive update process of the modality memory bank between different training iterations.

[0097] Specifically, after each round of training, the system re-executes the clustering and matching process based on the latest features of the current model, updates the feature centers of each cluster and the cross-modal correspondence, thereby achieving dynamic adaptive optimization of pseudo-labels and memory.

[0098] Let the center of the old cluster be c. y If the sample features in the current batch are x and the momentum coefficient is m, then:

[0099] c y ←m·c y +(1-m)·x

[0100] Normalization will be performed after the update:

[0101]

[0102] The new pseudo-label results will be used as new supervisory signals to input into the network in the next round of training, forming a closed-loop self-evolutionary training process;

[0103] The model integrates the feature vectors of the training data into the corresponding cluster center using momentum updates based on the pseudo-labels of the current batch of training data. This allows the cluster center to gradually approach more stable and representative modality-independent features as training progresses. In this way, the memory repository can be continuously and adaptively updated, thereby improving the robustness of pseudo-label learning and strengthening the feature consistency between daytime and nighttime modalities.

[0104] S42: Using X Day The calculated cluster center features are used to construct a daytime modal cluster map, which is then used to... Night The calculated cluster center features are used to construct the nighttime modal cluster map, and then the similarity matrix between the daytime modal cluster map and the nighttime modal cluster map is calculated.

[0105] Finally, using the daytime modal cluster graph and the nighttime modal cluster graph, a cross-modal cluster correspondence table is constructed through a progressive strategy;

[0106] The steps of the progressive strategy are as follows:

[0107] First, a similarity matrix is ​​constructed using the central features of the two-modality clusters. The reciprocal operation of the similarity matrix is ​​then performed to convert it into a cost matrix. The cost matrix is ​​then filled with zeros to ensure that it meets the matrix requirements for using the optimal allocation operation. Zero-filling means filling the gaps in the matrix with the number "0".

[0108] Secondly, based on the cost matrix, the Hungarian algorithm (which is an existing technology) is used to perform optimal allocation operations on the cost matrix, and the row index and column index of the matched matrix are output. The row index and column index of the matrix represent the correspondence between the two modal clusters; where the row index represents the number of the daytime modal cluster; the column index represents the number of the nighttime modal cluster to which the daytime modal cluster is assigned.

[0109] Next, for the remaining clusters that failed to match in the previous two steps, the submatrix corresponding to each remaining cluster is extracted from the cost matrix, and the optimal allocation operation is performed again using the Hungarian algorithm, thereby realizing the progressive matching of the remaining clusters and obtaining the complete correspondence between the two modal clusters.

[0110] Finally, a cross-modal cluster correspondence table is constructed using row and column indices that represent the correspondence between the two modal clusters.

[0111] The cross-modal cluster correspondence table includes the mapping relationship D2N from daytime pseudo-labels to nighttime pseudo-labels and the mapping relationship N2D from nighttime to daytime, which is mainly used to guide subsequent cross-modal comparison optimization. Through the above two-stage progressive strategy, the present invention can prioritize the establishment of mappings between high-confidence clusters and then gradually complete the remaining matching relationships, thereby improving the accuracy of pseudo-label generation.

[0112] S43: Input the pseudo-labeled training set into M. After passing through the shallow feature extraction module, the feature vector matrices of the daytime modality and the nighttime modality are obtained respectively. Then, the feature vector matrices of the daytime modality and the nighttime modality are concatenated. The concatenated feature vector matrix is ​​input into the shared deep feature extraction module, and the output is a dual-modality joint feature map. Next, global pooling is performed on the dual-modality joint feature map, and then normalization is performed on the globally pooled dual-modality joint feature map to finally obtain a normalized dual-modality joint feature vector map. The dual-modality joint feature map can enhance the structural consistency and global dependency between modalities; global pooling can compress the spatial dimension into a fixed-dimensional vector representation; batch normalization can obtain stable global vehicle features across illumination.

[0113] S44: Construct a loss function using the normalized bimodal joint feature vector map, cross-modal cluster correspondence table, daytime modality memory, and nighttime modality memory. The steps are as follows:

[0114] S44-1: From X Day and X Night Choose any feature vector x i Calculate x i With cluster center vector c k The cosine similarity is calculated using the following formula:

[0115]

[0116] Among them, s i,k Represents cosine similarity. x represents i transpose;

[0117] For s i,k Perform temperature scaling to obtain the scaled cosine similarity z. i,k The calculation formula is as follows:

[0118]

[0119] Where τ represents temperature;

[0120] Define the pseudo-label cluster of the i-th training sample as y. i Then the contrastive learning loss based on pseudo-labels is:

[0121]

[0122] Where k represents the k-th pseudo-label cluster, and K represents the total number of clusters in the modal memory;

[0123] when When, it represents the single-mode loss of the daytime mode; when At that time, it represents the single-mode loss of the night mode;

[0124] S44-2: Constructing a cross-modal loss function from night to day and cross-modal loss function from day to night A cross-modal correspondence table is provided to represent the cross-modal correspondence of image feature vectors. This cross-modal correspondence is then used to construct... and The calculation expression is as follows:

[0125]

[0126] Where, N night N represents the total number of nighttime images. day K represents the total number of daytime images. day [k′] represents the k′-th cluster center vector in the daytime modal memory, K night [k″] represents the k″-th cluster center vector in the nighttime modal memory. express The cross-modal corresponding cluster center vector, express The cross-modal corresponding cluster center vector, This represents the original image corresponding to the i-th training sample in the night modality training set. This represents the original image corresponding to the j-th training sample in the daytime modality training set. This represents the normalized feature vector of the image from the i-th night. Let represent the normalized feature vector of the j-th daytime image. During training, different cross-modal losses in different directions will be used alternately according to odd and even periods: when the period number is even, the contrast loss in the "day to night" direction will be used; when the period number is odd, the contrast loss in the "night to day" direction will be used.

[0127] S44-3: Constructing the loss function of M The calculation expression is as follows:

[0128]

[0129] Where λ represents the weight parameter, This represents the cross-modal loss function when the number of training batches is even. When the number of training batches is odd

[0130] S5: Select the daytime modal image y of the vehicle to be queried, input y into M', M' searches for several images similar to y in the nighttime modal candidate image library, calculates the feature similarity between several images and y, and sorts the obtained feature similarity in ascending order. The images corresponding to the top-K feature similarities are output as the cross-modal retrieval results of y.

[0131] In step S5, the calculation process for the feature similarity value is as follows:

[0132] S51: Input y into M', where y contains N. q N vehicle samples q Feature extraction is performed to obtain the corresponding query feature matrix F. q The formula is as follows:

[0133]

[0134] Where f(·) represents the normalized feature of the output; Indicates the Nth q One vehicle sample;

[0135] S52: Extract features from all images in the nighttime modality image library to be searched, and obtain the image library feature matrix F. g The formula is as follows:

[0136]

[0137] Where, N g This represents the total number of images in the night modal image library to be searched. Indicates the Nth g One image sample;

[0138] M' is based on F q With F g The cross-modal similarity matrix is ​​calculated by multiplying the vectors between them, as shown in the following formula:

[0139]

[0140] To satisfy the distance metric form for re-identification retrieval, M' takes the inverse of D(s,t) as the final distance matrix, as shown in the following expression:

[0141] dist(i,j) = -D(i,j)

[0142] This matrix measures the query image. Image library The distance across different lighting conditions is used as the feature similarity value.

[0143] Experimental content and results

[0144] To verify system performance, the system calculations include the following evaluation metrics:

[0145] CMC (Cumulative Matching Characteristic) curve value: measures the probability of a correct match appearing in a high-ranking position (using the recognition rate of key positions rank1 and rank5 as a performance indicator for cross-modal vehicle recognition).

[0146] mAP (mean Average Precision): Measures overall search accuracy;

[0147] The system ultimately outputs the following information:

[0148] 1. The vehicle matching results and corresponding rankings for each query image in the gallery;

[0149] 2. Evaluation indicators: CMC and mAP.

[0150] The invention will be further illustrated through experiments:

[0151] In this experiment, the images to be retrieved (query) are sorted in the base library gallery based on the calculated feature distance, and the images with the highest similarity are selected to achieve the goal of target retrieval.

[0152] This experiment uses the DN348 dataset.

[0153] The DN-348 dataset is a day-night dual-domain vehicle re-identification dataset, containing 34,077 images from 348 different vehicles, acquired under various lighting conditions and shooting angles. The training set contains 19,984 images for 200 vehicle identities; the test set contains 14,093 images for 148 vehicle identities. During the testing phase, this experiment evaluated the data in two cross-domain settings: day-to-night and night-to-day. The results are shown in Table 1.

[0154] Table 1 Comparison of test results on the DN348 dataset.

[0155]

[0156] As can be seen from the results in Table 1, the method proposed in this invention outperforms the comparative methods UntransReID and ADCA in both the Day-to-Night and Night-to-Day cross-domain directions.

[0157] Specifically: In the Day-to-Night direction, the Rank-1, Rank-5 and mAP of this method reached 43.78%, 61.55% and 52.69%, respectively; in the Night-to-Day direction, the Rank-1, Rank-5 and mAP of this method reached 41.08%, 59.19% and 49.75%, respectively.

[0158] This invention achieves better performance in unsupervised day-night cross-domain vehicle re-identification tasks, mainly due to the following three advantages:

[0159] 1. More comprehensive utilization of global information: In the DN-348 dataset, the number of vehicle image clusters under daytime and nighttime modalities often differs, leading to some clusters failing to find corresponding relationships. The cross-illumination feature matching module proposed in this invention obtains a cross-modal correspondence table through a progressive strategy, fully utilizing the global relationship between daytime and nighttime modalities and avoiding the problem of inaccurate pseudo-labels caused by relying solely on local information. Compared with UntransReID and ADCA, this invention improves the coverage and reliability of pseudo-labels, and can more effectively mine reliable correspondences between daytime and nighttime modalities.

[0160] 2. A network structure better suited for cross-modal vehicle scenarios: This invention employs a residual deep convolutional network based on ResNet50 as the feature extractor. Vehicle images exhibit strong geometric structure and weak global semantics in both daytime and nighttime scenes. Compared to the Transformer coding structure used by UntransReID, the residual convolutional structure can more effectively extract structural features such as edges and vehicle contours, while ensuring feature scale stability, making cross-lighting vehicle representations more discriminative.

[0161] 3. A more robust contrastive learning strategy: The bidirectional modality alignment optimization module proposed in this invention adopts an alternating update method, avoiding the noise amplification problem that may be caused by directly combining bidirectional contrastive losses. Compared with methods such as ADCA, this invention can more effectively reduce the feature differences between day and night modalities and improve the robustness of cross-domain recognition.

[0162] In summary, the unsupervised day-night cross-domain vehicle re-identification network based on illumination feature matching and pseudo-label adaptive optimization proposed in this invention achieves superior performance compared to existing methods on the DN-348 dataset, verifying its effectiveness and advancement.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An unsupervised vehicle diurnal cross-domain re-identification method based on contrastive learning, characterized in that: Comprising the following steps: S1: select a certain road section daytime modal vehicle original image dataset D Day and night modal vehicle original image dataset D Night ; S2: adopt two-stream backbone network to D Day and D Night independent modal feature extraction, output D Day corresponding feature vector set X Day and D Night corresponding feature vector set X Night ; the two-stream backbone network comprises a daytime modal feature extraction subnetwork and a night modal feature extraction subnetwork, the daytime modal feature extraction subnetwork and the night modal feature extraction subnetwork are both constructed based on a residual convolutional network, and comprise an input convolutional layer, a batch normalization layer, a nonlinear activation layer and a max-pooling layer. X Day and X Night X is obtained by sequentially applying the Jaccard similarity reordering algorithm and the DBSCAN unsupervised clustering algorithm. Day and X Night Their respective cluster sets, X Day and X Night Each set of clusters contains several clusters, and each cluster corresponds to a pseudo-label; each cluster contains several original images, and the pseudo-label of the original images contained in each cluster is the same as the pseudo-label of its cluster. D Day each original image and the pseudo label corresponding to each original image constitute a daytime modal dataset, D Night each original image and the pseudo label corresponding to each original image constitute a nighttime modal dataset; each original image in the daytime modal dataset or the nighttime modal dataset and its pseudo label combination as an original sample; Data augmentation operations are performed on both the daytime modal dataset and the nighttime modal dataset, the pseudo-labels of the enhanced images obtained through the augmentation operations are the same as the pseudo-labels of the original samples used for enhancement, and the enhanced daytime modal dataset and the enhanced nighttime modal dataset are integrated as a pseudo-labeled training set, wherein the images of the pseudo-labeled training set and the corresponding pseudo-labels thereof constitute a training sample; S3: Constructing a cross-domain re-identification model M based on ResNet50, the ResNet50 weights pre-trained on the ImageNet dataset are used to initialize M; M comprises a shallow feature extraction module and a shared deep feature extraction module; the shallow feature extraction module comprises a plurality of layers of convolutional networks in the front end of ResNet50, and the shared deep feature extraction module comprises a plurality of layers of convolutional networks in the back end of ResNet50; S4: Constructing the loss function of M Taking the pseudo-labeled training set as the input of M, adopting the alternating one-way cross-modal contrast learning, and optimizing the parameters of M through Performing back propagation and gradient update to optimize the parameters of M, stopping the training when the maximum number of iterations or the loss function is reached, and obtaining the trained M' at this time; S5: Selecting a daytime modal query vehicle image y, inputting y into M', searching for a plurality of images similar to y in the nighttime modal candidate image library, calculating the feature similarity between the plurality of images and y, and arranging the obtained feature similarity in ascending order, and outputting the images corresponding to the first Top-K feature similarities as the cross-modal retrieval results of y.

2. The unsupervised vehicle diurnal cross-domain re-identification method based on contrastive learning according to claim 1, wherein: In S2, the step of obtaining a corresponding pseudo-label for each original image is as follows: S21: Select the dth feature vector x Day from X d , compute the distance between x d and all the other feature vectors in X Day using the Jaccard re-ranked similarity method, and get the distance set X d between x D and all the other feature vectors. S22: defining a clustering threshold, using DBSCAN unsupervised clustering algorithm to X D Calculate, and consider other feature vectors with distance less than the clustering threshold as having similar features with x d x d The feature vectors with similar features are classified into a cluster, which is assigned a pseudo label, and all original images contained in the cluster obtain the same pseudo label as the cluster. S23: traverse all X Day and X Night , repeat S21-S22 to obtain pseudo labels of all original images.

3. The unsupervised vehicle diurnal cross-domain re-identification method based on contrastive learning of claim 2, wherein: In the S4, the loss function of M is constructed The steps are as follows: S41: Constructing a daytime modal memory bank and a nighttime modal memory bank: The average value of the feature vectors corresponding to all original image pairs in each cluster is calculated, and the average value corresponding to each cluster is taken as the cluster center feature c of the cluster k ; through the average value operation of the feature vectors, X Day and X Night correspond to a cluster set respectively to obtain a plurality of cluster center features; The obtained several cluster center features are normalized, and X Day corresponding daytime modal memory banks and X Night corresponding nighttime modal memory banks are obtained respectively by combining the pseudo-labeled training set. The daytime modal memory bank and the nighttime modal memory bank are adaptively updated during the training of M; S42: using X Day The cluster center features calculated are used to construct a daytime modal cluster graph, and using X Night The cluster center features calculated are used to construct a nighttime modal cluster graph, and then a similarity matrix between the daytime modal cluster graph and the nighttime modal cluster graph is calculated; Finally, the daytime modal cluster graph and the nighttime modal cluster graph are used to construct a cross-modal cluster correspondence table through a progressive strategy. The steps of the progressive strategy are as follows: First, a similarity matrix is constructed using the two-modal cluster center features, the similarity matrix is converted into a cost matrix through inverse operation, and the cost matrix is dimensionally filled in a zero-padded manner, at which time the cost matrix is considered to meet the matrix requirements for optimal allocation operation; Second, based on the cost matrix, the Hungarian algorithm is used to perform optimal allocation operation on the cost matrix, and the matched matrix row index and matrix column index are output, which represent the correspondence relationship between the two modal clusters; wherein the matrix row index represents the number of daytime modal clusters; the matrix column index represents the number of nighttime modal clusters to which the daytime modal cluster is assigned; Third, for the remaining clusters that failed to match in the foregoing two steps, a sub-matrix corresponding to each remaining cluster is extracted from the cost matrix, and the Hungarian algorithm is used again to perform optimal allocation operation, thereby realizing progressive matching of the remaining clusters and obtaining the complete correspondence relationship between the two modal clusters; Finally, the row index and the column index representing the correspondence relationship between the two modal clusters are used to construct a cross-modal cluster correspondence table. S43: input the pseudo-labeled training set into M, respectively obtain the feature vector matrix of the daytime modality and the feature vector matrix of the nighttime modality through the shallow feature extraction module, then splice the feature vector matrix of the daytime modality and the feature vector matrix of the nighttime modality, input the spliced feature vector matrix into the shared deep feature extraction module, and output to obtain a dual-modality joint feature map; then, the dual-modality joint feature map is subjected to global pooling processing, and then the dual-modality joint feature map subjected to the global pooling processing is subjected to normalization processing, and finally a normalized dual-modality joint feature vector map is obtained; S44: construct a loss function by using the normalized dual-modality joint feature vector map, the cross-modality cluster correspondence table, the daytime modality memory bank and the nighttime modality memory bank, and the steps are as follows: S44-1: From X Day and X Night optional feature vector x i , calculate the cosine similarity of x i and cluster center vector c k , the calculation formula is as follows: where s i,k denotes the cosine similarity, denotes the transpose of x i ; s i,k Perform temperature scaling to get scaled cosine similarity z i,k The calculation formula is as follows: Wherein, τ represents temperature; The pseudo label cluster defining the i-th training sample is denoted as y i The contrastive learning loss based on the pseudo label is: Wherein, k represents the kth pseudo-label cluster, and K represents the total number of clusters in the modality memory bank; when When, it represents the single-mode loss of the daytime mode; when At that time, it represents the single-mode loss of the night mode; S44-2: Constructing the cross-modal loss function from night to day and the cross-modal loss function from day to night The cross-modal correspondence of the image feature vectors is provided through the cross-modal cluster correspondence table, and the cross-modal loss function is constructed by using the cross-modal correspondence and The calculation expression is as follows: where N night denotes the total number of night images, N day denotes the total number of day images, K day [k'] denotes the k'th cluster center vector in the day modality memory bank, K night [k"] denotes the k"th cluster center vector in the night modality memory bank, denotes the cross-modality corresponding cluster center vector of denotes the cross-modality corresponding cluster center vector of denotes the original image corresponding to the i-th training sample in the night modality training set, denotes the original image corresponding to the j-th training sample in the day modality training set, denotes the normalized feature vector of the i-th night image, denotes the normalized feature vector of the j-th day image; S44-3: Constructing the loss function of M The calculation expression is as follows: where λ denotes a weight parameter, denotes the cross-modal loss function when the training batch is even when the training batch is odd, 4. The unsupervised vehicle diurnal cross-domain re-identification method based on contrastive learning of claim 3, wherein: In the S41, the process of self-adaptive updating of the daytime modality memory bank and the nighttime modality memory bank is as follows: When a new original image is assigned to a certain pseudo-label cluster, the feature vector corresponding to the original image will update the center representation of the cluster with a certain momentum weight, so that the cluster center can gradually complete the continuous self-adaptive updating process of the modality memory bank between different training iterations.

5. The unsupervised vehicle diurnal cross-domain re-identification method based on contrastive learning of claim 4, wherein: In the S5, the calculation process of the feature similarity value is as follows: S51: input y into M', y contains N q vehicle samples, N q extract features to obtain corresponding query feature matrix F q , the formula is as follows: where f(·) denotes the normalized features of the output; represents the N q th vehicle sample; S52: Feature extraction is performed on all images in the night mode gallery to be searched to obtain a gallery feature matrix F g The formula is as follows: wherein N g represents the total number of images in the night modality gallery to be searched, represents the N g th image sample; M' based on F q with F g The cross-modal similarity matrix is calculated by the vector product between M and F, as follows: In order to meet the distance measurement form of the re-identification retrieval, M' takes the inverse number of D(s,t) as the final distance matrix, and the expression is as follows: dist(i,j) = -D(i,j) The matrix measures the query image to the gallery image The distance across the lighting conditions as a feature similarity value.

Images