Unsupervised visible-infrared pedestrian re-identification method based on contrastive learning

Abstract

The present application relates to a kind of based on contrast learning's unsupervised visible light-infrared pedestrian re-identification method, belong to computer vision and image processing technical field.It includes the following steps: obtaining the original visible light photo and original infrared light photo of pedestrian;With ReID model as baseline, extract double mode shallow layer and deep layer feature;By clustering to generate pseudo label, construct deep and shallow two layers clustering prototype memory and momentum update, calculate modal clustering level contrast baseline loss;Introduce the modal confuser of gradient inversion layer and weaken modal distinction;Design double scale interactive neighbor learning module, fusion deep and shallow feature similarity constructs modal and cross-modal neighbor set, carries out instance level contrast learning;Add cross-modal relationship correction module, after hungarian matching and deep and shallow fusion similarity correction prototype corresponding relationship, construct cross-modal clustering level contrast loss;With joint total loss end-to-end training.The present application can effectively improve cross-modal feature alignment capability and pseudo corresponding reliability.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, specifically relating to an unsupervised visible light-infrared pedestrian re-identification method based on contrastive learning. Background Technology

[0002] Pedestrian re-identification aims to retrieve the same pedestrian across non-overlapping camera views. With the widespread application of video surveillance systems in public safety, relying solely on visible light cameras has significant limitations in nighttime or low-light environments. Infrared imaging provides complementary information by sensing thermal radiation, making visible-infrared person re-identification (VI-ReID) an important research direction. Most existing VI-ReID methods rely on large-scale, precisely labeled cross-modal identity tags, resulting in high labeling costs. To address this issue, unsupervised visible-infrared person re-identification (Unsupervised VI-ReID, US-VI-ReID) has attracted attention. Existing unsupervised methods mainly employ clustering to generate pseudo-labels or contrastive learning based on neighbor mining. However, due to the significant differences between visible and infrared modalities, cross-modal matching is prone to generating erroneous pseudo-correspondences (false positives / false negatives), leading to contamination of the contrastive supervision signal, unstable training, and difficulty in learning modality-invariant features. Furthermore, inconsistencies exist between deep and shallow features in neighbor mining, further reducing the reliability of instance-level contrastive learning. The ADCA method proposes using a dual-contrast learning framework to optimize the learning of modality-invariant features. Among methods for cross-modal learning, PGM proposes using graph matching to match centroid features of the visible and infrared light modalities. Later, CCLNet utilizes the visual-linguistic representation capabilities of the CLIP model to generate high-confidence pseudo-labels to supervise model training. SDCL uses neighbor contrastive learning between instances to obtain reliable neighbors and leverages the principle of shallow / deep ranking consistency to mine relationships between modalities. CSANet proposes a self-adjusting relationship network that mimics human progression from simple to complex cross-modal relationships.

[0003] Existing methods suffer from two problems: first, neighbor-based learning mechanisms cannot capture fine-grained relational structures, and neighbors with inconsistent feature depths may still benefit the model's learning; second, due to intermodal differences, cross-modal relationships generated by pseudo-supervision still contain noise, causing error propagation. Therefore, effectively suppressing modal differences and improving the reliability of cross-modal pseudo-correspondences under unsupervised conditions is a pressing technical problem. To obtain fine-grained neighbor sets and more accurate cross-modal relationships, this paper proposes an unsupervised visible-infrared pedestrian re-identification method based on cross-modal dual-contrast learning. Summary of the Invention

[0004] To achieve the above objectives, the present invention employs the following technical solution: This invention provides an unsupervised visible light-infrared pedestrian re-identification method based on contrastive learning, comprising the following steps: S1. Obtain raw visible light and raw infrared light images of pedestrians and construct a training dataset; S2. Construct a pedestrian re-identification model. The pedestrian re-identification model adopts the Transformer-based ReID model as the baseline model. The training dataset is input into the pedestrian re-identification model, and the loss function is used to train the pedestrian re-identification model to obtain the trained pedestrian re-identification model. S3. Input the images of the pedestrian to be identified and the known pedestrians into the trained pedestrian re-identification model to extract deep features, calculate the similarity between the deep features of the pedestrian to be identified and the deep features of the known pedestrians, sort them in descending order of similarity to obtain a sorted list of the deep features of the known pedestrians, obtain the k most similar deep features of the pedestrians, and finally return the images corresponding to the k deep features of the pedestrians in order.

[0005] Furthermore, in step S2, the baseline model specifically includes: The original visible light image is input into the baseline model for feature extraction, resulting in shallow and deep features of the original visible light image. The Transformer-based ReID model consists of 12 stacked transformer blocks, each including an attention mechanism, residual connections, normalization, and a fully connected layer. The original visible light image passes through the first transformer block to obtain shallow features, and through the 12 transformer blocks to obtain deep features. Similarly, the original infrared photo is input into the baseline model for feature extraction to obtain the shallow and deep features of the original infrared photo.

[0006] Furthermore, in step S2, the shallow and deep features of the original visible light image and the shallow and deep features of the original infrared image are clustered to generate pseudo-labels, and shallow clustering prototype memory and deep clustering prototype memory are constructed respectively; clustering-level contrastive loss is used for comparative learning of the baseline model.

[0007] Furthermore, shallow clustering prototype memories and deep clustering prototype memories are constructed: For the visible light mode, the shallow and deep features of the original visible light image are concatenated to obtain a concatenated vector. This concatenated vector is then used as a clustering feature to perform clustering operations using the DBSCAN clustering algorithm, resulting in pseudo-labels for the visible light image. Based on these pseudo-labels, shallow and deep clustering prototype memories for the visible light mode are constructed. Each cluster prototype is the average vector of the features of all samples within that cluster, and the resulting average vector is the prototype feature of each pseudo-label. Similarly, for the infrared light mode, the shallow and deep features of the original infrared light image are processed using the same method to obtain shallow and deep clustering prototype memories for the infrared light mode.

[0008] Further, in step S2, the baseline loss is constructed: Cluster-level contrastive loss functions are used to perform cluster-level contrastive learning on shallow and deep features within each modality. The cluster-level contrastive loss for shallow features is: , in, , Represents visible light modes, Indicates infrared light mode; Indicates the first Shallow features of each sample; Indicates the first Shallow prototype features of each cluster; This indicates the prototype of the cluster to which the current sample belongs; τ is the temperature parameter. The cluster-level contrastive loss for deep features takes the same form as that for shallow features. The centroid features in the shallow clustering prototype memory and the deep clustering prototype memory are updated using momentum, as follows: , in, Indicates the first The prototype memory of the next iteration A centroid feature, Represents the number of training iterations. Indicates the momentum coefficient. This indicates that it belongs to a pseudo tag. Instance features; baseline loss is = + + + ,in, Cluster-level contrast loss representing shallow features of visible light modes, Cluster-level contrast loss representing shallow features of infrared light modes, Cluster-level contrast loss representing deep features of visible light modes, Cluster-level contrast loss representing deep features of infrared light modes.

[0009] Furthermore, in step S2, the pedestrian re-identification model also includes a modality obfuscator module: The modality confusion module includes a binary modality classifier and a gradient inversion layer; the modality label of the original visible light image is set to 1, and the modality label of the original infrared light image is set to 0; the deep features of the original visible light image and the original infrared light image are input into the binary modality classifier, and the output result is subjected to cross-entropy loss with the modality label; Loss function of binary modality classifier for: ,in, This represents a batch stitching of deep features from the original visible light image and the original infrared image. Indicates modal label; This represents a binary modal classifier; Represents the normalized exponential function, Represents cross-entropy loss; loss function of gradient inversion layer for: ,in, This indicates a gradient inversion layer; The final loss of the modal confused module is obtained. , .

[0010] Furthermore, in step S2, the pedestrian re-identification model also includes a dual-scale interactive neighbor learning module: Through the dual-scale interactive neighbor learning module, the similarity between shallow and deep features is fused to construct the neighbor set within the modality and the neighbor set across modalities for each sample feature, and instance-level comparative learning is performed based on the neighbor set; For the original visible light image, calculate the cosine similarity between shallow features, the cosine similarity between deep features, the cosine similarity between shallow features of the original visible light image and shallow features of the original infrared image, and the cosine similarity between deep features of the original visible light image and deep features of the original infrared image. Intramodal fusion similarity is constructed from the original visible light image. ; Construct fusion similarity between the original visible light image and the original infrared image. For the first An original visible light photograph Based on fusion similarity, a threshold is used to construct the intra-modal neighbor set of the original visible light image. and modal neighbor set ; Using in-modal neighbor sets and modal neighbor set Calculate the instance-level contrastive loss function for deep features within the visible light mode. Instance-level contrastive loss function for shallow features between visible light modes Instance-level contrastive loss function for shallow features within the visible light mode and the instance-level contrastive loss function for deep features between visible light modes The formula for the instance-level contrastive loss function for deep features within the visible light mode is as follows: , in, Indicates the number of instances in the current batch. The exponential function is represented; the other three loss functions have the same form as the instance-level comparison loss function for deep features in the visible light modality. Similarly, for the original infrared image, the instance-level contrast loss function of deep features within the infrared mode is calculated. Instance-level contrast loss function for shallow features between infrared light modes Infrared light mode shallow feature instance-level contrast loss function and the deep feature instance-level contrast loss function between infrared light modes ; Finally, the two-scale interactive neighbor learning loss is obtained. , .

[0011] Furthermore, in step S2, the pedestrian re-identification model also includes a cross-modal relationship correction module: The Hungarian algorithm is used to process the visible light shallow-layer prototype features and the infrared light shallow-layer prototype features to obtain the correspondence between the visible light shallow-layer prototype features and the infrared light shallow-layer prototype features. Similarly, the correspondence between visible light deep prototype features and infrared light deep prototype features is obtained. Correspondence between infrared shallow prototype features and visible light shallow prototype features And the correspondence between infrared deep prototype features and visible light deep prototype features. ; The correspondence between visible light shallow-layer prototype features and infrared light shallow-layer prototype features. Correspondence between visible light deep prototype features and infrared light deep prototype features Inconsistent correspondences are extracted, and the inconsistencies are corrected by calculating prototype fusion similarity. The visible light deep prototype features and infrared deep prototype features corresponding to the maximum prototype fusion similarity are mapped to obtain the corrected correspondences. The consistent correspondences and the corrected correspondences are then merged to obtain the visible light to infrared light matching relationship. Similarly, the correspondence between shallow infrared prototype features and shallow visible light prototype features can be established. Correspondence between infrared deep prototype features and visible light deep prototype features The matching relationship between infrared and visible light was obtained. ; Constructing cross-modal clustering-level contrastive loss , ; Calculate the deep loss between the visible light instance and the infrared light prototype. The formula is expressed as follows: , in, Indicates the batch size of samples loaded into the dataset. Indicates pseudo-tags as The deep prototype features of infrared light corresponding to visible light. This represents the number of deep prototype features in infrared light; similarly, the shallow loss between visible light instances and infrared prototypes is obtained. Shallow loss between infrared light instances and visible light prototypes Deep loss between infrared light instances and visible light prototypes .

[0012] Furthermore, in step S2, the specific process for calculating the prototype fusion similarity is as follows: calculate the visible light deep prototype features. Infrared deep prototype features cosine similarity Visible light shallow prototype characteristics Similar to shallow prototype features of infrared light cosine similarity ,Will and The prototype fusion similarity is obtained by performing fusion. , , This represents the second balance coefficient.

[0013] Furthermore, the loss function adopts the joint optimization total loss function. The formula is expressed as follows: , in, Indicates baseline loss. This indicates the loss of the modal confused module. This represents the loss from two-scale interactive neighbor learning. This represents the cross-modal clustering level contrastive loss. This represents the third balance coefficient.

[0014] The advantages of this invention are: This invention integrates instance-level and class-level contrastive targets to learn robust modality-invariant representations under unlabeled supervision, thereby achieving reliable cross-modal retrieval. The unsupervised visible-infrared pedestrian re-identification method based on contrastive learning consists of three parts: a modality obfuscator module suppresses modality-specific cues and mitigates modality-induced bias at the feature representation level; a dual-scale interactive neighbor learning module utilizes the complementarity between shallow and deep features to refine the neighborhood structure, thereby strengthening instance-level contrastive supervision; and a cross-modal relationship correction module further corrects noisy cross-modal associations through prototype-level consistency, improving global ranking quality. Extensive experiments on public pedestrian re-identification benchmarks validate the effectiveness of the method and demonstrate its stable and competitive performance under different evaluation metrics. These results show that this invention can eliminate the bias of modality-specific information and correct spurious correspondences. Attached Figure Description

[0015] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0016] Figure 1 This is a flowchart of the steps of the method of the present invention; Figure 2 This is a data flow diagram of the method of the present invention; Figure 3 This is a comparison of experimental results between the baseline model and the method of this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1 In this embodiment, as Figure 1 As shown, this invention provides an unsupervised visible-infrared pedestrian re-identification method based on contrastive learning, the specific steps of which include: S1. Obtain the raw visible light and raw infrared light images of pedestrians and construct a training dataset; each pedestrian has a corresponding visible light image and infrared light image. S2. Construct a pedestrian re-identification model. The pedestrian re-identification model adopts the Transformer-based ReID model as the baseline model. The training dataset is input into the pedestrian re-identification model, and the loss function is used to train the pedestrian re-identification model to obtain the trained pedestrian re-identification model. The original visible light image is input into the baseline model for feature extraction, resulting in shallow and deep features of the original visible light image. The Transformer-based ReID model consists of 12 stacked transformer blocks, each including an attention mechanism, residual connections, normalization, and a fully connected layer. The original visible light image passes through the first transformer block to obtain shallow features, and through the 12 transformer blocks to obtain deep features. Similarly, the original infrared photo is input into the baseline model for feature extraction to obtain the shallow and deep features of the original infrared photo. Clustering is performed on the shallow and deep features of the original visible light image and the shallow and deep features of the original infrared image to generate pseudo-labels. Shallow clustering prototype memory and deep clustering prototype memory are constructed respectively. Clustering-level contrastive loss is used for contrastive learning of the baseline model to bring the image features of each pedestrian closer to their corresponding centroid features and push the distance between the pedestrian and other centroid features, that is, to bring similar samples closer and push dissimilar samples further apart. For the visible light modality, the shallow and deep features of the original visible light image are concatenated to obtain a concatenated vector. This concatenated vector is then used as a clustering feature and clustered using the DBSCAN clustering algorithm to obtain pseudo-labels for the visible light image. Each sample feature has a corresponding pseudo-label, and features with the same pseudo-label are considered to belong to the same pedestrian. Based on the pseudo-labels of the visible light image, a shallow clustering prototype memory and a deep clustering prototype memory for the visible light modality are constructed. Each cluster prototype is the average vector of all sample features within that cluster, and the resulting average vector is the prototype feature of each pseudo-label. Similarly, for the infrared light modality, the shallow and deep features of the original infrared light image are processed in the same way to obtain a shallow clustering prototype memory and a deep clustering prototype memory for the infrared light modality. Cluster-level contrastive loss functions are used to perform cluster-level contrastive learning on shallow and deep features within each modality. The cluster-level contrastive loss for shallow features is: , in, , Represents visible light modes, Indicates infrared light mode; Indicates the first Shallow features of each sample; Indicates the first Shallow prototype features of each cluster; This indicates the prototype of the cluster to which the current sample belongs; τ is the temperature parameter. The cluster-level contrastive loss for deep features takes the same form as that for shallow features. The centroid features in the shallow clustering prototype memory and the deep clustering prototype memory are updated using momentum, as follows: , in, Represents the first in the prototype memory bank A centroid feature, Represents the number of training iterations. Indicates the momentum coefficient. This indicates that it belongs to a pseudo tag. The instance features, the baseline loss is = + + + ,in, Cluster-level contrast loss representing shallow features of visible light modes, Cluster-level contrast loss representing shallow features of infrared light modes, Cluster-level contrast loss representing deep features of visible light modes, Cluster-level contrast loss representing deep features of infrared light modes.

[0019] The pedestrian re-identification model also includes a modality obfuscator module, a dual-scale interactive neighbor learning module, and a cross-modal relationship correction module.

[0020] The modality obfuscator module includes a binary modality classifier and a gradient inversion layer. The binary modality classifier is a fully connected layer, and the gradient inversion layer inverts the gradient during backpropagation. The modality label of the original visible light image is set to 1, and the modality label of the original infrared image is set to 0. The deep features of the original visible light image and the original infrared image are input into the binary modality classifier, and the output result is subjected to cross-entropy loss with the modality label. After obtaining the cross-entropy loss, the classification ability of the classifier is optimized. Since the classifier's role is to classify the modalities of the image, when backpropagating to optimize the baseline model, this classification ability is countered to optimize the model with the counter-loss, making the model unable to distinguish the modalities of the features.

[0021] Loss function of binary modality classifier for: ,in, This indicates a batch stitching of deep features from the original visible light image and the original infrared image; z represents the modal label. This represents a binary modal classifier; Represents the normalized exponential function, Represents cross-entropy loss; loss function of gradient inversion layer for: ,in, This indicates a gradient inversion layer.

[0022] The final loss of the modal confused module is obtained. , .

[0023] Through the dual-scale interactive neighbor learning module, the similarity between shallow and deep features is fused to construct the neighbor set within the modality and the neighbor set across modalities for each sample feature, and instance-level comparative learning is performed based on the neighbor set; For the original visible light image, calculate the cosine similarity between shallow features, the cosine similarity between deep features, the cosine similarity between shallow features of the original visible light image and shallow features of the original infrared image, and the cosine similarity between deep features of the original visible light image and deep features of the original infrared image. Intramodal fusion similarity is constructed from the original visible light image. The formula is expressed as follows: , in, Indicates the first A raw visible light photograph, Indicates the first One original visible light photograph; Indicates the first balance coefficient; Indicates the first Deep features of an original visible light photograph and the Deep features of an original visible light photograph Cosine similarity between them; Indicates the first Shallow features of an original visible light photograph and the Deep features of an original visible light photograph Cosine similarity between them; Build fusion similarity between the original visible light image and the original infrared image. The formula is expressed as follows: , in, Indicates the first One original infrared photograph; Indicates the first Deep features of an original visible light photograph and the Deep features of a raw infrared photograph Cosine similarity between them; Indicates the first Shallow features of an original visible light photograph and the Shallow features of a raw infrared photograph Cosine similarity between them.

[0024] For the An original visible light photograph Based on fusion similarity, a threshold is used to construct the intra-modal neighbor set of the original visible light image. and modal neighbor set The formula is expressed as follows: , , in, This represents the threshold, and `max` indicates the operation of taking the maximum value. This indicates the number of original visible light images. This indicates the number of original infrared images.

[0025] Intra-modal neighbor sets and inter-modal neighbor sets are used to perform instance-level comparative learning between visible light and visible light shallow and deep features. Inter-modal neighbor sets are used to perform instance-level comparative learning between visible light and infrared light shallow and deep features. Using in-modal neighbor sets and modal neighbor set Calculate the instance-level contrastive loss function for deep features within the visible light mode. Instance-level contrastive loss function for shallow features between visible light modes Instance-level contrastive loss function for shallow features within the visible light mode and the instance-level contrastive loss function for deep features between visible light modes The formula for the instance-level contrastive loss function for deep features within the visible light mode is as follows: , in, Indicates the number of instances in the current batch. The first loss function represents the exponential function; the other three loss functions have the same form as the deep feature instance-level contrast loss function in the visible light mode.

[0026] Similarly, for the original infrared image, the instance-level contrast loss function of deep features within the infrared mode is calculated. Instance-level contrast loss function for shallow features between infrared light modes Infrared light mode shallow feature instance-level contrast loss function and the deep feature instance-level contrast loss function between infrared light modes .

[0027] Finally, the two-scale interactive neighbor learning loss is obtained. , .

[0028] The cross-modal relationship correction module specifically includes: The Hungarian algorithm is used to process the visible light shallow-layer prototype features and the infrared light shallow-layer prototype features to obtain the correspondence between the visible light shallow-layer prototype features and the infrared light shallow-layer prototype features. Similarly, the correspondence between visible light deep prototype features and infrared light deep prototype features is obtained. Correspondence between infrared shallow prototype features and visible light shallow prototype features And the correspondence between infrared deep prototype features and visible light deep prototype features. s and d represent shallow and deep layers, respectively.

[0029] The correspondence between visible light shallow-layer prototype features and infrared light shallow-layer prototype features. Correspondence between visible light deep prototype features and infrared light deep prototype features Inconsistent correspondences are extracted, and the inconsistent correspondences are corrected by calculating prototype fusion similarity. The visible light deep prototype features and infrared deep prototype features corresponding to the maximum prototype fusion similarity are mapped to obtain the corrected correspondences. The consistent correspondences and the corrected correspondences are then merged to obtain the visible light to infrared light matching relationship. Similarly, the correspondence between shallow infrared prototype features and shallow visible light prototype features can be established. Correspondence between infrared deep prototype features and visible light deep prototype features The matching relationship between infrared and visible light was obtained. .

[0030] The specific process for calculating prototype fusion similarity is as follows: Calculate the deep prototype features in visible light. Infrared deep prototype features cosine similarity Visible light shallow prototype characteristics Similar to shallow prototype features of infrared light cosine similarity ,Will and The prototype fusion similarity is obtained by performing fusion. , , This represents the second balance coefficient.

[0031] Constructing cross-modal clustering-level contrastive loss , ; Calculate the deep loss between the visible light instance and the infrared light prototype. The formula is expressed as follows: , in, Indicates the batch size of samples loaded into the dataset. Indicates pseudo-tags as The deep prototype features of infrared light corresponding to visible light. This represents the number of deep prototype features in infrared light; similarly, the shallow loss between visible light instances and infrared prototypes is obtained. Shallow loss between infrared light instances and visible light prototypes Deep loss between infrared light instances and visible light prototypes .

[0032] The total loss function of joint optimization for , in, This indicates the loss of the modal confused module. This represents the loss from two-scale interactive neighbor learning. This represents the cross-modal clustering level contrastive loss. This is the third balance coefficient.

[0033] S3. Input the images of the pedestrian to be identified and the known pedestrians into the trained pedestrian re-identification model to extract deep features, calculate the similarity between the deep features of the pedestrian to be identified and the deep features of the known pedestrians, sort them in descending order of similarity to obtain a sorted list of the deep features of the known pedestrians, obtain the k most similar deep features of the pedestrians, and finally return the images corresponding to the k deep features of the pedestrians in order.

[0034] The data flow diagram of the method of this invention is as follows: Figure 2 As shown.

[0035] Example 2 In this embodiment, the baseline model is TransReID, which employs an enhanced dual-contrast learning strategy. The DBSCAN algorithm is used for clustering to obtain pseudo-labels. Visible light and infrared light images are set to 384*128 and fed into the neural network. Each mini-batch contains 64 visible light images and 64 infrared light images. Data augmentation is used, including channel augmentation, random cropping, random horizontal flipping, and random erasure. The initial learning rate is 3.5*10-4, which decays to 0.1 every 20 epochs. The model is trained for a total of 50 epochs.

[0036] During the testing process, the images in the query set and gallery set of the dataset were first scaled to a specified size of 384*128. Then, the images were fed into a trained model network with training weights to obtain the image features of the query set and gallery set. For each feature in the query set, its similarity with each feature in the gallery set was calculated. Based on the similarity, Rank-1, Rank-5, Rank-10, etc. were obtained. The pedestrian with the most similarity is the pedestrian represented by the feature corresponding to Rank-1.

[0037] Table 1 shows a comparison of the experimental results of this invention with other methods on the SYSU-MM01 dataset, and Table 2 shows a comparison of the experimental results of this invention with other methods on the RegDB dataset.

[0038] Table 1. Comparison of experimental results with other methods on the SYSU-MM01 dataset. For the SYSU-MM01 dataset, the results achieved in full search mode are 64.96, 62.41, and 49.03 on Rank-1, mAP, and mINP, respectively, and in indoor search mode, the results achieved are 70.91, 76.01, and 72.61 on Rank-1, mAP, and mINP, respectively.

[0039] Table 2 Comparison of experimental results with other methods on the RegDB dataset For the RegDB dataset, the results achieved in full search mode are 93.13, 86.69, and 73.23 on Rank-1, mAP, and mINP, respectively, and in indoor search mode, the results achieved are 92.65, 85.61, and 71.02 on Rank-1, mAP, and mINP, respectively.

[0040] To highlight the advantages, this invention compares them by incorporating its own solution into a benchmark model; that is, it uses both the benchmark model and its own model to retrieve Query images from the Gallery collection. The results are as follows: Figure 3 As shown, incorporating the method of this invention into the baseline model yields more accurate results.

[0041] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An unsupervised visible-infrared pedestrian re-identification method based on contrastive learning, characterized in that, Includes the following steps: S1. Obtain raw visible light and raw infrared light images of pedestrians and construct a training dataset; S2. Construct a pedestrian re-identification model. The pedestrian re-identification model adopts the Transformer-based ReID model as the baseline model. The training dataset is input into the pedestrian re-identification model, and the loss function is used to train the pedestrian re-identification model to obtain the trained pedestrian re-identification model. S3. Input the images of the pedestrian to be identified and the known pedestrians into the trained pedestrian re-identification model to extract deep features, calculate the similarity between the deep features of the pedestrian to be identified and the deep features of the known pedestrians, sort them in descending order of similarity to obtain a sorted list of the deep features of the known pedestrians, obtain the k most similar deep features of the pedestrians, and finally return the images corresponding to the k deep features of the pedestrians in order.

2. The unsupervised visible-infrared pedestrian re-identification method based on contrastive learning according to claim 1, characterized in that, In step S2, the baseline model specifically includes: The original visible light image is input into the baseline model for feature extraction, resulting in shallow and deep features of the original visible light image. The Transformer-based ReID model consists of 12 stacked transformer blocks, each including an attention mechanism, residual connections, normalization, and a fully connected layer. The original visible light image passes through the first transformer block to obtain shallow features, and through the 12 transformer blocks to obtain deep features. Similarly, the original infrared photo is input into the baseline model for feature extraction to obtain the shallow and deep features of the original infrared photo.

3. The unsupervised visible-infrared pedestrian re-identification method based on contrastive learning according to claim 2, characterized in that, In step S2, the shallow and deep features of the original visible light image and the shallow and deep features of the original infrared image are clustered to generate pseudo-labels, and shallow clustering prototype memory and deep clustering prototype memory are constructed respectively; clustering-level contrastive loss is used for comparative learning of the baseline model.

4. The unsupervised visible-infrared pedestrian re-identification method based on contrastive learning according to claim 3, characterized in that, Constructing shallow clustering prototype memories and deep clustering prototype memories: For the visible light mode, the shallow and deep features of the original visible light image are concatenated to obtain a concatenated vector. This concatenated vector is then used as a clustering feature to perform clustering operations using the DBSCAN clustering algorithm, resulting in pseudo-labels for the visible light image. Based on these pseudo-labels, shallow and deep clustering prototype memories for the visible light mode are constructed. Each cluster prototype is the average vector of the features of all samples within that cluster, and the resulting average vector is the prototype feature of each pseudo-label. Similarly, for the infrared light mode, the shallow and deep features of the original infrared light image are processed using the same method to obtain shallow and deep clustering prototype memories for the infrared light mode.

5. The unsupervised visible-infrared pedestrian re-identification method based on contrastive learning according to claim 3, characterized in that, In step S2, the baseline loss is constructed: Cluster-level contrastive loss functions are used to perform cluster-level contrastive learning on shallow and deep features within each modality. The cluster-level contrastive loss for shallow features is: ， in, , Represents visible light modes, Indicates infrared light mode; Indicates the first Shallow features of each sample; Indicates the first Shallow prototype features of each cluster; This indicates the prototype of the cluster to which the current sample belongs; τ is the temperature parameter. The cluster-level contrastive loss for deep features takes the same form as that for shallow features. The centroid features in the shallow clustering prototype memory and the deep clustering prototype memory are updated using momentum, as follows: ， in, Indicates the first The prototype memory of the next iteration A centroid feature, Represents the number of training iterations. Indicates the momentum coefficient. This indicates that it belongs to a pseudo tag. Instance features; baseline loss is = + + + ,in, Cluster-level contrast loss representing shallow features of visible light modes, Cluster-level contrast loss representing shallow features of infrared light modes, Cluster-level contrast loss representing deep features of visible light modes, Cluster-level contrast loss representing deep features of infrared light modes.

6. The unsupervised visible-infrared pedestrian re-identification method based on contrastive learning according to claim 1, characterized in that, In step S2, the pedestrian re-identification model further includes a modality obfuscator module: The modality confusion module includes a binary modality classifier and a gradient inversion layer; the modality label of the original visible light image is set to 1, and the modality label of the original infrared light image is set to 0; the deep features of the original visible light image and the original infrared light image are input into the binary modality classifier, and the output result is subjected to cross-entropy loss with the modality label; Loss function of binary modality classifier for: ,in, This represents the batch stitching of deep features from the original visible light image and the original infrared image. Indicates modal label; This represents a binary modal classifier; Represents the normalized exponential function, Represents cross-entropy loss; loss function of gradient inversion layer for: ,in, This indicates a gradient inversion layer; The final loss of the modal confused module is obtained. , .

7. The unsupervised visible-infrared pedestrian re-identification method based on contrastive learning according to claim 1, characterized in that, In step S2, the pedestrian re-identification model further includes a dual-scale interactive neighbor learning module: For the original visible light image, calculate the cosine similarity between shallow features, the cosine similarity between deep features, the cosine similarity between shallow features of the original visible light image and shallow features of the original infrared image, and the cosine similarity between deep features of the original visible light image and deep features of the original infrared image. Intramodal fusion similarity is constructed from the original visible light image. ; Construct fusion similarity between the original visible light image and the original infrared image. For the first An original visible light photograph Based on fusion similarity, a threshold is used to construct the intra-modal neighbor set of the original visible light image. and modal neighbor set ; Using in-modal neighbor sets and modal neighbor set Calculate the instance-level contrastive loss function for deep features within the visible light mode. Instance-level contrastive loss function for shallow features between visible light modes Instance-level contrastive loss function for shallow features within the visible light mode and the instance-level contrastive loss function for deep features between visible light modes The formula for the instance-level contrastive loss function for deep features within the visible light mode is as follows: ， in, Indicates the number of instances in the current batch. The exponential function is represented; the other three loss functions have the same form as the instance-level comparison loss function for deep features in the visible light modality. Similarly, for the original infrared image, the instance-level contrast loss function of deep features within the infrared mode is calculated. Instance-level contrast loss function for shallow features between infrared light modes Infrared light mode shallow feature instance-level contrast loss function and the deep feature instance-level contrast loss function between infrared light modes ; Finally, the two-scale interactive neighbor learning loss is obtained. , .

8. The unsupervised visible-infrared pedestrian re-identification method based on contrastive learning according to claim 4, characterized in that, In step S2, the pedestrian re-identification model further includes a cross-modal relationship correction module: The Hungarian algorithm is used to process the visible light shallow-layer prototype features and the infrared light shallow-layer prototype features to obtain the correspondence between the visible light shallow-layer prototype features and the infrared light shallow-layer prototype features. Similarly, the correspondence between visible light deep prototype features and infrared light deep prototype features is obtained. Correspondence between infrared shallow prototype features and visible light shallow prototype features And the correspondence between infrared deep prototype features and visible light deep prototype features. ; The correspondence between visible light shallow-layer prototype features and infrared light shallow-layer prototype features. Correspondence between visible light deep prototype features and infrared light deep prototype features Inconsistent correspondences are extracted, and the inconsistencies are corrected by calculating prototype fusion similarity. The visible light deep prototype features and infrared deep prototype features corresponding to the maximum prototype fusion similarity are mapped to obtain the corrected correspondences. The consistent correspondences and the corrected correspondences are then merged to obtain the visible light to infrared light matching relationship. ; Similarly, the correspondence between shallow infrared prototype features and shallow visible light prototype features can be established. Correspondence between infrared deep prototype features and visible light deep prototype features The matching relationship between infrared and visible light was obtained. ; Constructing cross-modal clustering-level contrastive loss , ; Calculate the deep loss between the visible light instance and the infrared light prototype. The formula is expressed as follows: ， in, Indicates the batch size of samples loaded into the dataset. Indicates pseudo-tags as The deep prototype features of infrared light corresponding to visible light. This represents the number of deep prototype features in infrared light; similarly, the shallow loss between visible light instances and infrared prototypes is obtained. Shallow loss between infrared light instances and visible light prototypes Deep loss between infrared light instances and visible light prototypes .

9. The unsupervised visible-infrared pedestrian re-identification method based on contrastive learning according to claim 8, characterized in that, In step S2, the specific process for calculating the prototype fusion similarity is as follows: calculate the visible light deep prototype features. Infrared deep prototype features cosine similarity Visible light shallow prototype characteristics Similar to shallow prototype features of infrared light cosine similarity ,Will and The prototype fusion similarity is obtained by performing fusion. , , This represents the second balance coefficient.

10. The unsupervised visible-infrared pedestrian re-identification method based on contrastive learning according to claim 1, characterized in that, The loss function uses the joint optimization total loss function. The formula is expressed as follows: in, Indicates baseline loss. This indicates the loss of the modal confused module. This represents the loss from two-scale interactive neighbor learning. This represents the cross-modal clustering level contrastive loss. This represents the third balance coefficient.

Images