Domain generalization method for person re-identification without paired samples

Abstract

The present invention provides a domain generalization method for pedestrian re-identification based on unpaired samples, and belongs to the field of image retrieval technology in computer vision. The present invention includes: obtaining a data set; learning a pedestrian identity prototype; using the pedestrian identity prototype, through a comparative learning method, learning identity-related prompts to make up for the missing cross-camera positive samples; introducing a learnable perturbation prompt, injecting perturbation prompt information into the learnable identity-related prompt, forcing the image encoder to perform poorly to update the learnable perturbation prompt; using the data of missing cross-camera positive samples and the identity-related prompts with perturbation prompt information added, by updating the parameters of the image encoder, to enhance the image encoder's ability to extract consistent features; using two objective evaluation indicators, cumulative matching curve CMC and mean average accuracy mAP, to evaluate the retrieval results of unknown domains. Through this method, the applicability and generalization ability of the model in real scenes are significantly improved.

Description

Technical Field

[0001] The present invention provides a pedestrian re-identification domain generalization method based on unpaired samples, and relates to the technical field of image retrieval in computer vision. Background Art

[0002] Person re-identification is a technology used to determine whether pedestrians in non-overlapping camera views are of the same identity. This technology has received widespread attention from academia and industry due to its potential applications in areas such as finding missing persons, and has spawned many efficient solutions. Initially, these solutions were mainly obtained through supervised training. However, the generalization ability of this type of method is weak, and ideal recognition results can only be achieved when training samples in the target domain participate in model training in a supervised learning manner. In order to overcome this limitation of supervised person re-identification models, researchers began to focus on person re-ID methods based on domain adaptation. However, this method requires that the model must be fine-tuned unsupervisedly based on the training samples in the target data, which limits the model's ability to be directly applied to unknown datasets.

[0003] Although existing domain-generalized person re-ID methods have achieved remarkable results, they all rely on supervised training in the source domain, which requires a large number of cross-camera paired training samples to be labeled. However, since pedestrian appearance is easily affected by the surrounding environment, pedestrian appearance under different cameras shows great differences, which makes it extremely challenging for machines to automatically and accurately label pedestrian samples across cameras. To address this problem, it is usually necessary to manually label a large number of training samples, which is not only laborious but also time-consuming. In addition, in open scenes, the probability of the same pedestrian appearing under different cameras is not high, especially when the distance between cameras is large, which further limits the number of samples of the same pedestrian across cameras that can be used for model training. Therefore, if the domain-generalized person re-ID model can be independent of labeled samples in the source domain, it will be of great practical significance. Unfortunately, this problem has not yet attracted enough attention from researchers. In reality, pedestrian samples under a single camera are relatively easy to obtain. If it is possible to use these pedestrian samples of a single identity under different cameras (i.e., there is no positive sample correspondence across cameras) to train a domain-generalized model, then the above problems will be fundamentally solved.

[0004] When the same pedestrian is photographed across cameras, their appearance may change significantly due to factors such as viewing angle and lighting, which may result in similar appearances between different pedestrians. That is, pedestrians who look dissimilar may actually be the same person, while pedestrians who look similar may belong to different individuals. This ambiguity is the difficulty in labeling cross-camera training samples. If a model training method can be developed so that model performance improvement does not rely on labeled cross-camera sample pairs, it will greatly reduce the human resources consumed by sample labeling. Summary of the invention

[0005] In response to the above problems, the present invention provides a pedestrian re-identification domain generalization method based on unpaired samples. This method breaks the traditional method's dependence on cross-camera positive samples. Through this method, not only the training difficulties caused by the lack of cross-camera positive samples are effectively compensated, but also the diversity of pedestrians' appearance features under multiple perspectives is successfully simulated, significantly improving the applicability and generalization ability of the model in real scenarios.

[0006] The technical solution of the present invention is: a domain generalization method for pedestrian re-identification without paired samples, comprising the following steps:

[0007] Step 1: Obtain the data set: obtain the data with missing positive samples across cameras, use it as a training set, and obtain a test set;

[0008] Step 2: Learn the pedestrian identity prototype: Use the training set image data divided in step 1 for preprocessing; input the preprocessed missing cross-camera positive sample data set into the image encoder with fixed parameters to extract features for training the identity classifier, thereby obtaining the pedestrian identity prototype;

[0009] Step 3: Learn identity-related cues: Using the pedestrian identity prototype in step 2, learn identity-related cues through contrastive learning to make up for the missing cross-camera positive samples;

[0010] Step 4: Generate perturbation prompts: introduce a learnable perturbation prompt, inject the perturbation prompt information into the learnable identity-related prompts, and force the image encoder performance to degrade to update the learnable perturbation prompts;

[0011] Step 5: Consistent feature learning: Using the data of the missing positive samples across cameras in step 1 and the identity-related prompts added with the perturbation prompt information in step 4, the parameters of the image encoder are updated to enhance the image encoder's ability to extract consistent features, thereby obtaining a trained image encoder.

[0012] Step 6: Evaluate the retrieval results of the unknown domain: Test the trained image encoder on the test set to obtain two objective evaluation indicators, the cumulative matching curve (CMC) and the average precision (mAP), to evaluate the retrieval results of the unknown domain.

[0013] As a further solution of the present invention, in step 1, obtaining the data set includes dividing it into a training set and a test set; a data set lacking cross-camera positive samples, that is, an identity can only appear under one camera, is used as a training set; and a test set is a data set with cross-camera positive samples, in which the identity labels of the training set and the test set do not overlap at all.

[0014] As a further solution of the present invention, in step 2, the image encoder includes the following processing: the image encoder cuts the input image into multiple blocks of fixed size through image segmentation, and then converts the blocks into high-dimensional feature vectors; adds position encoding to the high-dimensional feature vector of each image block, and the processed image blocks are then processed by a multi-layer Transformer encoder, each layer of which includes a self-attention mechanism and a feedforward network for integrating the information of all image blocks to obtain the global features of the pedestrian image.

[0015] As a further solution of the present invention, in step 2, learning the pedestrian identity prototype includes:

[0016] The global features of the pedestrian image are obtained through the image encoder, and the global features are used to train the identity classifier W. When the parameters in the identity classifier W are updated using the sample global features, the parameters of the image encoder E are fixed, and each vector in the identity classifier W is They are all used as corresponding identity prototypes, and the following identity loss is used to update the parameters in W to learn the pedestrian identity prototype:

[0017] ;

[0018] Among them, E represents the image encoder, W is the identity classifier, and B represents the number of batches. represents the pedestrian label, E( ) represents the global features of pedestrian images, represents the i-th pedestrian image, and Ce represents the cross entropy loss.

[0019] As a further solution of the present invention, in step 3, the learning identity related prompt includes:

[0020] Use pedestrian identity prototypes to learn identity-related prompts; assuming there are M pedestrian identities, the corresponding pedestrian identity prototype is ; The pedestrian identity related prompts learned from the pedestrian identity prototype are expressed as , The features obtained by the image encoder E with fixed parameters To be able to Keeping the same identity, the following softmax loss is used to constrain learning:

[0021] ;

[0022] in, Indicates the calculation of the i-th identity related prompt and the i-th pedestrian identity prototype The cosine similarity between them is minimized by the above loss function to ensure the pedestrian identity related prompts The result obtained after the image encoder E is the feature of identity-related prompts and have the same identity, but also guarantee It is different from the samples of the corresponding identity in the source domain.

[0023] As a further solution of the present invention, in step 4, generating a disturbance prompt includes:

[0024] Assume that the hint related to the i-th identity The corresponding disturbance is , the identity-related prompt result after adding the perturbation is expressed as ; The perturbation hint is obtained by minimizing the following loss :

[0025] ;

[0026] Among them, E represents the image encoder, W is the identity classifier, and B represents the number of batches. represents the pedestrian label, and Ce represents the cross entropy loss.

[0027] As a further solution of the present invention, in step 5, learning the consistency feature includes:

[0028] Added identity related hints after perturbation Used as pedestrian image Pedestrian samples under unknown camera perspectives constitute cross-perspective pedestrian sample pairs , using this sample pair, the image encoder E is given the ability to mine consistent discriminative features across perspectives. To achieve this goal, the loss function in the following formula is used to update the re-ID model, i.e., the image encoder E:

[0029] ;

[0030] Among them, E represents the image encoder, W is the identity classifier, and B represents the number of batches. represents the pedestrian label, Ce represents the cross entropy loss;

[0031] During sample collection, a camera classifier is learned by minimizing the classifier in the following formula: :

[0032] ;

[0033] in, Represents a camera tag.

[0034] As a further solution of the present invention, in step 2, preprocessing the training set image data includes: performing random cropping, random erasing, and flipping data enhancement operations on the data set.

[0035] As a further solution of the present invention, step 6 includes:

[0036] First, the test data is preprocessed and adjusted to the size of the training data; it is input into the trained image encoder, i.e., the retrieval model, to obtain the final retrieval features, and the similarity is calculated and sorted; the sorting results are evaluated using two objective evaluation indicators, the cumulative matching curve (CMC) and the mean average accuracy (mAP).

[0037] The beneficial effects of the present invention are:

[0038] 1. This paper proposes a novel idea, that is, in the absence of positive samples across cameras, the correspondence between the pedestrian identity classifier and the identity prototype is used to predict the missing positive samples. This idea breaks the reliance of traditional methods on positive samples across cameras and provides a new idea for the model to extract features without positive samples;

[0039] 2. This paper proposes to use the discriminant standard features corresponding to the identity in the pedestrian identity classifier as identity prototypes to guide the generation of cross-camera identity related prompts. During the model training process, these prompts effectively make up for the missing cross-camera positive samples, so that the model learns how to extract cross-camera identity consistency features;

[0040] 3. In order to further enhance the model's ability to extract discriminative features under different camera perspectives, the present invention also introduces a learnable perturbation prompt. This prompt can simulate the diversity of pedestrians' appearances caused by environmental changes under different camera perspectives. When the perturbation prompt information is injected, the model achieves an effective improvement in generalization performance under the witness of the pedestrian identity classifier;

[0041] 4. By introducing identity-related cues and perturbation cues, the present invention not only effectively compensates for the training difficulties caused by the lack of positive samples across cameras, but also successfully simulates the diversity of pedestrians' appearance features under multiple perspectives. This innovative method significantly improves the generalization ability of the model and further promotes the practical application of pedestrian re-identification technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is the overall flow chart of the present invention;

[0043] Figure 2 A schematic diagram of a prototype for learning pedestrian identities in the present invention;

[0044] Figure 3 A schematic diagram of identity-related prompt learning for the present invention;

[0045] Figure 4 This is a schematic diagram of disturbance prompt learning of the present invention;

[0046] Figure 5 This is a schematic diagram of learning the consistency features of the present invention;

[0047] Figure 6 The comparison results of the top 10 M_SCT→MS search results between the proposed method and the baseline; the red box represents the wrong search result, and the green box represents the correct search result; Figure 6 (a) represents the results retrieved by the baseline. Figure 6 (b) indicates the search results of the present invention;

[0048] Figure 7 This is the attention heat map of the present invention; Figure 7 (a) and Figure 7 (d) represents the original image, Figure 7 (b) and Figure 7 (e) represents the area that the Baseline focuses on. Figure 7 (c) and Figure 7 (f) represents the region of interest of the method of the present invention;

[0049] Figure 8 This is a visualization effect diagram of the characteristic distribution of the present invention; Figure 8 (a) represents the initialized domain distribution, Figure 8 (b) Domain distribution representing the baseline, Figure 8 (c) represents the domain distribution of our method. DETAILED DESCRIPTION

[0050] In order to more effectively illustrate the technical method of the present invention, we will describe the specific implementation method in detail in conjunction with the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. It is only an example of a system and method consistent with some aspects of the present invention described in the claims.

[0051] In order to change the traditional focus, we no longer strive to construct a training set by accurately determining whether pedestrians under different cameras are the same person. Instead, the present invention adopts a new assumption: pedestrians appearing in different cameras are regarded as having different identities. In realistic open scenes, especially when the distance between cameras is far, this assumption is valid in most cases due to the limited range of pedestrian activities. Based on this assumption, we no longer need to laboriously label a large number of training samples, but can directly determine that pedestrians under different cameras have independent identities. In order to train a pedestrian re-identification model that is both effective and domain generalized under this new assumption, an innovative training method is proposed that does not rely on positive sample pairs across cameras.

[0052] Based on the generalization method of pedestrian re-identification domain without paired samples, this paper proposes to use the discriminant standard features corresponding to the identity in the pedestrian identity classifier as the identity prototype to guide the generation of cross-camera identity-related prompts. During the model training process, these prompts effectively make up for the missing cross-camera positive samples, so that the model learns how to extract cross-camera identity consistency features. In addition, in order to further enhance the model's ability to extract discriminative features under different camera perspectives, we also specially introduce a learnable perturbation prompt. This prompt can simulate the appearance diversity of pedestrians caused by environmental changes under different camera perspectives. When the perturbation prompt information is injected, the model achieves an effective improvement in generalization performance under the witness of the pedestrian identity classifier. This method combines identity-related prompts and learnable perturbation prompts. While improving the model's ability to extract discriminative features across cameras, it also enhances the model's adaptability to complex and changeable environments in the real world, making its pedestrian identity recognition in real scenes more accurate and reliable.

[0053] like Figure 1-Figure 5 As shown, the present invention provides a pedestrian re-identification domain generalization method based on unpaired samples, comprising the following steps:

[0054] Step 1. Obtain the data set: obtain data with missing cross-camera positive samples, use it as the training set, and obtain the test set; the data set with missing cross-camera positive samples, that is, an identity can only appear under one camera; the test set is a data set with cross-camera positive samples, in which the identity labels of the training set and the test set do not overlap at all.

[0055] Step 2: Learn the pedestrian identity prototype: Use the training set image data divided in step 1 for preprocessing; input the preprocessed missing cross-camera positive sample data set into the image encoder with fixed parameters to extract features for training the identity classifier, thereby obtaining the pedestrian identity prototype;

[0056] The image encoder includes the following processing: the image encoder cuts the input image into multiple fixed-size blocks through image segmentation, and then converts these blocks into high-dimensional feature vectors; then, in order to preserve the spatial relationship, position encoding is added to the high-dimensional feature vector of each image block, and these processed image blocks are then processed by a multi-layer Transformer encoder, each layer of which contains a self-attention mechanism and a feedforward network to integrate the information of all image blocks and obtain the global features of pedestrian images. Preprocessing the training set image data includes: random cropping, random erasing, and flipping data enhancement operations on the data set.

[0057] In step 2, learning the pedestrian identity prototype includes:

[0058] The global features of pedestrian images are obtained through the image encoder, and the global features are used to train the identity classifier W. When the parameters in the identity classifier W are updated using the global features of the samples, the image encoder E is fixed. The parameters of the image encoder E are fixed, which helps to give strong high-level semantic information to the pedestrian identity prototype in W. If the identity classifier W can correctly classify pedestrian samples, it means that each vector in the identity classifier W is The identity information of pedestrians is integrated, and this information is valid for the samples of the same pedestrian under the input single camera. The parameters of the image encoder E are fixed, and each vector in the identity classifier W is They are all used as corresponding identity prototypes, and the following identity loss is used to update the parameters in W to learn the pedestrian identity prototype:

[0059] ;

[0060] Among them, E represents the image encoder, W is the identity classifier, and B represents the number of batches. represents the pedestrian label, E( ) represents the global features of pedestrian images, represents the i-th pedestrian image, and Ce represents the cross entropy loss.

[0061] Step 3: Learn identity-related cues: Using the pedestrian identity prototype in step 2, learn identity-related cues through contrastive learning to make up for the missing cross-camera positive samples;

[0062] As a further solution of the present invention, in step 3, the learning identity related prompt includes:

[0063] Use pedestrian identity prototypes to learn identity-related prompts; assuming there are M pedestrian identities, the corresponding pedestrian identity prototype is ; The pedestrian identity related prompts learned from the pedestrian identity prototype are expressed as , The features obtained by the image encoder E with fixed parameters To be able to Keeping the same identity, the following softmax loss is used to constrain learning:

[0064] ;

[0065] in, Indicates the calculation of the i-th identity related prompt and the i-th pedestrian identity prototype The cosine similarity between them is minimized by the above loss function to ensure the pedestrian identity related prompts The result obtained after the image encoder E is the feature of identity-related prompts and have the same identity, but also guarantee This is because the encoder E only learns from the input hints Extraction and The corresponding identity information cannot be guaranteed This feature is exactly the same as the pedestrian images used in training. This makes it possible to replace the missing cross-camera positive samples to effectively update the model.

[0066] Step 4: Generate perturbation prompts: introduce a learnable perturbation prompt, inject the perturbation prompt information into the learnable identity-related prompts, and force the image encoder performance to degrade to update the learnable perturbation prompts;

[0067] As a further solution of the present invention, in step 4, generating a disturbance prompt includes:

[0068] Assume that the hint related to the i-th identity The corresponding disturbance is , the identity-related prompt result after adding the perturbation is expressed as To make It can simulate the changes in the appearance of pedestrians under different cameras. The disturbance prompts generated can make the previous image encoder E lose its due effect. This simulates the fluctuations in the appearance of pedestrians caused by changes in the imaging environment under different camera perspectives. The disturbance prompts are obtained by minimizing the following losses :

[0069] ;

[0070] Among them, E represents the image encoder, W is the identity classifier, and B represents the number of batches. represents the pedestrian label, and Ce represents the cross entropy loss.

[0071] Step 5: Consistent feature learning: Using the data of the missing positive samples across cameras in step 1 and the identity-related prompts added with the perturbation prompt information in step 4, the parameters of the image encoder are updated to enhance the image encoder's ability to extract consistent features, thereby obtaining a trained image encoder.

[0072] As a further solution of the present invention, in step 5, learning the consistency feature includes:

[0073] Added identity related hints after perturbation Used as pedestrian image Pedestrian samples under unknown camera perspectives constitute cross-perspective pedestrian sample pairs , using this sample pair, the image encoder E is given the ability to mine consistent discriminative features across perspectives. To achieve this goal, the loss function in the following formula is used to update the re-ID model, i.e., the image encoder E:

[0074] ;

[0075] Among them, E represents the image encoder, W is the identity classifier, and B represents the number of batches. represents the pedestrian label, Ce represents the cross entropy loss;

[0076] During sample collection, a camera classifier is learned by minimizing the classifier in the following formula: :

[0077] ;

[0078] in, Represents a camera tag.

[0079] Step 6: Evaluate the retrieval results of the unknown domain: Test the trained image encoder on the test set to obtain two objective evaluation indicators, the cumulative matching curve CMC and the mean average accuracy mAP, to evaluate the retrieval results of the unknown domain. Step 6 includes:

[0080] First, the test data is preprocessed and adjusted to the size of the training data; it is input into the trained image encoder, i.e., the retrieval model, to obtain the final retrieval features, and the similarity is calculated and sorted; the sorting results are evaluated using two objective evaluation indicators, the cumulative matching curve (CMC) and the mean average accuracy (mAP).

[0081] In order to illustrate the effect of the present invention, the present invention has done the following experiments for verification;

[0082] The experiments used five datasets, Market-1501 (M), Market-SCT (M-SCT), MSMT17 (MSMT), MSMT17-SCT (MSMT-SCT) and CUHK03-NP (C3-NP), to verify the effectiveness of our proposed method. Among them, the M-SCT and MSMT-SCT datasets are designed for cross-camera pedestrian re-identification tasks in cross-regional scenarios. These datasets are derived from the M and MSMT datasets, respectively, and are modified to adapt to cross-regional scenarios. In these two datasets, the identities of pedestrians across cameras do not overlap, and the identities under each camera are known. Specific information can be obtained from Table 1.

[0083] Table 1 shows the details of the dataset used.

[0084]

[0085] In order to illustrate the effect of the present invention, the present invention uses three different paradigms to prove that our method can obtain strong robustness for training the model with missing cross-camera positive samples.

[0086] In the first paradigm, we use M-SCT and MSMT-SCT as source domains to train person Re-ID models, and deploy the trained models to unknown target datasets for testing. The purpose of this scheme is to prove that our method can maintain good generalization even when there are no cross-shot positive samples involved in model training. The models trained by the proposed method are compared with existing domain generalization (DG) models, including CBN, QAConv, IL, OSNet-IBN, OSNet-AIN, PAT, etc. These models are trained on M_SCT and MS_SCT datasets, respectively, and tested on different datasets. We used the code provided by the original authors and kept the original parameter settings to retrain these models. From the results in the table, our model shows relatively superior test performance on the target domain in the dataset setting without cross-camera positive samples. As shown in Table 2, our method leads the second-best performance by 12.5% ​​and 14.5% on the "average" of map and rank-1, respectively. Table 2 shows the performance comparison under the first paradigm, where ‘a→b’ means ‘a’ is the source domain and ‘b’ is the unseen target domain; * represents the experimental results of retraining under Protocol-1.

[0087] Table 2 shows the performance comparison under the first paradigm

[0088]

[0089] In the second paradigm, we merge the cross-camera positive samples from the C3-NP training set with the training samples of M-SCT and MSMT-SCT, respectively, to create new source domains of M-SCT+C3-NP and MSMT-SCT+C3-NP. Next, we train our model using these new source domain data as well as MSMT-mixSCT. This protocol aims to verify that our method can still train a Re-ID model with excellent generalization performance even in the presence of cross-camera positive sample pairs and cross-camera samples without corresponding positive samples. As shown in Table 3, in the presence of a small number of positive samples, the model trained by our method shows relatively good test performance on the target domain. In M_SCT+C3→MS, our method is 6.7% and 10.8% higher than PAT in map and rank-1, respectively. In MS_SCT+C3→M, our method is 13.0% and 2.9% higher than QAConv-GS in map and rank-1, respectively. Table 3 shows the performance comparison under the second paradigm, where ‘a→b’ means ‘a’ is the source domain and ‘b’ is the unseen target domain, and * represents the experimental results of retraining under Protocol-2.

[0090] Table 3 shows the performance comparison under the second paradigm

[0091]

[0092] In the third paradigm, we selected two datasets from the pool containing M, MSMT and C3-NP as source domains to train the Re-ID model, and tested it on other unseen datasets. The purpose of this protocol is to evaluate the generalization ability of the model on different datasets when there are cross-camera positive samples in the source domain samples. According to Table 4, by using multiple datasets containing cross-camera positive samples for joint training, the model trained by the proposed method exceeds other methods in most performance indicators. Although the rank-1 performance in Ms+c3→M is slightly lower than that of QAConv-GS, the model trained by the proposed method exceeds them in mAP. In M+C3→MS, our method leads the suboptimal performance by 8.9% and 6.3% in map and rank-1, respectively. Table 4 is the performance comparison under the third paradigm, where 'a→b' means 'a' is the source domain, 'b' is the unseen target domain, and '-' means there is no corresponding data;

[0093] Table 4 shows the performance comparison under the third paradigm

[0094]

[0095] To verify the effectiveness of each component of the present invention, we use the datasets M-SCT and MSMTSCT to train the model, which are particularly lacking in cross-camera positive samples, and test it on the target datasets MSMT, M and C3-NP. We selected a pre-trained CLIP model (using the ViT-B / 16 structure) as the baseline for our study. The method proposed in this paper mainly includes three core parts: the construction of identity-related prompts (IRP), the generation of perturbation prompts (PP), and consistent feature learning (CFL). In subsequent experiments, we added these three core components to the baseline model in turn and verified their effectiveness through comparative analysis.

[0096] Validity of IRP:

[0097] We use each vector in the pedestrian classifier W As a pedestrian prototype, identity-related hints are constructed to make up for the problem of missing positive samples across cameras. As can be seen from Table 5, the performance of Backbone + IRP is 7.1% and 4.5% higher than that of Backbone in map and rank-1 on average, respectively.

[0098] Effectiveness of PP:

[0099] As shown in Table 5, PP also improves model performance. Compared with Backbone + IRP, the "average" map and rank-1 are improved by 0.3% and 1.1% respectively. Since the perturbation prompt simulates the fluctuation of pedestrian appearance characteristics caused by changes in the imaging environment under different camera perspectives, the improvement effect is different for different datasets. Adding perturbation prompts to MS_SCT→M improves map and rank-1 by 0.5% and 2.1% respectively compared with Backbone + IRP.

[0100] Effectiveness of CFL:

[0101] As shown in Table 5, CFL achieves significant improvement in model performance. Compared with Backbone + IRP + PP, the "average" map and rank-1 are improved by 3.9% and 4.7% respectively. This also proves that when the model is trained without cross-camera positive samples, it is easy to over-focus on the camera style information. Therefore, we introduce a camera classifier so that the pedestrian image encoder E can focus on extracting features related to pedestrian identity while ignoring camera-related effects. Table 5: Ablation results under M_SCT→MS, M_SCT→C, MS_SCT→M, MS_SCT→C settings, where IRP represents identity-related cues and PP represents perturbation cues;

[0102] Table 5

[0103]

[0104] Perform visual analysis:

[0105] In order to verify the visualization effect of the method proposed in this invention, Figure 6 , which represents the retrieval results of the baseline and the proposed method on M_SCT→MS. It can be seen that the proposed method significantly improves the retrieval performance under cross-domain conditions. The present invention also uses the heat attention map to visualize the model's attention to pedestrian areas. Figure 7 As shown in Figure 2, compared with the baseline model that only focuses on part of the pedestrian information, the present invention can cover most of the pedestrian information in the target domain. This shows that the present invention can effectively focus on domain-independent information and can be successfully generalized to the target domain. In addition, t-SNE is used to visualize the distribution of the learned features. Figure 8 As shown in Figure 2, the feature distribution of the baseline model in the target domain shows greater separability. This shows that the method of the present invention can effectively reduce the impact of the domain shift between the source domain and the target domain on the features, which is also the key factor for the success of the present invention.

[0106] The specific implementation modes of the present invention have been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A domain generalization method for person re-identification without paired samples, characterized by: The steps include: Step 1: Obtain the data set: obtain the data with missing positive samples across cameras, use it as a training set, and obtain a test set; Step 2: Learn pedestrian identity prototypes: Use the training set image data divided in step 1 for preprocessing; The preprocessed missing cross-camera positive sample dataset is input into an image encoder with fixed parameters to extract features for training the identity classifier to obtain the pedestrian identity prototype. Step 3: Learn identity-related cues: Using the pedestrian identity prototype in step 2, learn identity-related cues through contrastive learning to make up for the missing cross-camera positive samples; Step 4: Generate perturbation prompts: introduce a learnable perturbation prompt, inject the perturbation prompt information into the learnable identity-related prompts, and force the image encoder performance to degrade to update the learnable perturbation prompts; Step 5: Consistent feature learning: Using the data of the missing positive samples across cameras in step 1 and the identity-related prompts added with the perturbation prompt information in step 4, the parameters of the image encoder are updated to enhance the image encoder's ability to extract consistent features, thereby obtaining a trained image encoder. Step 6: Evaluate the retrieval results of the unknown domain: Test the trained image encoder on the test set and obtain two objective evaluation indicators, the cumulative matching curve (CMC) and the mean average precision (mAP), to evaluate the retrieval results of the unknown domain. In step 1, obtaining the data set includes dividing into a training set and a test set; a data set lacking positive samples across cameras, i.e., an identity can only appear under one camera, is used as a training set; a test set is a positive sample across cameras, where the identity labels of the training set and the test set do not overlap at all; In step 2, learning the pedestrian identity prototype includes: The global features of the pedestrian image are obtained through the image encoder, and the global features are used to train the identity classifier W. When the parameters in the identity classifier W are updated using the sample global features, the parameters of the image encoder E are fixed, and each vector in the identity classifier W is They are all used as corresponding identity prototypes, and the following identity loss is used to update the parameters in W to learn the pedestrian identity prototype: ； Among them, E represents the image encoder, W is the identity classifier, and B represents the number of batches. represents the pedestrian label, E( ) represents the global features of pedestrian images, represents the i-th pedestrian image, Ce represents the cross entropy loss; In step 3, the learning identity related prompts include: Use pedestrian identity prototypes to learn identity-related prompts; assuming there are M pedestrian identities, the corresponding pedestrian identity prototype is ; The pedestrian identity related prompts learned from the pedestrian identity prototype are expressed as , The features obtained by the image encoder E with fixed parameters To be able to Keeping the same identity, the following softmax loss is used to constrain learning: ； in, Indicates the calculation of the i-th identity related prompt and the i-th pedestrian identity prototype The cosine similarity between them is minimized by the above loss function to ensure the pedestrian identity related prompts The result obtained after the image encoder E is the feature of identity-related prompts and have the same identity, but also guarantee There is a difference between the samples with the corresponding identity in the source domain; In step 5, learning the consistency feature includes: Added identity related hints after perturbation Used as pedestrian image Pedestrian samples under unknown camera perspectives constitute cross-perspective pedestrian sample pairs , using this sample pair, the image encoder E is given the ability to mine consistent discriminative features across perspectives. To achieve this goal, the loss function in the following formula is used to update the re-ID model, i.e., the image encoder E: ； Among them, E represents the image encoder, W is the identity classifier, and B represents the number of batches. represents the pedestrian label, Ce represents the cross entropy loss; During sample collection, a camera classifier is learned by minimizing the classifier in the following formula: : ； in, Represents a camera tag.

2. The method for domain generalization of person re-identification based on unpaired samples according to claim 1, characterized in that: In step 2, the image encoder includes the following processing: the image encoder cuts the input image into multiple blocks of fixed size by image segmentation, and then converts the blocks into high-dimensional feature vectors; adds position encoding to the high-dimensional feature vector of each image block, and the processed image blocks are then processed by a multi-layer Transformer encoder, each layer of which includes a self-attention mechanism and a feedforward network for integrating the information of all image blocks to obtain the global features of the pedestrian image.

3. The method for domain generalization of person re-identification based on unpaired samples according to claim 1, characterized in that: In step 4, generating a disturbance prompt includes: Assume that the hint related to the i-th identity The corresponding disturbance is , the identity-related prompt result after adding the perturbation is expressed as ; The perturbation hint is obtained by minimizing the following loss : ； Among them, E represents the image encoder, W is the identity classifier, and B represents the number of batches. represents the pedestrian label, and Ce represents the cross entropy loss.

4. The method for domain generalization of person re-identification based on unpaired samples according to claim 1, characterized in that: In step 2, preprocessing the training set image data includes: performing random cropping, random erasing, and flipping data enhancement operations on the data set.

5. The method for domain generalization of person re-identification based on unpaired samples according to claim 1, characterized in that: The step 6 includes: First, the test data is preprocessed and adjusted to the size of the training data; it is input into the trained image encoder, i.e., the retrieval model, to obtain the final retrieval features, and the similarity is calculated and sorted; the sorting results are evaluated using two objective evaluation indicators, the cumulative matching curve (CMC) and the mean average accuracy (mAP).

Images