A generalizable pedestrian re-identification method based on visual language multi-granularity distillation

Abstract

The application discloses a generalizable pedestrian re-identification method based on visual language multi-granularity distillation. The application is established on a two-stage prompt learning process, and integrates global / local information of images and texts as multi-granularity features to solve the generalizable pedestrian re-identification task. In the application, a multi-granularity visibility control mechanism based on attention mask is proposed to aggregate multi-granularity information, so that the information is independently fused at each granularity level. At the same time, a prototype contrast loss function is proposed to supervise and optimize the learning process, further improving the performance of the generalizable pedestrian re-identification. A large number of experimental results prove that the application has superior performance on multiple public data sets in the field, reaching the current most advanced level.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology and relates to a generalizable pedestrian re-identification method, specifically a generalizable pedestrian re-identification method based on multi-granularity distillation of visual language. Background Technology

[0002] Pedestrian re-identification is a common task in surveillance scenarios, aiming to design an image encoder to extract the identity features of pedestrians appearing under different cameras, thereby matching pedestrians with the same identity across cameras. Currently, most pedestrian re-identification is implemented based on fully supervised deep learning-trained image encoders, but it suffers from the following problems:

[0003] 1. Poor generalization. The image encoder overfits to the training dataset and has poor ability to extract the identity features of pedestrians that have never been seen before in real-world scenarios, resulting in a high matching failure rate.

[0004] 2. High training cost. The training of image encoders relies on images with a large number of pedestrian identification labels. Labeling new, unseen scenes is difficult and costly in terms of time and money.

[0005] To address the above issues, scholars have proposed research on generalizable person re-identification methods, exploring how to use existing labeled datasets to train image encoders that can extract more generalizable features. Existing research on generalizable person re-identification generally focuses on the following aspects: 1) identity feature decoupling methods; 2) normalization methods; 3) hybrid expert methods; 4) meta-learning methods; 5) direct image matching methods; 6) unsupervised pre-training methods; 7) methods using additional multimodal datasets for training; and 8) frequency domain-based methods.

[0006] The aforementioned methods have effectively promoted the development of generalizable person re-identification tasks, but there is still considerable room for improvement. Currently, existing methods have three significant limitations:

[0007] 1. Use global features. Existing methods focus on extracting global features from pedestrian images, while ignoring local detailed features that are helpful in distinguishing different identities.

[0008] 2. Using a single visual modality. Most existing methods only use information from images for encoder training, which leads to the coupling of pedestrian identity features and irrelevant features (such as background, lighting, and viewpoint of the training dataset), causing the encoder to overfit the dataset and thus limiting generalization.

[0009] 3. Additional training data is required. Some recent works have considered using image-text pair datasets for pre-training to introduce multimodal information and improve generalization, but in practice, obtaining image-text pair datasets is difficult. Summary of the Invention

[0010] To further address the problems existing in generalizable person re-identification in the background technology, this invention proposes a generalizable person re-identification method based on visual-language multi-granularity distillation. This invention acquires global and local image information through a sliding window to obtain multi-granularity features; it utilizes a multi-granularity visibility control mechanism based on attention masks for multimodal global and local information interaction; considering that the generalization of language description can break down domain barriers between different datasets and avoid overfitting to the training dataset, this invention, based on multi-granularity features, employs a two-stage distillation with cue learning to inject the generalization knowledge of the visual-language model into the image encoder; for the two-stage distillation process, a multi-granularity prototype contrast loss (including cross-modal multi-granularity prototype contrast loss and intra-modal multi-granularity prototype contrast loss) is designed for training. Compared with other published methods, this invention significantly outperforms other generalizable person re-identification methods in multiple performance metrics.

[0011] The technical solution adopted in this invention is:

[0012] I. A generalizable pedestrian re-identification method based on multi-granularity distillation of visual language

[0013] 1) Construct a multi-granularity distillation feature extraction network model. The multi-granularity distillation feature extraction network model includes a sliding window module, a Transformer-structured text encoder, and a Transformer-structured image encoder. The sliding window module is connected to the image encoder, and the input of the text encoder contains trainable multi-granularity text tokens.

[0014] 2) Freeze the weights of the text encoder and image encoder, and use the pedestrian re-identification dataset to perform visual-to-language distillation on the multi-granularity distillation feature extraction network model to complete the optimization of the trainable multi-granularity text token weights and save their weights.

[0015] 3) Freeze the weights of the text encoder and the trainable multi-granularity text tokens, unfreeze the weights of the image encoder, use the person re-identification dataset to perform language-to-visual distillation on the multi-granularity distillation feature extraction network model, optimize the weights of the image encoder and save its weights, and obtain the trained image encoder.

[0016] 4) Input the image to be detected into the trained image encoder, extract the multi-granularity image features of the image, and then use the multi-granularity image features to perform pedestrian re-identification to obtain the pedestrian re-identification result.

[0017] In 1), the input of the text encoder also includes a global text token and a local text token;

[0018] The input image of the multi-granularity distillation feature extraction network model is input into the image encoder after passing through the sliding window module. The image encoder also includes a global image token and a local image token.

[0019] In step 1), for both text encoders and image encoders, token visibility is controlled by combining a mask during self-attention calculation, as shown in the following formula:

[0020]

[0021]

[0022] Where Q, K, and V represent the query, key, and value matrices in the self-attention mechanism, and D represents the feature dimension; softmax(·) represents the conversion of attention scores into probability scores, where attention scores with values ​​of negative infinity will result in a probability score of zero; M is the text attention mask in the text encoder or the image attention mask in the image encoder; M ij Let Vis(i) represent the element in the i-th row and j-th column of mask M, and Vis(i) represent the extraction of other tokens with the same granularity as the i-th token.

[0023] In step 2), when performing vision-to-language distillation on the multi-granularity distillation feature extraction network model using the pedestrian re-identification dataset, the loss function employed includes cross-modal multi-granularity prototype contrast loss. The formula is as follows:

[0024]

[0025] in, and Let represent the text features of the descriptive sentence representing the multi-granularity image feature center and the identity of the c-th pedestrian, respectively. Let represent the multi-granularity image feature center of the j-th pedestrian, C represent the total number of pedestrian identities, and s(·,·) represent the cosine similarity.

[0026] In step 3), when performing language-to-visual distillation on the multi-granularity distillation feature extraction network model using the pedestrian re-identification dataset, the loss function used for re-identification ability learning includes identity classification loss. and based on image prototype feature library Intramodal multi-granularity prototype contrast loss The loss function used for generalization ability learning includes cross-modal classification loss.

[0027] The image prototype feature library Intramodal multi-granularity prototype contrast loss The formula is as follows:

[0028]

[0029]

[0030] in, Represents multi-granularity image features, y i Let represent the category of the current i-th sample, τ be the temperature hyperparameter controlling the intensity of the similarity calculation, and s(·,·) represent the cosine similarity. The multi-granularity image prototype feature represents the category to which the current i-th sample belongs. γ represents the multi-granularity image prototype features of the j-th category; γ is the momentum hyperparameter controlling the intensity of feature library updates, v mg* It is the hardest sample in each pedestrian identity image sample in a training batch, and ← indicates iterative update.

[0031] The cross-modal classification loss The formula is as follows:

[0032]

[0033] in, Representing multi-granularity image features, This represents the multi-granularity text prototype for the c-th category. Let represent the multi-granularity text prototype of the j-th category, C represent the total number of pedestrian identities, s(·,·) represent the cosine similarity, and q c A soft tag representing the identity of the c-th pedestrian.

[0034] II. A computer device

[0035] The device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method.

[0036] III. A computer-readable storage medium

[0037] The medium stores a computer program that, when executed by a processor, implements the steps of the method.

[0038] IV. A computer program product

[0039] The product includes a computer program / instructions that, when executed by a processor, implement the steps of the method.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] This invention proposes a generalizable pedestrian re-identification method based on visual language multi-granularity distillation. The domain-invariant property of language modality is injected into the image encoder through a two-stage distillation process of cue learning, making the extracted pedestrian identity features more generalizable and achieving state-of-the-art performance in relevant metrics.

[0042] This invention achieves better performance on generalizable person re-identification tasks without requiring additional datasets for pre-training.

[0043] This invention demonstrates the potential of multimodal information and multigranular features in generalizable pedestrian re-identification tasks. Attached Figure Description

[0044] Figure 1 This is a flowchart of a method according to an embodiment of the present invention.

[0045] Figure 2 This is a model structure diagram of an embodiment of the present invention.

[0046] Figure 3 This is a schematic diagram of a multi-granularity visibility control mechanism based on a self-attention mask in an embodiment of the present invention. Detailed Implementation

[0047] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. The embodiments and specific implementation processes of the present invention are as follows, such as... Figure 1 As shown, the present invention includes the following steps:

[0048] like Figure 1 As shown, the present invention includes the following steps:

[0049] 1) Construct a multi-granularity distillation feature extraction network model. This model includes a sliding window module, a Transformer-structured text encoder, and a Transformer-structured image encoder. The sliding window module is connected to the image encoder. The text encoder's input includes trainable multi-granularity text tokens; it also includes trainable global and local text tokens. These trainable multi-granularity text tokens, combined with other fixed words, form descriptive sentences for pedestrians with different identities. After serialization, the descriptive sentences are input to the Transformer-structured text encoder, which aggregates global / local information into the global / local text tokens through a self-attention mechanism. The global and local text tokens output from the final layer of the text encoder are selected and concatenated as multi-granularity text features.

[0050] The input image of the multi-granularity distillation feature extraction network model is passed through a sliding window module before being input into the image encoder. The image encoder also includes trainable global image tokens and local image tokens. The sliding window module is a sliding window of size κ and step size δ, which divides the image sequence into three sub-regions (i.e., patches) from top to bottom, corresponding to three sub-sequences. Each local image token corresponds to one sub-sequence, that is, one sub-region of the original image. The image encoder aggregates global / local information into global / local image tokens through a self-attention mechanism. The global and local image tokens output from the last layer of the image encoder are selected and concatenated as multi-granularity image features.

[0051] For text encoders and image encoders, token visibility is controlled by combining masks during self-attention computation, such as... Figure 3 As shown. Among them. Figure 3 (a) describes the motivation for visibility control. Figure 3 (b) and Figure 3 (c) provides mask examples for text and images respectively, where attention scores for darker areas are retained and attention scores for lighter areas are discarded. The specific formula is as follows:

[0052]

[0053]

[0054] Where Q, K, and V represent the query, key, and value matrices in the self-attention mechanism, and D represents the feature dimension; softmax(·) represents the conversion of attention scores into probability scores, where attention scores with values ​​of negative infinity will result in probability scores of zero, thereby controlling the visibility between tokens and enabling information interaction at different granularities; M is the text attention mask in the text encoder or the image attention mask in the image encoder; M ij Let M be the element in the i-th row and j-th column of the mask M. Physically, this represents whether the i-th and j-th tokens can exchange information (for calculating the attention score). ij A value of 0 indicates that the two tokens can interact, while a value of negative infinity indicates that they cannot interact. By design, only tokens with the same granularity are allowed to interact. Vis(i) represents retrieving other tokens with the same granularity as the i-th token.

[0055] 2) Freeze the weights of the text encoder and image encoder, and use the person re-identification dataset (i.e., the source domain dataset) to perform vision-to-language distillation on the multi-granularity distillation feature extraction network model, i.e., extract generalization knowledge, complete the optimization of trainable multi-granularity text token weights, and save their weights; in vision-to-language distillation, the weights of both the image and text encoders are frozen, but the trainable multi-granularity text token weights are optimized. The network structure diagram is as follows. Figure 2 As shown in “Stage-1: Visual to Language Distillation”.

[0056] When performing vision-to-language distillation on a multi-granularity distillation feature extraction network model using a person re-identification dataset, the loss function employed includes cross-modal multi-granularity prototype contrastive loss. The formula is as follows:

[0057]

[0058] in, and Let the text features representing the feature centers of the multi-granularity image and the description sentence features of the identity of the c-th pedestrian be referred to as the multi-granularity image and the text prototype, respectively. Let represent the multi-granularity image feature center of the j-th pedestrian, C represent the total number of pedestrian identities, and s(.,.) represent the cosine similarity. The optimization process aligns the image and text prototypes of pedestrians with the same identity, and distills the domain-independent information (encoding pedestrian identity) in the image into a trainable multi-granularity text token to alleviate the coupling with identity-independent information in the training dataset.

[0059] 3) Freeze the weights of the text encoder and the trainable multi-granularity text tokens, unfreeze the weights of the image encoder, and use them to train the image encoder to extract multi-granularity image features for re-identification. Utilize the pedestrian re-identification dataset to perform language-to-visual distillation on the multi-granularity distillation feature extraction network model, that is, inject the generalization knowledge of language modalities into the image encoder, complete the optimization of the image encoder weights, and save their weights to obtain the trained image encoder; the network structure diagram is as follows. Figure 2 As shown in "Phase 2: Language to Visual Distillation".

[0060] When performing language-to-visual distillation on a multi-granularity distillation feature extraction network model using a pedestrian re-identification dataset, the image encoder weights are unfrozen and optimized to learn two capabilities: re-identification capability and generalization capability. The re-identification capability learning constrains the image encoder to extract features that distinguish different pedestrian identities, while the generalization capability learning reinjects domain-independent knowledge from the visual-to-language distillation process into the image encoder, preventing overfitting to the source domain dataset during the re-identification capability learning process. The loss function used for re-identification capability learning includes identity classification loss. and based on image prototype feature library Intramodal multi-granularity prototype contrast loss The generalization ability learning is responsible for injecting the generalization knowledge extracted from the visual-to-linguistic distillation into the linguistic modality into the image encoder through training. The loss function used includes cross-modal classification loss.

[0061] Identity Classification Loss The formula is as follows:

[0062]

[0063] Where W is a fully connected layer classifier used to classify image features. Indicates the yth i A classifier for each class, W j This represents the classifier for the j-th class.

[0064] Based on image prototype feature library Intramodal multi-granularity prototype contrast loss The formula is as follows:

[0065]

[0066]

[0067] in, Represents multi-granularity image features, y i Let represent the category of the current i-th sample, τ be the temperature hyperparameter controlling the intensity of the similarity calculation, and s(·,·) represent the cosine similarity. The multi-granularity image prototype feature represents the category to which the current i-th sample belongs. Multi-granularity image prototype features representing the j-th category; Image prototype feature library The feature base is initialized using the latest image feature centers before the start of the current iteration and updated with momentum as samples from the training batches begin. γ is a momentum hyperparameter that controls the intensity of feature base updates, and v... mg* It is the hardest sample in each pedestrian identity image sample in a training batch, and ← indicates iterative update.

[0068] Cross-modal classification loss The formula is as follows:

[0069]

[0070] in, Representing multi-granularity image features, Represents a multi-granularity text prototype. Let represent the multi-granularity text prototype of the j-th category, C represent the total number of pedestrian identities, s(·,·) represent the cosine similarity, and q cA soft tag representing the identity of the c-th pedestrian.

[0071] 4) Input the image to be detected (i.e. the target domain dataset) into the trained image encoder, extract the multi-granularity image features of the obtained image, and then use the multi-granularity image features to perform pedestrian re-identification to obtain the pedestrian re-identification result.

[0072] To verify the effectiveness of this invention, its generalizable person re-identification performance was validated on multiple publicly available datasets and compared with state-of-the-art methods. The datasets included Market1501, DukeMTMC-reID, MSMT17, and CUHK03-NP, abbreviated as MA, D, MS, and C3, respectively. CUHK03-NP used a more challenging subset of detections. The size of these datasets is described in Table 1 by the number of cameras included, the number of person identities included, and the number of image samples included.

[0073] Table 1 summarizes the dataset size.

[0074]

[0075] Based on the consensus of generalizable pedestrian re-identification tasks, two protocols were used for performance validation: a single-source protocol and a multi-source protocol. The single-source protocol used the MA and MS datasets, with the model trained on the training set of one dataset and validated on the test set of the other. The multi-source protocol used leave-one-out across all datasets, i.e., validating on the test set of one dataset and training on the training set formed by merging the remaining datasets. Both protocols were quantitatively evaluated using the mean average precision (mAP) and the first-hit accuracy (Rank-1) of the cumulative matching feature curve. Post-processing re-ranking techniques were not used in the validation.

[0076] The specific experimental configuration is as follows: The backbone network uses the image and text encoders of the vision-language pre-trained model CLIP. To prevent training instability, the weights of the image serialization layer of the image encoder are frozen. The input image resolution is 256×128 and undergoes random horizontal flipping, cropping, and erasing enhancement processing. For the MA to MS generalization protocol, the sliding window hyperparameters κ and δ are set to 14 and 1, respectively; for other generalization protocols, they are set to 10 and 3, respectively. The trainable multi-granularity text token feature dimension D is set to 512, and N=4 trainable tokens are used to describe the current granularity. For both modalities, local tokens use the same positional encoding as global tokens.

[0077] In the visual-to-language distillation, the initial learning rate was set to 3.5 × 10⁻⁶. -4 The cosine decaying learning rate planner was used to train the program for 120 epochs.

[0078] In the language-to-visual distillation, the initial learning rate was set to 5.0 × 10⁻⁶. -6 The system was trained for 60 epochs using a step learning rate planner combined with a linear warm-up strategy. The momentum hyperparameter γ of the image prototype feature library was set to 0.2. The temperature hyperparameter τ of the intramodal multi-granularity prototype contrast loss was set to 0.01.

[0079] In the two-stage distillation, the Adam optimizer was used to update the model weights, with a 1.0 × 10⁻⁶ ohmmeter. -4 The weight decay factor was applied. The sample batch size was set to 64, and 4 samples were randomly selected from 16 randomly chosen pedestrian identities.

[0080] The experiment mainly consists of two parts: the first part is a comparative experiment between the method of the present invention and the most advanced generalizable pedestrian re-identification method; the second part is an ablation experiment of each design in the present invention to illustrate the effectiveness of each design in the present invention.

[0081] 1) Comparative experiment with state-of-the-art methods

[0082] Table 2 presents the comparison results between the proposed method and state-of-the-art methods on single-source protocols. The complete method of this invention (denoted as ViLaMD-PCL) demonstrates a significant advantage over other methods in every generalization test. Compared to the visual Transformer-based model PAT, which uses only visual modal information, the proposed method achieves a 7.8% mAP and 11.0% Rank-1 improvement on the MA to MS generalization protocol, demonstrating the effectiveness of textual modal knowledge in improving generalization. Compared to the DMF method, which utilizes the additional FineGPR dataset for pre-training to incorporate textual information, the proposed method achieves better performance using only the downstream task dataset, namely a 2.3% mAP and 0.6% Rank-1 improvement on the MS to MA generalization protocol, and a 4.5% mAP and 3.2% Rank-1 improvement on the more challenging MA to MS generalization protocol.

[0083] Table 2 compares the performance of the present invention and the current state-of-the-art method on single-source protocols.

[0084]

[0085]

[0086] Table 3 presents the comparison results between the method of this invention and the state-of-the-art methods on multi-source protocols. The complete method of this invention (denoted as ViLaMD-PCL) significantly outperforms the state-of-the-art methods under all four target domain settings of multi-source protocols. Especially on the most challenging MA+D+C3 to MS generalization protocol, this invention significantly surpasses the current best method PAT, achieving an improvement of 10.2% mAP and 15.1% Rank-1.

[0087] Table 3 compares the performance of the present invention and the current state-of-the-art methods on multi-source protocols.

[0088]

[0089] 2) Ablation experiments of each design

[0090] Ablation experiments are used to verify the effectiveness of each design in this invention.

[0091] Effectiveness of Multi-Granularity Features: To verify the effectiveness of the multi-granularity feature design, experiments were conducted under two settings: visual modality only and simultaneous visual and linguistic modalities, as shown in Table 4. First, the effectiveness of multi-granularity features was tested under the visual modality only setting, using CLIP's image encoder for re-recognition training. Compared to using only global features, multi-granularity feature training achieved better results. Under the simultaneous visual and linguistic modalities, experiments were conducted to verify the performance of the baseline model (denoted as Baseline-PCL) and the complete method of this invention (denoted as ViLaMD-PCL). The baseline model only performed global feature distillation, while the complete method performed multi-granularity feature distillation. Experimental results show that the complete method exhibits better performance than the baseline model, further demonstrating the effectiveness of multi-granularity features.

[0092] Table 4 shows the multi-granularity feature ablation experiments on a single-source protocol.

[0093]

[0094] Effectiveness of Multi-Granularity Visibility Control: As shown in Table 5, experiments investigated the effectiveness of the attention mask-based multi-granularity visibility control mechanism. Compared to calculating all attention scores without using an attention mask (denoted as ViLaMD-PCL*), the complete model of this invention (denoted as ViLaMD-PCL) delivers performance improvements across all metrics, except for a slight decrease in the Rank-1 metric of the MA to MS generalization protocol.

[0095] Table 5 shows the ablation experiments based on the multi-granularity visibility control mechanism of attention masking.

[0096]

[0097] Effectiveness of Prototype Comparison Loss: As shown in Table 6, experiments investigated the prototype comparison loss (i.e., cross-modal multi-granularity prototype comparison loss) proposed in this invention for use in the two-stage distillation process. Comparison loss with intramodal multi-granularity prototype The effectiveness of the model is assessed. The base model (represented by ViLaMD) uses classic sample-level contrastive loss in the first-stage distillation. and The second-stage distillation uses classic sample-level triplet contrast loss. when When used alone (represented by ViLaMD-PCL1), only a small amount of performance fluctuation was observed, indicating that using only one This can replace the classic two-sample-level loss. and Without causing performance degradation; when It is used alone (represented by ViLaMD-PCL2) to replace the classic sample-level loss. Significant performance improvements were observed, indicating that the prototype level performs better than the comparison sample level. Finally, the complete model of this invention (represented by ViLaMD-PCL) simultaneously uses... and It achieved optimal performance.

[0098] Table 6 shows the prototype-comparative loss ablation experiment.

[0099]

[0100] Hyperparameter analysis: Experiments investigated the impact of key hyperparameters on performance in this invention, including the sliding window step size δ (when outputting a fixed number of sub-regions, the window size κ can be automatically determined by the selection of step size δ, therefore no active analysis of κ is required) and The temperature hyperparameter τ.

[0101] As shown in Table 7, the experiment selected all available δ values ​​when the sliding window generates three sub-regions. No obvious fluctuations were observed in the experimental results, proving that the method proposed in this invention is robust to the selection of δ.

[0102] Table 7 shows the hyperparameter analysis of the sliding window step size.

[0103]

[0104]

[0105] As shown in Table 8, several values ​​of the temperature hyperparameter τ were selected in the experiment. τ controls the severity of the contrast loss and has a significant impact on the final performance. Based on the experimental results, the optimal performance is achieved at a temperature value of τ = 0.01.

[0106] Table 8 shows the temperature hyperparameter analysis.

[0107]

[0108] The above description is merely a specific embodiment of the present invention and is not intended to limit the present invention in any way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above examples based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A generalizable pedestrian re-identification method based on multi-granularity distillation of visual language, characterized in that, Includes the following steps: 1) Construct a multi-granularity distillation feature extraction network model. The multi-granularity distillation feature extraction network model includes a sliding window module, a Transformer-structured text encoder, and a Transformer-structured image encoder. The sliding window module is connected to the image encoder, and the input of the text encoder contains trainable multi-granularity text tokens. The input to the text encoder also includes global text tokens and local text tokens; The input image of the multi-granularity distillation feature extraction network model is input into the image encoder after passing through the sliding window module. The image encoder also includes a global image token and a local image token. For text encoders and image encoders, token visibility is controlled by combining masks during self-attention calculation, as shown in the following formula: in, , and This represents the query, key, and value matrix in a self-attention mechanism. Indicates feature dimension; This indicates the conversion of attention scores into probability scores, where attention scores with values ​​of negative infinity will result in probability scores of zero. For text attention masks in text encoders or image attention masks in image encoders; Indicates mask The Line number Column elements, Indicates taking out and the first Each token has other tokens with the same granularity; 2) Freeze the weights of the text encoder and image encoder, and use the pedestrian re-identification dataset to perform visual-to-language distillation on the multi-granularity distillation feature extraction network model to complete the optimization of the trainable multi-granularity text token weights and save their weights. 3) Freeze the weights of the text encoder and the trainable multi-granularity text tokens, unfreeze the weights of the image encoder, use the person re-identification dataset to perform language-to-visual distillation on the multi-granularity distillation feature extraction network model, optimize the weights of the image encoder and save its weights, and obtain the trained image encoder. 4) Input the image to be detected into the trained image encoder, extract the multi-granularity image features of the image, and then use the multi-granularity image features to perform pedestrian re-identification to obtain the pedestrian re-identification result.

2. The generalizable pedestrian re-identification method based on visual language multi-granularity distillation according to claim 1, characterized in that, In step 2), when performing vision-to-language distillation on the multi-granularity distillation feature extraction network model using the pedestrian re-identification dataset, the loss function employed includes cross-modal multi-granularity prototype contrast loss. The formula is as follows: in, and Representing the multi-granularity image feature centers and the first Text features of sentences describing the identity of pedestrians Indicates the first Multi-granularity image feature centers for individual pedestrians This indicates the total number of pedestrian identities. This represents the cosine similarity.

3. The generalizable pedestrian re-identification method based on visual language multi-granularity distillation according to claim 1, characterized in that, In step 3), when performing language-to-visual distillation on the multi-granularity distillation feature extraction network model using the pedestrian re-identification dataset, the loss function used for re-identification ability learning includes identity classification loss. and based on image prototype feature library Intramodal multi-granularity prototype contrast loss The loss function used for generalization ability learning includes cross-modal classification loss. .

4. The generalizable pedestrian re-identification method based on visual language multi-granularity distillation according to claim 3, characterized in that, The image prototype feature library Intramodal multi-granularity prototype contrast loss The formula is as follows: in, Representing multi-granularity image features, Indicates the current number The categories of each sample, It is the degree of drastic change in the similarity calculation results controlled by temperature hyperparameters. Represents cosine similarity. Indicates the current number Multi-granularity image prototype features of the category to which each sample belongs; Indicates the first Multi-granularity image prototype features for each category; It is the momentum hyperparameter that controls the intensity of feature library updates. It is the hardest sample among all pedestrian identity image samples in a training batch. This indicates iterative updates.

5. The generalizable pedestrian re-identification method based on visual language multi-granularity distillation according to claim 3, characterized in that, The cross-modal classification loss The formula is as follows: in, Representing multi-granularity image features, Indicates the first Multi-granular text prototypes for each category Indicates the first Multi-granular text prototypes for each category This indicates the total number of pedestrian identities. Represents cosine similarity. Indicates the first A soft label for a pedestrian's identity.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 5.

Images