Lifelong target re-identification method based on descriptive text prompt prototype compensation

Abstract

The invention discloses a lifelong target re-identification method based on descriptive text prompt prototype compensation, relates to the field of computer vision, and aims to display and solve the problem of feature drift caused by model increment update by introducing a prompt prototype compensation module. The module generates a multi-modal vision and text prototype for each target category through a vision language pre-training model, and maps old task features to a new task feature space through a cross attention mechanism so as to realize fine-grained semantic drift compensation. In addition, a boundary perception prototype correction module is designed to solve the problem of boundary confusion caused by updating of old prototypes. The module dynamically samples false features in compensation distribution to correct old prototypes with fuzzy boundaries, focuses on categories easy to confuse to maintain clear decision boundaries, optimizes cross-task and intra-task correction loss to reduce category confusion, effectively alleviates the problem of disastrous forgetting, and improves the adaptability and generalization ability of the model.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a lifelong target re-identification method based on descriptive text prompt prototype compensation. Background Art

[0002] Target re-identification refers to the recognition and matching of the same target by comparing and analyzing the target's visual features under different perspectives or time points. This technology can achieve continuous tracking of specific targets under cross-camera conditions and has important applications in surveillance and security, intelligent transportation, retail and other fields. Traditional target re-identification methods are mostly trained on a single dataset and evaluate the model performance in static scenes. However, in real-world environments, target image data is rarely provided all at once and is usually acquired step by step in the form of a sequential input stream. To this end, the target re-identification model needs to have the ability of lifelong learning, which means it can continuously learn new knowledge from a continuous data stream while retaining the memory of old knowledge. This is particularly important in dynamically changing application scenarios, such as intelligent monitoring and unmanned driving systems, where the system needs to adapt to the identity information of new targets in real time without losing existing knowledge.

[0003] The primary challenge facing lifelong object re-identification (REI) is catastrophic forgetting, whereby the model tends to forget previously learned knowledge when adapting to a new task. During lifelong learning, the continuous updating of the object re-identification model causes the feature space to constantly shift, which directly leads to catastrophic forgetting. Due to the fine-grained nature of target feature distribution, the inter-class differences between different targets are small, and the distribution boundaries are prone to overlapping interference. When feature drift occurs, samples at the distribution boundaries are easily forgotten, and the identities of objects with similar feature distributions are more easily confused. Summary of the Invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a lifelong target re-identification method based on prototype compensation with descriptive text prompts, which solves the catastrophic forgetting problem existing in the prior art.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: a lifelong target re-identification method based on descriptive text prompt prototype compensation, comprising the following steps: S1: Get pre-trained CLIP image encoder, CLIP text encoder and A dataset, introducing visual cues and initialization; S2: Add the initialized visual cues to each layer of the Transformer encoder of the CLIP image encoder; S3: Set up training steps , optimizing visual cues in the CLIP image encoder via a new task loss on the first dataset; S4: Use CLIP image encoder to calculate the visual prototype and variance of each target identity category in the dataset; S5: Optimize the descriptive text of each target identity category in the dataset through contrast loss to obtain a category-specific text description of each target identity; S6: Input the optimized category-specific text description of each target identity into the CLIP text encoder to obtain the encoded text features of each target identity category as the text prototype; S7: Training Step , to determine whether the current training step is , if yes, then obtain the text prototype of the current dataset and complete the lifelong target re-identification based on the descriptive text prompt prototype compensation, otherwise go to step S8; S8: Optimizing visual cues in the CLIP image encoder via a new task loss on the current dataset; S9: Optimize the compensator by prompting the prototype compensation loss, and use the optimized compensator to update the old class visual prototype in the previous dataset; S10: Based on the updated results, we optimize the visual cues in the CLIP image encoder via cross-task correction loss, intra-task correction loss, and prototype-based distillation loss. S11: Based on the optimization results, use the CLIP image encoder to calculate the visual prototype and variance of each target identity category in the current dataset; S12: Optimize the descriptive text of each target identity category in the current dataset through contrast loss to obtain a category-specific text description for each target identity; S13: Input the optimized category-specific text description of each target identity into the CLIP text encoder, obtain the encoded text features of each target identity category as a text prototype, and return to step S7.

[0006] Furthermore, the CLIP image encoder in S1 is formed based on the ViT model architecture and serves as the backbone network for feature extraction and representation learning; The CLIP text encoder is based on the Transformer model architecture and is used to convert the natural language text description related to the target identity into a text feature representation; The input form of the dataset is a set of data streams divided by time steps , No. Datasets Expressed as:

[0007] in, For the training step Middle The target image of samples, is the identity label corresponding to the target image, For the training step The total number of samples in ; The dimension of the visual cue is ; in, is the number of layers of the ViT model, is the sequence length of the prompt, Encodes the dimension of the feature.

[0008] Furthermore, in said S2, for the layer Transformer encoder, through the splicing operation Layer visual cues and the output features of the previous layer of the Transformer encoder Combined to form a new input feature, the formula is:

[0009] in, For the The output features of the layer Transformer encoder, For the The model parameters of the layer Transformer encoder, Represents a splicing operation; go through After the layer Transformer encoder is encoded, the visual features obtained From Layer output features The category token vector in is extracted and the formula is:

[0010] in, For the input image in the image encoder The encoding, is the category token vector of the final output feature.

[0011] Furthermore, the new task loss is:

[0012] in, Loss for new missions, is the cross entropy loss, is the triplet loss, is a hyperparameter;

[0013] in, is the batch size, is the number of categories, Marked the Does the sample belong to categories, For the classifier The sample prediction is The probability of the categories, is the classifier parameter;

[0014] in, is the maximum value, is the Euclidean distance, is the feature of the anchor sample, is the feature of the positive sample, is the feature of negative samples, is a pre-set boundary value.

[0015] Furthermore, the calculation formula for the visual prototype and variance of each target identity category in the dataset is:

[0016] in, For identity The visual prototype, For identity The number of samples, For the The visual features of samples, For the The identity labels of the samples;

[0017] in, For identity The variance of Indicates extracting the main diagonal elements of the matrix, Represents the transpose operation of a matrix.

[0018] Furthermore, the contrast loss includes image-to-text contrast loss and text-to-image contrastive loss , the formula is:

[0019] in, To calculate the cosine similarity between visual features and text features, The first The text features of samples, The first Text features of samples;

[0020] in, For batch The index set of all samples of the same category, Indicates the number of elements in the collection. For the batch samples, The first The visual features of samples, The first The visual features of samples, For the The text features corresponding to the identity category of each sample; By optimizing descriptive text prompts Minimize image-to-text contrastive loss and text-to-image contrastive loss , the formula is:

[0021] in, is the image-to-text contrast loss and text-to-image contrastive loss the sum of is the number of learnable text tokens.

[0022] Furthermore, the compensator in S9 includes a multi-head cross attention layer and a feedforward network layer, and S9 includes the following sub-steps: S91: Calculate the drift value predicted by the old visual prototype through the multi-head cross attention layer and the drift value obtained by predicting the old class text prototype , the formula is:

[0023]

[0024] in, Indicates multi-headed cross attention, is sampled from the current dataset The visual features extracted from the old model obtained in the previous training step for each training sample, is the old class visual prototype calculated in the previous training step, The prototype of the old class text calculated in the previous training step; S92: The drift value predicted by the old visual prototype and the drift value obtained by predicting the old class text prototype Merge to obtain the drift prediction of the current training sample from the old model to the new model feature space. The formula is:

[0025] in, is the predicted visual feature of the current time step, is the drift value predicted by the old class visual or text prototype; S93: Optimize the compensator by minimizing the mean square error between the drift prediction value and the true value, as follows:

[0026] in, To prompt the prototype to compensate for the loss, For the training step Middle The predicted visual features of samples, For the training step Middle The true visual features of samples; S94: The optimized compensator Update the old class visual prototypes in the previous dataset , the formula is:

[0027] in, From the training step To the training step Compensated old class visual prototype.

[0028] Furthermore, the cross-task correction loss in S10 is: pass and The pairwise cosine similarity between defines the degree of boundary fuzziness, and the formula is:

[0029] in, is the sample feature of the current task Hedi The fuzziness between the old class samples, For the previous dataset The visual prototype features after target category compensation, For the previous dataset The visual prototype features after target category compensation, is a predefined temperature parameter; From the fuzzy distribution mid-sampling old classes and Gaussian distribution from each old class prototype The sampled pseudo features are used to calculate the cross-task correction loss :

[0030]

[0031]

[0032] in, For the batch The index set of all samples with the same sample category, is the normalization term of the positive sample, is the normalization term of the pseudo sample, is the old class pseudo feature sampled.

[0033] Furthermore, the intra-task correction loss in S10 is:

[0034] in, is the within-task correction loss.

[0035] Furthermore, the distillation loss based on the prototype in S10 is:

[0036] in, is the prototype-based distillation loss, is the Kullback-Leibler divergence, is the Softmax normalization function, is the pairwise similarity matrix between the current task sample and the compensated old class visual prototype, From the current dataset Sampling The visual feature matrix consists of samples.

[0037] The beneficial effects of the present invention are as follows: 1. Replay-free fine-tuning of lifelong learning mechanisms. This invention eliminates the need to store and replay original image data from previous task target categories, effectively avoiding privacy leaks caused by data replay. By building visual prototypes, this invention reduces reliance on historical samples, thereby reducing storage requirements and computational costs. Furthermore, this invention achieves efficient parameter fine-tuning of pre-trained CLIP models by introducing only a small number of learnable cue parameters, accelerating model training while improving model adaptability.

[0038] 2. Multimodal Prototype Compensation and Textual Prompt Guidance Mechanism. The multimodal prototype compensation method proposed in this paper fuses visual and textual information to construct robust class prototypes with rich discriminative features. By utilizing the visual and textual prototypes generated by the CLIP model and combining them with descriptive textual prompts, this paper accurately compensates for fine-grained semantic feature drift, effectively alleviating the inconsistency problem in the feature space and improving the model's adaptability and robustness in multi-task lifelong learning scenarios.

[0039] 3. Cross-task and intra-task correction mechanisms. The cross-task and intra-task correction mechanisms proposed in this paper can effectively alleviate the problem of catastrophic forgetting in the lifelong object re-identification task. By introducing cross-task and intra-task correction losses, this paper can preserve the model's memory of previously learned knowledge when learning new tasks, maintaining clear decision boundaries between different object identities, and effectively improving the model's anti-forgetting and generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Flowchart of a lifelong object re-identification method based on prototype compensation with descriptive text cues.

[0041] Figure 2 A network model structure diagram for lifelong target re-identification based on prototype compensation of descriptive text prompts. DETAILED DESCRIPTION

[0042] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0043] This paper proposes a lifelong target re-identification method based on descriptive text prompt prototype compensation. By introducing a prompt prototype compensation module to explicitly solve the feature drift problem caused by incremental model updates, this module generates multimodal visual and text prototypes for each target category through the visual language pre-training model CLIP, and maps the old task features to the new task feature space through the cross-attention mechanism, thereby achieving fine-grained semantic drift compensation. In addition, a boundary-aware prototype correction module is designed to solve the boundary confusion problem caused by the update of old class prototypes. This module dynamically samples pseudo-features in the compensation distribution to correct old class prototypes with blurred boundaries, focusing on easily confused categories to maintain clear decision boundaries, and optimizing cross-task and intra-task correction losses to reduce category confusion, effectively alleviating the catastrophic forgetting problem and improving the adaptability and generalization ability of the model.

[0044] like Figure 1 As shown, a lifelong target re-identification method based on prototype compensation of descriptive text prompts includes the following steps: S1: Get pre-trained CLIP image encoder, CLIP text encoder and A dataset, introducing visual cues and initialization; S2: Add the initialized visual cues to each layer of the Transformer encoder of the CLIP image encoder; S3: Set up training steps , optimizing visual cues in the CLIP image encoder via a new task loss on the first dataset; S4: Use CLIP image encoder to calculate the visual prototype and variance of each target identity category in the dataset; S5: Optimize the descriptive text of each target identity category in the dataset through contrast loss to obtain a category-specific text description of each target identity; S6: Input the optimized category-specific text description of each target identity into the CLIP text encoder to obtain the encoded text features of each target identity category as the text prototype; S7: Training Step , to determine whether the current training step is , if yes, then obtain the text prototype of the current dataset and complete the lifelong target re-identification based on the descriptive text prompt prototype compensation, otherwise go to step S8; S8: Optimizing visual cues in the CLIP image encoder via a new task loss on the current dataset; S9: Optimize the compensator by prompting the prototype compensation loss, and use the optimized compensator to update the old class visual prototype in the previous dataset; S10: Based on the updated results, we optimize the visual cues in the CLIP image encoder via cross-task correction loss, intra-task correction loss, and prototype-based distillation loss. S11: Based on the optimization results, use the CLIP image encoder to calculate the visual prototype and variance of each target identity category in the current dataset; S12: Optimize the descriptive text of each target identity category in the current dataset through contrast loss to obtain a category-specific text description for each target identity; S13: Input the optimized category-specific text description of each target identity into the CLIP text encoder, obtain the encoded text features of each target identity category as a text prototype, and return to step S7.

[0045] like Figure 2 As shown, the CLIP image encoder in S1 is formed based on the ViT model architecture and serves as the backbone network for feature extraction and representation learning; In this example, the ViT-B / 16 model architecture is used, where "B" represents the base model and 16 indicates that the model divides the input image into 16×16 pixel blocks. ViT-B / 16 consists of 12 Transformer encoder layers, and each image block is encoded into a 768-dimensional visual feature embedding vector. The CLIP text encoder is based on the Transformer model architecture and is used to convert the natural language text description related to the target identity into a text feature representation; The CLIP text encoder is a 12-layer Transformer model that uses byte pair encoding to process text. The dimension of the text feature embedding vector is 512. The input form of the dataset is a set of data streams divided by time steps , No. Datasets Expressed as:

[0046] in, For the training step Middle The target image of samples, is the identity label corresponding to the target image, For the training step The total number of samples in ; The present invention selects We use three commonly used object re-identification datasets for incremental training, namely Market1501, CUHK-SYSU, DukeMTMC, MSMT17, and CUHK03.

[0047] The dimension of the visual cue is , is a set of additional learnable parameter vectors that are added to each layer of the image encoder. The role of visual cues is to guide the image encoder to continuously learn new tasks through the prompt tuning strategy without modifying the backbone network parameters of the ViT model; in, is the number of layers of the ViT model, is the sequence length of the prompt, Encodes the dimension of the feature.

[0048] In the S2, for layer Transformer encoder, through the splicing operation Layer visual cues and the output features of the previous layer of the Transformer encoder Combined to form a new input feature, the formula is:

[0049] in, For the The output features of the layer Transformer encoder, For the The model parameters of the layer Transformer encoder, Represents a splicing operation; go through After the layer Transformer encoder is encoded, the visual features obtained From Layer output features The category token vector in is extracted and the formula is:

[0050] in, For the input image in the image encoder The encoding, is the category token vector of the final output feature.

[0051] The new task loss is:

[0052] in, Loss for new missions, is the cross entropy loss, is the triplet loss, is a hyperparameter used to balance the triplet loss contribution to the calculation of losses;

[0053] Cross Entropy Loss Use a classifier to measure the difference between the model's predicted probability of different target identities and the true label; in, The batch size is set to 64 in the experiment, including 16 randomly selected target categories and 4 images per category. is the number of categories, Marked the Does the sample belong to categories (0 or 1), For the classifier The sample prediction is The probability of the categories, is the classifier parameter;

[0054] in, is the maximum value, is the Euclidean distance, is the feature of the anchor sample, is the feature of the positive sample, is the feature of negative samples, is a pre-set boundary value.

[0055] Triplet loss Used for deep metric learning, it ensures that samples of the same category are closer in the feature space, while pushing the distance between samples of different categories further.

[0056] The present invention trains each data set for 60 epochs and uses the Adam optimizer for optimization. The learning rate of the first 10 epochs is Increase linearly to , followed by learning rate decay at the 30th and 50th epochs, and data augmentation of the training images using random horizontal flipping, padding, cropping, and erasing strategies.

[0057] The formula for calculating the visual prototype and variance of each target identity category in the dataset is:

[0058] in, For identity The visual prototype, For identity The number of samples, For the The visual features of samples, , For the The identity labels of the samples;

[0059] in, For identity The variance of Indicates extracting the main diagonal elements of the matrix, Represents the transpose operation of a matrix.

[0060] Through the above calculation method, the identity The characteristic distribution of can be modeled as a Gaussian distribution .

[0061] The contrast loss includes image-to-text contrast loss and text-to-image contrastive loss , the formula is:

[0062] in, To calculate the cosine similarity between visual features and text features, The first The text features of samples, The first Text features of samples;

[0063] in, For batch The index set of all samples of the same category, , and Indicates the / The identity category to which the samples belong, Indicates the number of elements in the collection. For the batch samples, The first The visual features of samples, The first The visual features of samples, For the The text features corresponding to the identity category of the sample, ; In this embodiment, a descriptive text prompt template "A photo of a person." to optimize the target identity-specific text description for each target category, where Represents the word embedding text token vector. The number of learnable text tokens is set in the experiment. .

[0064] By optimizing descriptive text prompts Minimize image-to-text contrastive loss and text-to-image contrastive loss , the formula is:

[0065] in, is the image-to-text contrast loss and text-to-image contrastive loss the sum of is the number of learnable text tokens.

[0066] Specifically, the present invention trains each data set for 120 epochs, uses the Adam optimizer for optimization, and initializes the learning rate to , followed by learning rate decay through a cosine scheduler. After training, each target identity has its own category-specific text description.

[0067] To solve the catastrophic forgetting problem caused by feature drift, the present invention learns a compensator It is used to map features from the old feature space to the new feature space. The network structure of the compensator consists of a multi-head cross attention layer and a feedforward network layer.

[0068] The compensator in S9 includes a multi-head cross attention layer and a feedforward network layer. S9 includes the following steps: S91: Calculate the drift value predicted by the old visual prototype through the multi-head cross attention layer and the drift value obtained by predicting the old class text prototype , the formula is:

[0069]

[0070] in, Indicates multi-headed cross attention, is sampled from the current dataset The visual features extracted from the old model obtained in the previous training step for each training sample, is the old class visual prototype calculated in the previous training step, Represents the previous task dataset The number of target categories included, represents the feature dimension, is the old class text prototype calculated in the previous training step, is the set of real numbers; S92: The drift value predicted by the old visual prototype and the drift value obtained by predicting the old class text prototype Merge to obtain the drift prediction of the current training sample from the old model to the new model feature space. The formula is:

[0071] in, is the predicted visual feature of the current time step, is the drift value predicted by the old class visual or text prototype; S93: Optimize the compensator by minimizing the mean square error between the drift prediction value and the true value, as follows:

[0072] in, To prompt the prototype to compensate for the loss, For the training step Middle The predicted visual features of samples, For the training step Middle The true visual features of samples; This training process keeps all parameters of the CLIP image encoder frozen to train the compensator independently Specifically, the present invention trains each data set for 120 epochs, uses the Adam optimizer for optimization, and initializes the learning rate to , then the learning rate is decayed through the cosine scheduler, and the parameters of the compensator are frozen after training, which is only used to update the old class visual prototype.

[0073] S94: The optimized compensator Update the old class visual prototypes in the previous dataset , the formula is:

[0074] in, From the training step To the training step Compensated old class visual prototype.

[0075] The cross-task correction loss in S10 is: In the old visual prototype After compensation, The distribution of the new class prototype may be The distribution of overlaps. To address the boundary confusion problem caused by prototype update, this paper corrects the loss across tasks. To maintain boundary distinction: pass and The pairwise cosine similarity between defines the degree of boundary fuzziness, and the formula is:

[0076] in, is the sample feature of the current task Hedi The fuzziness between the old class samples, For the previous dataset The visual prototype features after target category compensation, For the previous dataset The visual prototype features after target category compensation, is a predefined temperature parameter; From the fuzzy distribution mid-sampling old classes and Gaussian distribution from each old class prototype Six pseudo features are sampled from the dataset to calculate the cross-task correction loss. :

[0077]

[0078]

[0079] in, For the batch The index set of all samples with the same sample category, is the normalization term of the positive sample, is the normalization term of the pseudo sample, In the sampling process, the old class prototypes with higher similarity after compensation are more likely to be sampled for prototype correction, thereby better alleviating the forgetting problem of the old class prototypes that are easily confused.

[0080] To reduce boundary confusion within the current task, the task correction loss To correct the inter-class discrimination boundary: The intra-task correction loss in S10 is:

[0081] in, is the intra-task correction loss, Calculate the relationship between all samples (including positive and negative samples) and sample The similarity of the visual features of is used to normalize the output.

[0082] In order to transfer the old class prototype knowledge to the current task model to reduce the forgetting of old knowledge, the prototype-based distillation loss is calculated : The prototype-based distillation loss in S10 is:

[0083] in, is the prototype-based distillation loss, is the Kullback-Leibler divergence, is the Softmax normalization function, is the pairwise similarity matrix between the current task sample and the compensated old class visual prototype, From the current dataset Sampling The visual feature matrix composed of samples, Calculated The similarity matrix between samples of visual features, The similarity matrix between samples with the old class prototype as the reference coordinate is calculated.

[0084] The overall loss of the prompt prototype compensation framework proposed in this invention is calculated as follows:

[0085] in, is a constant, and , , , which is used to balance the contribution of each loss term in the overall loss calculation. Specifically, the present invention trains each data set for 60 epochs, uses the Adam optimizer for optimization, and initializes the learning rate to , followed by learning rate decay at the 30th and 50th epochs.

[0086] After completing the After training steps, increase Value , and returns to step S7 to continue processing the next dataset until all datasets are processed. It should be noted that the method of the present invention only constructs a class prototype for each target category during the training phase. During the inference phase, the present invention does not need to calculate the class prototype for testing. Instead, it directly uses the image encoder to extract the visual features of each test sample and performs target matching between the query set and the gallery set features to calculate the mAP and R@1 accuracy indicators.

[0087] The performance results of continuous lifelong learning of the present invention on five target re-identification datasets, namely Market1501, CUHK-SYSU, DukeMTMC, MSMT17 and CUHK03, are shown in Table 1. The comparison algorithms include LwF, AKA, PatchKD, LSTKC, C2R, DKP and PAEMA.

[0088] Table 1 Comparison of performance results of the method of the present invention and other methods

[0089] The experimental results in Table 1 demonstrate that our method achieves the best results in both test performance and overall average performance on five object re-identification datasets. Compared to the state-of-the-art methods, our method achieves a 3.2% / 3.7% improvement in average mAP / R@1 performance. These experimental results demonstrate that our proposed descriptive text-cue prototype compensation and boundary-aware prototype correction strategies effectively overcome catastrophic forgetting and enhance adaptability to new tasks.

[0090] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the invention.

Claims

1. A lifelong object re-identification method based on prototype compensation with descriptive text prompts, characterized in that: The following steps are involved: S1: Get the pre-trained CLIP image encoder, CLIP text encoder, and A dataset, introducing visual cues and initialization; S2: Add the initialized visual cues to each layer of the Transformer encoder of the CLIP image encoder; S3: Set up training steps , optimizing visual cues in the CLIP image encoder via a new task loss on the first dataset; S4: Use CLIP image encoder to calculate the visual prototype and variance of each target identity category in the dataset; S5: optimizing the descriptive text of each target identity category in the dataset by contrastive loss to obtain a category-specific text description of each target identity; S6: Input the optimized category-specific text description of each target identity into the CLIP text encoder to obtain the encoded text features of each target identity category as the text prototype; S7: Training Step , to determine whether the current training step , if yes, then obtain the text prototype of the current data set and complete the lifelong target re-identification based on the descriptive text prompt prototype compensation, otherwise go to step S8; S8: Optimizing visual cues in the CLIP image encoder via a new task loss on the current dataset; S9: Optimize the compensator by prompting the prototype compensation loss, and use the optimized compensator to update the old class visual prototype in the previous dataset; S10: Based on the updated results, the visual cues in the CLIP image encoder are optimized by cross-task correction loss, intra-task correction loss, and prototype-based distillation loss; S11: Based on the optimization results, the CLIP image encoder is used to calculate the visual prototype and variance of each target identity category in the current dataset; S12: Optimize the descriptive text of each target identity category in the current dataset through contrast loss to obtain a category-specific text description of each target identity; S13: Input the optimized category-specific text description of each target identity into the CLIP text encoder, obtain the encoded text features of each target identity category as a text prototype, and return to step S7.

2. The method for lifelong target re-identification based on prototype compensation of descriptive text prompts according to claim 1, characterized in that: The CLIP image encoder in S1 is formed based on the ViT model architecture and serves as a backbone network for feature extraction and representation learning; The CLIP text encoder is formed based on the Transformer model architecture and is used to convert the natural language text description related to the target identity into a text feature representation; The input form of the dataset is a set of data streams divided by time steps , No. Datasets It is expressed as: in, For the training step Middle The target image of samples, is the identity label corresponding to the target image, For the training step The total number of samples in ; The dimension size of the visual cue is ; in, is the number of layers of the ViT model, is the sequence length of the prompt, Encodes the dimension of the feature.

3. The method for lifelong target re-identification based on prototype compensation of descriptive text prompts according to claim 2, characterized in that: In S2, for the The Transformer encoder of the first layer is concatenated Layer Visual Cues And the output features of the previous layer of Transformer encoder Combined to form new input features, the formula is: in, For the The output features of the layer Transformer encoder, For the The model parameters of the layer Transformer encoder, Represents a splicing operation; go through After layer Transformer encoder encoding, the visual features obtained From Layer output features The category token vector in is extracted and the formula is: in, For the input image in the image encoder The encoding, is the category token vector of the final output feature.

4. The method for lifelong target re-identification based on prototype compensation with descriptive text prompts according to claim 3, characterized in that: The new task loss is: in, Loss for new tasks, is the cross entropy loss, is the triplet loss, is a hyperparameter; in, is the batch size, is the number of categories, Marked the Does the sample belong to categories, For the classifier The sample prediction is The probability of the categories, is the classifier parameter; in, is the maximum value, is the Euclidean distance, is the feature of the anchor sample, is the feature of the positive sample, is the feature of negative samples, is a pre-set boundary value.

5. The method for lifelong target re-identification based on prototype compensation with descriptive text prompts according to claim 4, characterized in that: The calculation formula for the visual prototype and variance of each target identity category in the dataset is: in, For identity The visual prototype, For identity The number of samples, For the The visual features of samples, For the The identity labels of the samples; in, For identity The variance of It means extracting the main diagonal elements of the matrix. Represents the transpose operation of a matrix.

6. The method for lifelong target re-identification based on prototype compensation with descriptive text prompts according to claim 5, characterized in that: The contrast loss includes image-to-text contrast loss and text-to-image contrastive loss , the formula is: in, To calculate the cosine similarity between visual features and text features, The first The text features of samples, The first Text features of samples; in, For batch The index set of all samples of the same category, Indicates the number of elements in a collection. For the batch samples, The first The visual features of samples, The first The visual features of samples, For the The text features corresponding to the identity category to which the sample belongs; By optimizing the descriptive text prompt Minimize image-to-text contrastive loss and text-to-image contrastive loss , the formula is: in, is the image-to-text contrast loss and text-to-image contrastive loss the sum of is the number of learnable text tokens.

7. The method for lifelong target re-identification based on prototype compensation with descriptive text prompts according to claim 6, characterized in that: The compensator in S9 includes a multi-head cross attention layer and a feedforward network layer, and S9 includes the following sub-steps: S91: Calculate the drift value predicted by the old visual prototype through a multi-head cross attention layer and the drift value predicted by the old class text prototype , the formula is: in, Indicates multi-head cross attention, is sampled from the current dataset The visual features extracted from the training samples on the old model obtained in the previous training step, is the old class visual prototype calculated in the previous training step, The prototype of the old class text calculated in the previous training step; S92: The drift value predicted by the old visual prototype and the drift value predicted by the old class text prototype Merge to obtain the drift prediction of the current training sample from the old model to the new model feature space. The formula is: in, is the predicted visual feature of the current time step, is the drift value predicted by the old class visual or text prototype; S93: Optimize the compensator by minimizing the mean square error between the drift prediction value and the true value, the formula is: in, To prompt the prototype to compensate for the loss, For the training step Middle The predicted visual features of samples, For the training step Middle The real visual features of samples; S94: The optimized compensator Update the old class visual prototypes in the previous dataset , the formula is: in, From the training step To the training step Compensated old class visual prototype.

8. The method for lifelong target re-identification based on prototype compensation with descriptive text prompts according to claim 7, characterized in that: The cross-task correction loss in S10 is: pass and The pairwise cosine similarity between defines the degree of boundary blurring, and the formula is: in, is the sample feature of the current task and The fuzziness between old class samples, For the previous data set The visual prototype features after target category compensation, For the previous data set The visual prototype features after target category compensation, is a predefined temperature parameter; From the fuzzy distribution Medium Sampling old classes, and from each old class prototype Gaussian distribution The sampled pseudo features are used to calculate the cross-task correction loss : in, For the batch The index set of all samples with the same sample category, is the normalization term of the positive sample, is the normalization term of the pseudo sample, is the old class pseudo feature sampled.

9. The method for lifelong target re-identification based on prototype compensation with descriptive text prompts according to claim 8, characterized in that: The intra-task correction loss in S10 is: in, is the within-task correction loss.

10. The method for lifelong target re-identification based on prototype compensation with descriptive text prompts according to claim 9, characterized in that: The prototype-based distillation loss in S10 is: in, is the prototype-based distillation loss, is the Kullback-Leibler divergence, is the Softmax normalization function, is the pairwise similarity matrix between the current task sample and the compensated old class visual prototype, From the current dataset Sampled in The visual feature matrix consists of samples.

Images