Reloading pedestrian re-identification method and system based on visual language pre-training model

Abstract

The invention relates to the technical field of computer vision, in particular to a reloading pedestrian re-identification method based on a visual language pre-training model. The method comprises the following steps: a training stage: inputting an image to obtain a clothing mask image, and generating clothing irrelevant / relevant prompts; text encoder parameters are fixed, prompt weights are optimized, text prompts are input into an encoder to obtain features, and a classifier is constructed to achieve image-text alignment through cross entropy loss; using a visual encoder to extract mask pattern features, and constraining a class center Euclidean distance to realize image-image alignment; stripping clothes characteristics: extracting clothes area characteristics and corresponding text characteristics, optimizing by a classifier, and introducing orthogonal loss to decouple clothes correlation; in the reasoning stage, a query image is input into a trained image encoder to extract features, and cosine similarity ranking and result returning are calculated according to the features of the image library. According to the technical scheme, the recognition accuracy of the pedestrian re-recognition method under the condition of pedestrian clothing change can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and system for re-identifying pedestrians after changing clothes based on a visual language pre-training model. Background Art

[0002] Person re-identification (ReID) technology aims to accurately identify and retrieve pedestrians with the same identity across different cameras, and has important applications in fields such as intelligent surveillance and smart security. However, existing methods are mostly limited to pedestrian re-identification tasks within a short period of time and a limited space, and generally assume that pedestrians wear the same clothing when viewed by different cameras. In real-world applications, pedestrians with the same identity often change their clothing at different times or occasions, and suspects often use this method to evade capture. Directly applying traditional pedestrian re-identification methods to these scenarios results in a significant performance degradation due to their over-reliance on clothing information (such as pattern and color) as a discriminative clue.

[0003] To address this issue, the clothing-changing person re-identification task was proposed. Existing methods fall into two main categories: one focuses on extracting biometric features (such as silhouette sketches, body shape, facial features, and gait information); the other uses feature decoupling methods to separate clothing features from pedestrian characteristics. These methods typically employ an "image encoder + ImageNet pre-trained weights + one-hot labels" architecture. However, the limited information provided by the one-hot labels limits the model's ability to learn fine-grained semantic features.

[0004] The visual language pre-training model CLIP (Contrastive Language-Image Pre-Training) connects visual representations with high-level semantic language descriptions through image-text contrast learning, providing a new approach to solving the aforementioned problems. However, when CLIP is directly applied to the task of re-identifying people with changed clothing, the generated textual prompts tend to describe clothing-related clues rather than identity information, resulting in poor performance. Summary of the Invention

[0005] The purpose of the present invention is to propose a method and system for re-identifying pedestrians with changing clothing based on a visual language pre-training model, which can improve the recognition accuracy of the pedestrian re-identification method under the condition of pedestrian clothing changes.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a method for re-identifying pedestrians after changing clothes based on a visual language pre-training model, comprising: Input image and perform clothes masking to get masked image: Generate clothing-related and clothing-independent text prompts: Image-text alignment and image-image alignment training, including: Fixed weights of the text encoder and weights of the learnable prompts; The generated text prompt is input into the text encoder to obtain the text features of each identity. This is used as the identity prototype to construct a fixed-weight classifier. The cross-entropy loss is used to encourage the pedestrian features output by the visual encoder to be correctly classified in the classifier, thus achieving image-text alignment. An additional visual encoder is used to encode the masked image into pedestrian features. The Euclidean distance between the class centers of the masked image and the normal image of the same identity is calculated. The Euclidean distance is used to constrain the two to be close in the feature space to achieve image-to-image alignment. A feature minimization module is introduced to extract clothing-related features, including: Extracting image segments containing clothing areas from the original image, introducing a learnable global optimization projection matrix, and multiplying pedestrian features with the projection matrix to extract clothing features; Inputting a preset clothing category text prompt into a text encoder to obtain clothing text features, constructing a clothing text classifier, inputting the clothing features into the clothing text classifier for classification, and optimizing through cross entropy loss; Orthogonal loss is introduced to penalize the cosine similarity between pedestrian features and clothing features, so that the two maintain a distance in the feature space; In the inference phase, the trained and optimized image encoder is used to extract features from the input image; the cosine similarity between the query image features and the library image features is calculated, and the retrieval results are returned based on the similarity sorting.

[0007] Beneficial effects of the basic scheme: This scheme actively reduces the interference of clothing information on the model by masking clothes in the image; at the same time, it uses clothing-independent text prompts to guide the model to focus on stable features such as human outline, hairstyle, and posture. Compared with traditional pedestrian re-identification models, which often rely too much on clothing features and have a significant performance drop in pedestrian changing scenes, this scheme enables the model to maintain stable recognition accuracy in complex scenes with frequent clothing changes, significantly enhancing its robustness in practical applications.

[0008] Breaking through the limitations of traditional single visual modality, this approach leverages a pre-trained visual language model to generate textual prompts, deeply integrating image information with textual semantics. Clothing-independent textual prompts focus on core identity features, while clothing-related textual prompts provide a reference for feature extraction.

[0009] Through image-text alignment training, the model prioritizes learning stable identity-related features and uses text features as "prototypes" to strengthen their representation. This ensures that the features output by the visual encoder are both visually recognizable and semantically consistent, significantly improving feature discrimination and generalization compared to methods that rely solely on image labels. By introducing a feature minimization module, an orthogonal loss is used to enforce spatial separation of pedestrian and clothing features. Simultaneously, a clothing text classifier is used to accurately remove clothing interference (such as color and style). These two factors work together to achieve a purer feature extraction, effectively preventing identity features from being "contaminated" by clothing information.

[0010] This solution introduces an auxiliary module to optimize feature representation during the training phase, while retaining only the backbone image encoder during the inference phase. Therefore, it does not introduce additional inference costs and has good practicality and deployment value.

[0011] The method of the present invention is suitable for pedestrian re-identification tasks in public security monitoring, trajectory tracking, crowd analysis, and other complex conditions involving long time, multiple scenes, clothing changes, etc., and has important engineering application value and promotion potential.

[0012] As an implementable preferred solution, inputting an image and performing clothing shielding processing includes: Get the original image containing the target pedestrian and resize the image to a uniform parameter; The original image is semantically segmented through a human body parsing network to obtain a mask containing a clothing area. The mask is used to set the clothing area pixels in the original image to a constant to obtain a shielded image. Otherwise, the clothing area pixels are retained to obtain a clothing image.

[0013] As an implementable preferred solution, generating clothing-independent text prompts includes: Construct an initial text prompt template, where the fixed text is converted into a text embedding through CLIP's token embedding layer and concatenated with the learnable tensor to obtain a text embedding describing the identity; Input the masked image into the image encoder of CLIP to obtain visual features, and embed the text into the text encoder of CLIP to obtain text features; The learnable tensor is optimized to express identity-related and clothing-independent semantic information through text-image and image-text contrastive losses. The loss functions are as follows:

[0014] in, and is the image-text contrast loss and text-image contrast loss used to train identity text prompt generation; Use the generated clothing-independent text prompts to guide the visual encoder training and design the loss function:

[0015] in, is the identity loss, including cross entropy loss and triple loss, is the cross entropy loss, used to align pedestrian features and text features; To constrain the identity loss of the shielded network, The image-text cross entropy loss is used to constrain the masking network. The difference is that it is used to train the network to extract masked image features; Used to constrain the shielding feature class center and the ordinary pedestrian feature class center to be close; To control the hyperparameters of the loss term weight, set it as required.

[0016] As an implementable preferred solution, generating clothing-related text prompts includes: Construct an initial text prompt template, where the fixed text is converted into a text embedding through CLIP's token embedding layer and concatenated with the learnable tensor to obtain a text embedding describing the clothing; Input the clothing image into the image encoder of CLIP to obtain visual features, and embed the text into the text encoder of CLIP to obtain text features; The learnable tensor is optimized to express clothing-related and identity-independent semantic information through text-image and image-text contrastive losses. The loss functions are as follows:

[0017] in, and is the image-text and text-image contrast loss used to train clothing text prompt generation; Use the generated clothing-related text prompts to guide the visual encoder training and design the loss function:

[0018] in, is the cross entropy loss, which is used to align clothing features and clothing text features; Used to constrain pedestrian features away from clothing features, To control the hyperparameters of the loss term weight, set it as required.

[0019] As an implementable preferred solution, the orthogonal loss is achieved by minimizing the cosine similarity between pedestrian features and clothing features:

[0020] in, represents the cosine similarity, Represents the pedestrian identity feature vector of the i-th sample; represents the clothing-related feature vector of the i-th sample; Indicates the number of samples in the current training batch.

[0021] As an implementable optimal solution, the Adam optimizer is used in the training process, combined with the cosine learning rate decay strategy to dynamically adjust the learning rate. In the first stage, the prompt learner is trained with a learning rate of 3.5e-4 and 120 training rounds; in the second stage, the image encoder and projection layer are fine-tuned with a learning rate of 5e-6 and 40 training rounds.

[0022] In the second aspect, the present invention also provides a pedestrian re-identification system for changing clothes based on a visual language pre-training model, which uses the above-mentioned pedestrian re-identification method for changing clothes based on a visual language pre-training model, including an invariant feature prompt module and a clothing feature minimization module.

[0023] As an implementable preferred solution, the invariant feature prompt module is used to extract clothing-independent semantic features from visual images; a human body parsing model is used to mask the clothing area of ​​the input image to generate a masked image; and a visual language model is used to generate identity-related text prompts that do not contain clothing information.

[0024] As an implementable optimal solution, the clothing feature minimization module is used to suppress the residual clothing information in the model; by generating text prompts describing the clothing, the model is guided to separate clothing-related features from the image; with the help of feature projection and discriminative loss function, the model is optimized to learn pedestrian identity features that are orthogonal to clothing features. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Schematic diagram of the costume-changing pedestrian re-identification framework based on the visual language pre-training model proposed in this invention.

[0026] Figure 2 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0027] Reference numerals: electronic device 500 , processor 501 , communication interface 502 , memory 503 , bus 504 . DETAILED DESCRIPTION

[0028] In order to make the technical solution and advantages of the present application clearer, the technical solution of the present invention will be further described in detail below with reference to the accompanying drawings. It will be understood that the specific embodiments described herein are only partial embodiments of the present invention, which are only used to explain the present application, rather than to limit the present application. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be considered to be isolated, and they can be combined with each other to achieve better technical effects. The same reference numerals appearing in the drawings of the following embodiments represent the same features or components, which can be applied to different embodiments.

[0029] In addition, unless otherwise defined, technical or scientific terms used in the description of the present invention should have the common meanings understood by those skilled in the art in the art to which the present invention belongs.

[0030] The present invention will be further described in detail below with reference to the accompanying drawings: The method for pedestrian re-identification after changing clothes based on a visual language pre-training model includes the following steps.

[0031] Step S100, inputting an image and performing clothing masking processing, including: Step S101: Obtain the original image containing the target pedestrian , the image size is adjusted to a unified parameter, and data enhancement strategies can be applied to improve image diversity.

[0032] In step S102, semantic segmentation is performed on the image using a human parsing network (e.g., the SCHP model, Self-Correction Human Parsing). The pedestrian image is divided into 20 semantic regions, including background, hair, hat, glasses, top, skirt, coat, pants, dress, T-shirt, face, left arm, right arm, left leg, right leg, left shoe, right shoe, etc.

[0033] Extract the pixels corresponding to the clothing category to generate a mask containing the clothing area .

[0034] Use the mask to set the pixels in the clothing area of ​​the original image to a constant (such as 255) to construct a masked image: .

[0035] On the contrary, retaining the pixels in the clothing area can obtain the clothing image: .

[0036] Step S200, generating a clothing-related text prompt, including: Step S201 involves a text prompt template, including: Use the initial text prompt template: "A photo of a [X]1... [X]M person”.

[0037] The fixed text A photo of a person is converted into text embeddings using CLIP’s built-in token embedding layer.

[0038] Fixed embeddings and learnable tensors are concatenated to form text embeddings that describe the corresponding identities.

[0039] [X]1... [X] M It consists of initialized learnable tensors, one for each identity.

[0040] Step S202, extracting text features, includes: The masked image is input into the image encoder of CLIP to obtain visual features.

[0041] Combined with an initial text prompt template containing learnable tokens, the concatenated text is input into the CLIP text encoder to obtain text features.

[0042] Optimization via text-image and image-text contrastive losses [X]1... [X] M The corresponding learnable tensor can learn to express semantic information related to identity but not related to clothing, making the text description reliable. The loss function is as follows:

[0043] in, and It is the image-text contrastive loss and text-image contrastive loss used to train identity text prompt generation.

[0044] Step S203: Use the generated clothing-independent text prompts to guide the visual encoder training and design the loss function:

[0045] in, is the identity loss, including cross entropy loss and triple loss, is the cross entropy loss, used to align pedestrian features and text features; To constrain the identity loss of the shielded network, The image-text cross entropy loss is used to constrain the masking network. The difference is that it is used to train the network to extract masked image features; Used to constrain the shielding feature class center and the ordinary pedestrian feature class center to be close; To control the hyperparameters of the loss term weight, set according to the requirements. In this embodiment, Set to 0.1.

[0046] Step S300, generating clothing-related text prompts, including: Step S301: designing a text prompt template, including: Using the initial text prompt template, "A photo of a [X]1... [X] M clothes”.

[0047] The fixed text A photo of a clothes is converted into text embedding through the token embedding layer that comes with CLIP.

[0048] [X]1... [X] M It consists of initialized learnable tensors, one for each set of clothing.

[0049] The fixed embedding and the learnable tensor are concatenated to form a text embedding describing the corresponding clothing item.

[0050] Step S302, extracting text features, includes: The clothing image is input into CLIP’s image encoder to obtain visual features.

[0051] Combined with an initial text prompt template containing learnable tokens, the concatenated text is fed into the CLIP text encoder to obtain text features. These features are optimized using text-image and image-text contrastive losses [X]1... [X] M The corresponding learnable tensor is used to learn to express semantic information related to clothing but not to identity, making the text description reliable. The loss function is as follows:

[0052] in, and is the image-text and text-image contrast loss used to train clothing text prompt generation; Step S303: Use the generated clothing-related text prompts to guide the visual encoder training and design the loss function:

[0053] in, is the cross entropy loss, which is used to align clothing features and clothing text features; Used to constrain pedestrian features away from clothing features, To control the hyperparameters of the loss term weight, set according to the requirements. In this embodiment, Set to 1.

[0054] Step S400, image-text alignment and image-image alignment training, includes: Step S401 , setting fixed parameters, fixing the weights of the text encoder and the weights of the learnable prompts.

[0055] Step S402, image-text alignment, includes: The textual prompts from step S200 are fed into a text encoder to obtain textual features for each identity. The textual prompts for all identities are used as identity prototypes, and a fixed-weight classifier is constructed to enable the visual encoder to encode clothing-independent pedestrian feature representations.

[0056] The cross entropy loss is used to encourage pedestrian feature representation to be correctly classified in the text classifier and achieve image-text alignment.

[0057] Step S403, image-to-image alignment, includes: Due to the inherent modality difference between image and text, an additional visual encoder is used to encode the image after masking clothes into pedestrian features.

[0058] Compute the Euclidean distance between the class center representations of the masked and normal images of the same identity:

[0059] in, and are the feature class centers of the original image and the masked image respectively, and N represents the number of samples in the current training batch, which is the number of image samples processed simultaneously during one forward propagation and backpropagation process.

[0060] The Euclidean distance is used to constrain the class center representations of masked images and normal images with the same identity within a batch to be close to each other.

[0061] Constrain the original image and the masked image to be consistent in the feature space to enhance the discriminability of clothing-independent pedestrian features:

[0062] in, is the predicted probability distribution of masked image features, represents the true identity label of the i-th sample, and N represents the number of samples in the current training batch.

[0063] Step S500: introducing a feature minimization module to further remove clothing-related features, including: Step S501: Extract image segments containing clothing regions from the original image. A learnable, globally optimized projection matrix is ​​introduced during visual encoder training. The pedestrian features are multiplied by the projection matrix to extract clothing features, projecting the clothing features from the overall pedestrian features.

[0064] Step S502: Classify clothing features to ensure that the extracted clothing features have good discriminability, including: Input all preset clothing category text prompts into the text encoder to extract the corresponding text features.

[0065] A clothing text classifier is constructed, and the projected clothing features are fed into the classifier for classification, and supervised optimization is performed using cross entropy loss.

[0066] Step S503: feature orthogonality constraint is applied to ensure that pedestrian features do not contain clothing features. Specifically, an orthogonal loss is introduced to penalize the similarity between pedestrian features and clothing features:

[0067] in, Represents the cosine similarity, so that the two maintain a distance in the feature space, thereby effectively eliminating clothing features. The pedestrian identity feature vector of the i-th sample is the deep feature representation extracted from the original image; The clothing-related feature vector of the i-th sample is the clothing feature projected and separated from the original pedestrian feature; Indicates the number of samples in the current training batch.

[0068] Step S600, the inference phase, includes: Step S601, feature extraction, uses only the trained and optimized image encoder to extract features from the input image, without calculating clothing-obscured images or text prompts, and without introducing additional computing overhead.

[0069] Step S602: Pedestrian search, calculate the cosine similarity between the query image features and the library image features: return the search results based on the similarity sorting.

[0070] This example implements a person re-identification method for changing clothes based on a visual language pre-training model on the PyTorch deep learning framework. All experiments are performed on an A100 GPU. The backbone network uses the CLIP image encoder, and the input image is uniformly adjusted to , and applied common data augmentation strategies to improve image diversity, including random horizontal flipping, padding, random cropping, and random erasing operations.

[0071] In the first stage, two prompt learners are trained with an initial learning rate of 3.5e-4, 120 training epochs, and a batch size of 64.

[0072] In the second phase, two image encoders and a projection layer are fine-tuned for 40 training epochs, with an initial learning rate of 5e-6. Both phases use the Adam optimizer, combined with a cosine learning rate decay strategy to dynamically adjust the learning rate. Each batch contains 64 images, corresponding to 16 pedestrian identities, with 4 images sampled for each identity. The hyperparameters and are set to 0.1 and 1, respectively.

[0073] Reference Figure 1 and Figure 2 The method of the present invention has been fully experimentally verified on multiple typical clothing-changing pedestrian re-identification datasets, achieving excellent recognition performance. On the PRCC dataset, the Rank-1 accuracy in the clothing change test scenario reached 70.4%, and the mAP value was 63.7%; on the LTCC dataset, the Rank-1 accuracy was 42.9%, and the mAP was 20.1%; on the VC-Clothes dataset, the Rank-1 accuracy reached 88.6%, and the mAP was 87.2%; on the DeepChange large-scale real-world monitoring dataset, the Rank-1 accuracy was 59.6%, and the mAP was 21.0%. The above results show that the method of the present invention can still maintain stable and accurate identity recognition capabilities under complex conditions of large scale, multiple scenes, and strong clothing changes, verifying its robustness and wide applicability in practical applications.

[0074] Table 1 Performance comparison of different methods on LTCC, PRCC and VC-Clothes datasets.

[0075]

[0076] Table 2 Performance comparison of different methods on the DeepChange dataset.

[0077]

[0078] This embodiment also provides a system for re-identifying pedestrians after changing clothes based on a visual language pre-training model, which can execute the above-mentioned method for re-identifying pedestrians after changing clothes based on a visual language pre-training model.

[0079] Reference Figure 1 The system framework consists of two core modules: the Invariant Feature Prompting Module (IFP) and the Clothing Feature Minimization Module (CFM). By combining multimodal information from images and language, the model is guided to learn clothing-independent discriminative identity features, thereby enhancing its recognition robustness under different clothing conditions.

[0080] IFP generates text prompts related to identity but not to clothing by mining the potential knowledge of the vision-language pre-training model CLIP; clothing feature minimization generates text prompts related to clothing for subsequent feature stripping.

[0081] The invariant feature hinting module extracts clothing-independent semantic features from visual images. It uses a human body parsing model to mask clothing regions in the input image to generate a masked image. It then leverages a visual language model to generate text hints that are relevant to the identity but exclude clothing information. Using image-text alignment and image-image alignment strategies, the model locates and enhances areas of the original image that are highly relevant to the identity, while ignoring the distracting effects of clothing.

[0082] The clothing feature minimization module suppresses residual clothing information in the model. By generating textual cues specifically describing clothing, the model is guided to isolate clothing-related features from the image. Then, using feature projection and a discriminative loss function, the model is optimized to learn pedestrian identity features that are orthogonal to clothing features. Through an iterative "detect-separate-weaken" approach, the module enhances the model's ability to learn clothing-irrelevant features through negative constraints.

[0083] This embodiment of the present application also provides an electronic device 500 that utilizes the aforementioned method for re-identifying a person wearing a different outfit based on a visual language pre-training model. The device includes a memory, a processor, and a computer program stored in the memory and executable by the processor. When the processor executes the program, the steps of the aforementioned method for re-identifying a person wearing a different outfit based on a visual language pre-training model are implemented. In this embodiment of the present application, the processor serves as the control center of the computer method and can be a processor of either a physical machine or a virtual machine.

[0084] Reference Figure 2 The electronic device 500 includes: at least one processor 501, at least one communication interface 502, at least one memory 503, and at least one bus 504. The bus 504 is used to implement connection and communication between these components, the communication interface 502 is used to communicate signaling or data with other node devices, and the memory 503 stores machine-readable instructions executable by the processor 501. When the electronic device 500 is running, the processor 501 communicates with the memory 503 via the bus 504. When the machine-readable instructions are called by the processor 501, the steps of the above-described method for pedestrian re-identification based on a visual language pre-training model are executed.

[0085] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor of an electronic device, it can implement the steps of the method for re-identifying pedestrians after changing clothes based on a visual language pre-training model as described above.

[0086] Those skilled in the art will understand that all or part of the processes in the method for re-identifying a person in a different outfit based on a visual language pre-training model can be implemented by instructing related hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of various embodiments of the method for re-identifying a person in a different outfit based on a visual language pre-training model. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0087] The above contents are merely embodiments of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. A person of ordinary skill in the art is aware of all common technical knowledge in the technical field to which the invention belongs before the filing date or priority date, is able to obtain all existing technologies in the field, and has the ability to apply conventional experimental means before that date. A person of ordinary skill in the art can, under the guidance of this application, improve and implement this scheme in combination with his or her own abilities. Some typical known structures or known methods should not become an obstacle for a person of ordinary skill in the art to implement this application. It should be pointed out that for a person of ordinary skill in the art, several variations and improvements can be made without departing from the structure of the present invention, which should also be regarded as the scope of protection of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection claimed in this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A method for re-identifying pedestrians after changing clothes based on a visual language pre-training model, characterized by: include: Input the image and perform the clothes masking process to obtain the masked image; Generate clothing-independent text prompts and clothing-related text prompts; Image-text alignment and image-image alignment training, including: Fixed weights of the text encoder and weights of the learnable prompts; The generated text prompt is input into the text encoder to obtain the text features of each identity. This is used as the identity prototype to construct a fixed-weight classifier. The cross-entropy loss is used to encourage the pedestrian features output by the visual encoder to be correctly classified in the classifier, thus achieving image-text alignment. An additional visual encoder is used to encode the masked image into pedestrian features. The Euclidean distance between the class centers of the masked image and the normal image of the same identity is calculated. The Euclidean distance is used to constrain the two to be close in the feature space to achieve image-to-image alignment. A feature minimization module is introduced to extract clothing-related features, including: Extracting image segments containing clothing areas from the original image, introducing a learnable global optimization projection matrix, and multiplying pedestrian features with the projection matrix to extract clothing features; Inputting a preset clothing category text prompt into a text encoder to obtain clothing text features, constructing a clothing text classifier, inputting the clothing features into the clothing text classifier for classification, and optimizing through cross entropy loss; Orthogonal loss is introduced to penalize the cosine similarity between pedestrian features and clothing features, so that the two maintain a distance in the feature space; In the inference phase, the trained and optimized image encoder is used to extract features from the input image; the cosine similarity between the query image features and the library image features is calculated, and the retrieval results are returned based on the similarity sorting.

2. The method for pedestrian re-identification based on a visual language pre-training model according to claim 1 is characterized in that: Input image and perform clothing masking processing, including: Get the original image containing the target pedestrian and resize the image to a uniform parameter; The original image is semantically segmented through a human body parsing network to obtain a mask containing a clothing area. The mask is used to set the clothing area pixels in the original image to a constant to obtain a shielded image. Otherwise, the clothing area pixels are retained to obtain a clothing image.

3. The method for pedestrian re-identification based on a visual language pre-training model according to claim 1 is characterized in that: Generate clothing-related text prompts, including: Construct an initial text prompt template, where the fixed text is converted into a text embedding through CLIP's token embedding layer and concatenated with the learnable tensor to obtain a text embedding describing the identity; Input the masked image into the image encoder of CLIP to obtain visual features, and embed the text into the text encoder of CLIP to obtain text features; The learnable tensor is optimized by text-image and image-text contrastive losses to express identity-related and clothing-independent semantic information. The loss functions are as follows: in, and is the image-text contrast loss and text-image contrast loss used to train identity text prompt generation; Use the generated clothing-independent text prompts to guide the visual encoder training and design the loss function: in, is the identity loss, including cross entropy loss and triplet loss, is the cross entropy loss, used to align pedestrian features and text features; To constrain the identity loss of the shielded network, The image-text cross entropy loss is used to constrain the masking network. The difference is that it is used to train the network to extract masked image features; Used to constrain the shielding feature class center and the ordinary pedestrian feature class center to be close; To control the hyperparameters of the loss term weight, set it as required.

4. The method for re-identifying pedestrians after changing clothes based on a visual language pre-training model according to claim 1 is characterized in that: Generate clothing-related text prompts, including: Construct an initial text prompt template, where the fixed text is converted into a text embedding through CLIP's token embedding layer and concatenated with the learnable tensor to obtain a text embedding describing the clothing; Input the clothing image into the image encoder of CLIP to obtain visual features, and embed the text into the text encoder of CLIP to obtain text features; The learnable tensor is optimized to express clothing-related and identity-independent semantic information through text-image and image-text contrastive losses. The loss functions are as follows: in, and is the image-text and text-image contrast loss used to train clothing text prompt generation; Use the generated clothing-related text prompts to guide the visual encoder training and design the loss function: in, is the cross entropy loss, which is used to align clothing features and clothing text features; Used to constrain pedestrian features away from clothing features, To control the hyperparameters of the loss term weight, set it as required.

5. The method for re-identifying pedestrians after changing clothes based on a visual language pre-training model according to claim 1 is characterized in that: The orthogonal loss is achieved by minimizing the cosine similarity between pedestrian features and clothing features: in, represents the cosine similarity, Represents the pedestrian identity feature vector of the i-th sample; represents the clothing-related feature vector of the i-th sample; Indicates the number of samples in the current training batch.

6. The method for re-identifying pedestrians after changing clothes based on a visual language pre-training model according to claim 5 is characterized in that: The Adam optimizer is used in the training process, and the learning rate is dynamically adjusted in combination with the cosine learning rate decay strategy. In the first stage, the prompt learner is trained with a learning rate of 3.5e-4 and 120 training rounds. In the second stage, the image encoder and projection layer are fine-tuned with a learning rate of 5e-6 and 40 training rounds.

7. A pedestrian re-identification system based on a visual language pre-training model, characterized by: A method for re-identifying pedestrians after changing clothes based on a visual language pre-training model as described in any one of claims 1 to 6 is used, comprising an invariant feature prompting module and a clothing feature minimization module.

8. The pedestrian re-identification system based on visual language pre-training model according to claim 7 is characterized in that: The invariant feature hint module is used to extract clothing-independent semantic features from visual images; A human body parsing model is used to mask the clothing area of ​​the input image to generate a masked image; a visual language model is used to generate text prompts related to the identity but without clothing information.

9. The pedestrian re-identification system based on visual language pre-training model according to claim 7 or 8, characterized in that: The clothing feature minimization module is used to suppress residual clothing information in the model; by generating textual prompts describing clothing, it guides the model to separate clothing-related features from the image; With the help of feature projection and discriminative loss function, the model is optimized to learn pedestrian identity features that are orthogonal to clothing features.

Images