Pedestrian re-identification method combined with text guidance

Abstract

The invention discloses a pedestrian re-identification method combined with text guidance, which comprises the following steps: collecting diversified pedestrian images, extracting image features of the images by using a deep convolutional neural network, directly generating text description from an original image through an image description model, extracting text features, and optimizing the comparison loss of the images and the text to obtain the pedestrian re-identification result. Aligning the visual features and the text features; re-recognition classification loss is optimized, it is ensured that the model can effectively distinguish identities of different pedestrians, the performance of the model is evaluated on a verification set, fine adjustment is conducted according to feedback, and the trained model is used for pedestrian recognition. According to the method, the training of the model is supervised by introducing text description, so that the limitation of a traditional pedestrian recognition technology on capturing detail features is solved. According to the method, text description (such as garment styles, colors, textures and the like) is used for guiding and optimizing the training process of a deep learning model, and it is ensured that the model can more accurately learn and recognize the detail features.

Description

Technical Field

[0001] The present invention relates to the field of image recognition research, and in particular to a pedestrian re-identification method combined with text guidance. Background Art

[0002] Pedestrian recognition technology has important application value in security, surveillance, and intelligent video analysis. Traditional pedestrian recognition methods mainly rely on deep learning technologies such as convolutional neural networks to identify individuals by extracting features from images. These technologies usually focus on extracting global features, such as body shape and posture, and local features, such as face and clothing.

[0003] Although existing technologies have made significant progress in pedestrian recognition, there are still limitations in capturing and identifying pedestrian details. For example, traditional methods may not be sensitive enough to accurately identify clothing styles, texture details, or hairstyles. These details are crucial for distinguishing different individuals or identifying the same pedestrian in a changing environment. Therefore, improving the ability to recognize these details is an important direction for the development of pedestrian recognition technology. Summary of the invention

[0004] In order to overcome the shortcomings of the existing technology, the present invention introduces text descriptions to supervise the training of the model, thereby solving the limitations of traditional pedestrian recognition technology in capturing detailed features. This method involves using text descriptions (such as descriptions of clothing styles, colors, textures, and other personal characteristics) to guide and optimize the training process of the deep learning model, ensuring that the model can learn and recognize these detailed features more accurately. The technical solution is as follows:

[0005] A pedestrian re-identification method combined with text guidance includes the following steps:

[0006] Step 1: Collect diverse pedestrian images to ensure coverage of different lighting, background, and posture conditions, providing rich and varied training data for the model.

[0007] Step 2: Capture the core visual information of pedestrians and use deep convolutional neural networks to extract image features.

[0008] In step 3, a text description is generated directly from the original image through the image description model, providing the model with additional, fine-grained semantic level information.

[0009] Preferably, the image description can be generated by combining the image model description with manual annotation to ensure the accuracy of the description.

[0010] Step 4: Use the text feature extraction model to process the generated text description, extract text features, and convert the semantic information in the text into a feature representation that matches the image features, with the same dimension as the image features.

[0011] Step 5: Calculate the contrast loss between the image and the text. By optimizing this loss, the model learns to align visual and text features, enhancing sensitivity to details and descriptive capabilities.

[0012] Preferably, the contrast loss calculation formula is as follows:

[0013]

[0014] where f i is the image feature, g i is the text feature, τ is the temperature parameter, sim is the similarity calculation function, and the specific formula is:

[0015] Preferably, the temperature parameter τ=2.

[0016] Step 6: Calculate the re-identification classification loss based on image features. This step ensures that the model can effectively distinguish the identities of different pedestrians and enhances the model's ability to be applied in actual scenarios.

[0017] Preferably, the re-identification classification loss formula is:

[0018]

[0019] where w i is the weight of the i-th column of the last linear classification layer of the model.

[0020] In step 7, the contrast loss and classification loss are weightedly combined to form the final loss function. Through this combination, the model can focus on the overall identity and detailed features of the pedestrian at the same time, improving the comprehensive recognition performance.

[0021] In step 8, the total loss is minimized by the optimization algorithm, and the model improves its recognition ability by learning the association between images and texts and pedestrian identity classification.

[0022] Preferably, the optimization algorithm uses the Adam optimizer with a learning rate of 1×10 -4 .

[0023] Step 9: Further improve the accuracy and robustness of the model by evaluating the model performance on the validation set and fine-tuning based on the feedback.

[0024] Step 10: After the model is trained, take out the image feature extraction model and extract features from the two pedestrian images m and n to obtain f m 、f n , calculate the similarity score of the feature = sim(f m ,f n ), when the similarity is less than the threshold, they are identified as the same person.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows: through the present invention, the training process of the pedestrian recognition model will pay more attention to the detailed features of the individual, thereby significantly improving the accuracy and meticulousness of recognition. The application of this technology can enhance the model's ability to recognize clothing texture, style and other key personal features, especially in complex backgrounds and changing environments. The recognition effect will be significantly improved. Therefore, this improvement will greatly enhance the effectiveness and reliability of pedestrian recognition in security, monitoring and other related applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Flowchart of the pedestrian re-identification method. DETAILED DESCRIPTION

[0027] In order to illustrate the technical solution and working principle of the present invention, the embodiments of the present invention will be described in further detail below. All the above optional technical solutions can be combined in any way to form optional embodiments of the present invention, which will not be described one by one here.

[0028] The terms "step 1", "step 2", "step 3", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those described herein.

[0029] The present disclosure provides a method for pedestrian re-identification combined with text guidance, comprising the following steps:

[0030] Step 1, image acquisition: Collect diverse pedestrian images to ensure coverage of different lighting, background and posture conditions, providing rich and varied training data for the model.

[0031] Step 2, image feature extraction: Use a deep convolutional neural network (such as ResNet-50) to extract the image feature f of image i i ∈R K , K is the feature dimension. The key to this step is to capture the core visual information of pedestrians and lay the foundation for deep feature understanding.

[0032] Step 3, image description generation: Generate text description t directly from the original image i through an image description model (such as the Transformer-based BLIP model) i ,These descriptions include pedestrians’ clothing, color, and other details, providing the model with additional, fine-grained semantic level information.

[0033] Image description can also be generated by combining image model description with manual annotation to ensure the accuracy of the description.

[0034] Step 4, text feature extraction: Use a text feature extraction model (such as the CLIP model) to process the generated text description t i , extract and text features g i ∈R K , converting the semantic information in the text into feature representations that match the image features, ensuring that g i Dimension and f i same.

[0035] Step 5, contrast loss calculation: The formula is as follows:

[0036]

[0037]

[0038] Where sim is the similarity calculation function, τ is the temperature parameter, and preferably, τ = 2. By optimizing this loss, the model learns to align visual and text features and enhances sensitivity to details and descriptive ability.

[0039] Step 6, re-identification loss calculation: Calculate the re-identification classification loss based on image features, using cross entropy loss:

[0040]

[0041] where w i The i-th column weight of the final linear classification layer of the model. This step ensures that the model can effectively distinguish the identities of different pedestrians and enhances the model's ability to be applied in actual scenarios.

[0042] In addition to classification, other re-identification losses can also be used.

[0043] Step 7, weighted loss synthesis: Combine the contrast loss and classification loss to form the final loss function:

[0044] L total =αL contrastive +βL classification

[0045] Through this combination, the model can focus on both the overall identity and detailed features of pedestrians, improving the overall recognition performance.

[0046] Step 8: Model training: Use an optimization algorithm (e.g., Adam optimizer, learning rate 1×10 -4 ) minimize the total loss L total,In this stage, the model improves its recognition ability by learning the ,association between images and texts and the identity classification of ,pedestrians.

[0047] Step 9, model optimization: Further improve the accuracy and robustness of the model by evaluating the model performance on the validation set and fine-tuning based on the feedback.

[0048] Step 10, model reasoning: After the model is trained, take out the image feature extraction model, extract features from two pedestrian images m and n respectively to obtain f m 、f n , calculate the similarity score of the feature = sim(f m ,f n ), when the similarity is less than the threshold, they are identified as the same person.

[0049] The above is an exemplary description of the present invention. Obviously, the specific implementation of the present invention is not limited to the above-mentioned method. All non-substantial improvements made by adopting the method concept and technical solution of the present invention; or directly applying the above-mentioned concept and technical solution of the present invention to other occasions without improvement or equivalent replacement, are within the protection scope of the present invention.

Claims

1. A pedestrian re-identification method combined with text guidance, characterized in that: The following steps are involved: Step 1: Collect diverse pedestrian images to ensure coverage of different lighting, background, and posture conditions, providing rich and varied training data for the model; Step 2: Capture the core visual information of pedestrians and use deep convolutional neural networks to extract image features; Step 3: Generate text descriptions directly from the original image through the image description model to provide the model with additional, fine-grained semantic level information; Step 4: Use the text feature extraction model to process the generated text description, extract text features, and convert the semantic information in the text into a feature representation that matches the image features, and its dimension is the same as the image features; Step 5: Calculate the contrast loss between the image and the text. By optimizing this loss, the model learns to align visual and text features, enhancing sensitivity to details and descriptive capabilities. Step 6: Calculate the re-identification classification loss based on image features. This step ensures that the model can effectively distinguish the identities of different pedestrians and enhances the model's application capabilities in actual scenarios. Step 7: Combine the contrast loss and classification loss by weight to form the final loss function. Through this combination, the model can focus on the overall identity and detailed features of the pedestrian at the same time, thus improving the comprehensive recognition performance. Step 8: By optimizing the algorithm to minimize the total loss, the model improves its recognition ability by learning the association between images and texts and the classification of pedestrian identities; Step 9: Further improve the accuracy and robustness of the model by evaluating the model performance on the validation set and fine-tuning based on the feedback; Step 10: After the model is trained, take out the image feature extraction model and extract features from the two pedestrian images m and n to obtain f m 、f n , calculate the similarity score of the feature = sim(f m ,f n ), when the similarity is less than the threshold, they are identified as the same person.

2. The method for person re-identification combined with text guidance according to claim 1, characterized in that: The contrast loss calculation formula is as follows: where f i is the image feature, g i is the text feature, τ is the temperature parameter, sim is the similarity calculation function, and the specific formula is:

3. The method for person re-identification combined with text guidance according to claim 2, characterized in that: Temperature parameter τ=2.

4. The method for person re-identification combined with text guidance according to claim 2, characterized in that: The re-identification classification loss formula is: where w i is the weight of the i-th column of the last linear classification layer of the model.

5. The method for person re-identification combined with text guidance according to claim 4, characterized in that: The optimization algorithm in step 8 uses the Adam optimizer with a learning rate of 1×10 -4 .

6. The method for person re-identification combined with text guidance according to claim 1, characterized in that: Step 3 generates an image description by combining the image model description with manual annotation to ensure the accuracy of the description.

Images