Shielding perception pedestrian re-identification method based on regional shielding and prompt distribution collaboration

Abstract

The invention relates to the technical field of computer vision, in particular to an occlusion perception pedestrian re-identification method based on regional occlusion and prompt distribution collaboration, and the method specifically comprises the following steps: collecting pedestrian images for processing, and then dividing the pedestrian images into a training set and a test set; constructing an occluded pedestrian re-identification model which comprises a text feature extraction module and an image feature extraction module which are arranged in parallel, and inputting pedestrian image data in the training set into the model to train the model; respectively optimizing the model through the loss function in the training process to obtain a trained model; verifying the performance of an image encoder in the trained model by using pedestrian images in the test set, and extracting retrieval features; and performing similarity matching on the retrieval features and features of pedestrian images in a retrieval library to obtain a pedestrian re-recognition result. According to the method, the sheltered pedestrian re-identification model is constructed and the performance of the sheltered pedestrian re-identification model is optimized, so that sheltering interference in pedestrian re-identification can be overcome, and the accuracy of sheltered pedestrian re-identification is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a method for occlusion-aware person re-identification based on cooperation of regional occlusion and prompt distribution. BACKGROUND

[0002] Person re-identification (ReID) is an important research direction in the field of computer vision, aiming to solve the problem of person identity recognition and retrieval across different camera scenes. This technology extracts the appearance features and deep semantic features of pedestrians, establishes a robust feature representation, and accurately matches pedestrians with the same identity in camera networks with different temporal and spatial distributions. With the development of deep learning technology, modern ReID systems can overcome practical challenges such as changes in lighting and differences in viewing angles, significantly improving the accuracy of cross-camera pedestrian tracking. This technology can be used in intelligent monitoring systems for applications such as suspect tracking and missing person search, and can be used in intelligent security systems to build city-level video monitoring networks to provide technical support for public security.

[0003] In real security scenarios, pedestrians often face three types of severe occlusion challenges: dynamic occlusion, which is caused by mutual occlusion between pedestrians and moving objects, leading to the loss of key identity features; static occlusion, which is caused by persistent body part occlusion from fixed facilities; and self-occlusion, which is caused by partial feature shielding caused by objects carried by pedestrians. Such occlusion phenomena lead to fundamental limitations of traditional ReID methods: feature integrity is destroyed, with the proportion of occluded regions in the original image reaching 30%-70%, making key discriminative regions invisible; noise interference is enhanced, with occluding objects introducing irrelevant interference features; and the false match rate increases, with the global feature pooling mechanism relied on by existing methods significantly decreasing in accuracy under occlusion scenarios.

[0004] In recent years, various solutions have been developed to address the problem of occluded person re-identification: for example, local feature alignment strategies focus on visible body parts, human parsing techniques are used to construct segmented feature representations, attention mechanisms are used to enhance non-occluded region responses, or generative adversarial networks are used to complete occluded features; another approach is to use feature purification strategies, using spatial mask mechanisms to suppress the feature contribution of occluded regions, thereby improving the robustness to occlusion interference. However, these methods still have significant limitations: local alignment strategies are prone to fragment misalignment under heavy occlusion, attention mechanisms are prone to focus shift due to occlusion boundary interference, and generative completion is prone to introduce semantically distorted features. To address the above bottlenecks, the present application proposes a method for occlusion-aware person re-identification based on cooperation of regional occlusion and prompt distribution, aiming to improve robustness in real occlusion scenarios.

[0005] Therefore, the present application proposes a method of occlusion-aware pedestrian re-identification based on the cooperation of regional occlusion and prompt distribution to solve the above problems. SUMMARY

[0006] The present application is directed to the deficiencies of the prior art, and develops a method of occlusion-aware pedestrian re-identification based on the cooperation of regional occlusion and prompt distribution. The present application constructs an occlusion pedestrian re-identification model and optimizes its performance, thereby overcoming the occlusion interference in pedestrian re-identification and improving the accuracy of occlusion pedestrian re-identification.

[0007] The technical solution of the present application for solving the technical problem is a method of occlusion-aware pedestrian re-identification based on the cooperation of regional occlusion and prompt distribution, comprising the following steps: S1. Collecting pedestrian images and performing text description, while labeling the true identity information of the pedestrian images, then pre-processing the pedestrian image data, and then dividing the pre-processed pedestrian image data into a training set and a test set; S2. Constructing an occlusion pedestrian re-identification model, including a text feature extraction module and an image feature extraction module arranged in parallel, inputting the pedestrian image data in the training set into the model to train the model; The text feature extraction module is constructed based on the CLIP-ReID model combining visual language model and pedestrian re-identification technology, and the text extraction module includes learnable text labels of specific identity ID, a unified sentence template, a text encoder, a dual prompt enhancement module and a text feature projection; The image feature extraction module includes a three-enhancement module, an image encoder, a bottleneck layer and a classifier; S3. In the training process, the text feature extraction module and the image feature extraction module in the occlusion pedestrian re-identification model are optimized by the loss function respectively until the change of the loss function tends to be stable, the training of the model is ended, and then a trained model is obtained; S4. The performance of the image encoder in the trained model is verified by the pedestrian images in the test set, the pedestrian images are input into the image encoder for feature extraction to obtain image feature vectors, and then the image feature vectors are projected to the feature space through the linear mapping layer and normalized to obtain retrieval features; S5. The retrieval features are matched with the features of the pedestrian images in the retrieval library in terms of similarity, the pedestrian images are sorted in order of similarity score from high to low, and the sorting result is output to obtain the pedestrian re-identification result.

[0008] S1 is as follows: Pedestrian images of different perspectives are collected by a multi-scene camera system, and corresponding text description information is labeled for each image; The pedestrian image is preprocessed, and the preprocessing operation includes size unification, horizontal random flip, random cropping and region priority erasing, wherein the region priority erasing is specifically performed by analyzing the real occlusion data, setting the human body region as the key erasing region, and using the erasing probability to control the erasing region; The preprocessed pedestrian image has corresponding text description information and real pedestrian identity information classification label for detection; Finally, the preprocessed pedestrian image dataset is divided into a training set and a test set.

[0009] The operation in the text feature extraction module is specifically as follows: The text feature extraction module has a text feature extraction method, one of which is an original text feature extraction method, and the other is an enhanced text feature extraction method; (1) The original text feature extraction method: the text description corresponding to the pedestrian image in the training set is input into the text encoder of the text feature extraction module to obtain text embedding, and the text embedding is projected to output the original text feature; (2) Enhanced text feature extraction method: the text description corresponding to the pedestrian image in the training set is input into the text feature extraction module, and a specific identity ID learnable text label is introduced for the text description of the pedestrian image. Each identity ID is independent, and a set of token vectors is assigned to each identity ID. The token vector is embedded into a unified sentence template to obtain a text prompt word of the pedestrian image; The text prompt word is input into the dual prompt enhancement module to obtain an enhanced prompt word, and the enhanced prompt word is input into the text encoder to obtain an enhanced embedding; Finally, the enhanced embedding is projected to obtain a text enhanced feature.

[0010] The dual prompt enhancement module is specifically as follows: The text prompt word is input into the dual prompt enhancement module, and first passes through the prompt learner of the CLIP-ReID model to learn the prompt word. A set of basic prompt vectors is generated according to the class labels of the prompt words; Then, based on the preset enhancement type and enhancement parameter in the dual prompt enhancement module, the basic prompt vector set is subjected to double independent data enhancement operation to generate an enhanced prompt word, and the enhanced prompt word includes a first enhanced prompt word set and a second enhanced prompt word set, Wherein, the preset enhancement type includes mask, exchange or mixing operation, and the preset enhancement parameter is the parameter setting in the enhancement type; According to the first enhanced prompt word set and the second enhanced prompt word set, a text contrast learning loss is calculated to optimize the dual prompt enhancement module.

[0011] The image feature extraction module is specifically as follows: The image feature extraction mode in the image feature prompting module includes an original image feature extraction mode and an enhanced image feature extraction mode. (1) The original image feature extraction mode: inputting the pedestrian image in the training set into the image encoder of the text feature extraction module to output the original image feature. (2) The enhanced image feature extraction mode: performing an enhancement operation on the basis of the original image feature, performing enhancement processing on the original image feature through a three-enhancement module, the form of the original image feature being a feature sequence, the three-enhancement module including a feature erasing unit, a feature transformation unit and a feature noise adding unit, and the operation of the three-enhancement module being specifically as follows: The feature erasing unit: randomly positioning the feature sequence of the original image feature, performing local feature erasing on the positioned continuous region, taking the lower half region as a priority erasing region, and controlling the erasing position selection through a probability threshold; then, creating a binary mask matrix, setting the feature values of the selected continuous region to zero, and generating erasing enhanced features. The feature transformation unit: randomly selecting two non-overlapping continuous regions in the feature sequence of the original image feature, replacing the feature vectors of the first region with the corresponding feature vectors of the second region, performing feature space transformation through position exchange, and generating transformation enhanced features. The feature noise adding unit: randomly selecting a continuous region in the feature sequence of the original image feature, injecting random noise conforming to a Gaussian distribution into the region, controlling the disturbance amplitude through a noise intensity coefficient, and generating noise-added enhanced features. The erasing enhanced features, the transformation enhanced features and the noise-added enhanced features constitute three-enhanced features. Then, the three-enhanced features are respectively input to the classifier after dimension reduction by the bottleneck layer. The three-enhanced features are respectively input to the classifier after dimension reduction by the bottleneck layer, and corresponding identity classification prediction results are generated.

[0012] The occluded pedestrian re-identification model is optimized and trained through calculation of a loss function, and the specific process is as follows: The total loss is composed of a text feature loss of the text feature extraction module , a comprehensive loss of the image feature extraction module , an image-text contrast loss , an adversarial loss function and a triplet loss The occluded pedestrian re-identification model is optimized through the total loss until the variation region of the result of each training of the total loss function is gentle, the training of the occluded pedestrian re-identification model is ended, and the best performance occluded pedestrian re-identification model is obtained. The loss function of the text feature extraction module is composed of three parts, which are image-text contrast loss, text-image contrast loss and text-text contrast loss. The calculation of the loss function of the image feature extraction module adopts a step-by-step adversarial optimization strategy, including two stages, the first stage is classifier adversarial pre-optimization, and the second stage is global feature joint optimization.

[0013] The loss function calculation process of the text feature extraction module is as follows: According to the original image features and the original text features, the image-text contrast loss and the text-image contrast loss are calculated, and according to the text enhanced features corresponding to the first prompt word set and the text enhanced features corresponding to the second prompt word set, the text-text contrast loss is calculated; The similarity between the identity categories of the original image features and the identity categories of the original text features, and the similarity between the identity categories of the two text enhanced features are calculated, and the image-text contrast loss, the text-image contrast loss and the text-text contrast loss are further calculated through the class cross-entropy loss; The three losses are added to obtain the text feature loss The proportion of loss addition is determined according to its contribution.

[0014] The loss function calculation process of the image feature extraction module is as follows: First stage: Synchronous execution of classifier parameter update in the forward propagation process of the image feature extraction module, and calculation of the original classification loss of the original image features and the original text features through cross-entropy loss ; The classification loss of the triple enhanced features is calculated through cross-entropy loss, which specifically calculates the erasure loss between the identity classification prediction result of the erasure enhanced feature and the actual identity, the transformation loss between the identity classification prediction result of the transformation enhanced feature and the actual identity, and the noise enhanced feature identity classification prediction result and the actual identity, and set negative weight coefficients for each loss; Add the original classification loss and the classification loss of the triple enhanced features with negative weight coefficients to obtain the adversarial loss function ; Update the classifier through the gradient back propagation of the adversarial loss function , and keep the updated parameters for subsequent global optimization; Second stage: Based on the output of the first stage, the comprehensive loss of the multi-modal features is calculated , and the comprehensive loss The identity recognition loss The triplet loss And the enhanced feature regularization loss ; Wherein, the identity recognition loss And the triplet loss Are respectively obtained by calculating the original image features after different processing and the original text features through cross-entropy loss; the enhanced feature regularization loss Is composed of the original classification loss And the classification loss of the triple enhanced feature with a set positive weight coefficient.

[0015] The effects provided in the summary of the invention are only the effects of the embodiments, not all the effects of the invention, and the above technical solutions have the following advantages or beneficial effects: The application discloses a kind of based on region occlusion and prompt distribution cooperation's occlusion perception pedestrian re-identification method, its core innovation is in constructing triple anti-interference cooperative architecture; In the data processing module, a region priority erasing strategy is adopted, a region priority erasing module for data preprocessing is used to simulate a real occlusion scene in an occlusion scene, and a certain probability of structured occlusion simulation is applied to the lower body region to generate physical occlusion enhanced samples.The present module is based on prior knowledge and analysis of real scene, and realizes pixel-level occlusion simulation, which accurately matches the real scene from the picture level. In the text feature extraction module, a double prompt enhancement module is constructed, a mask / exchange / mix triple prompt word perturbation strategy is adopted, a text prompt missing situation in an occlusion scene is simulated, and a text-level occlusion simulation is constructed.When the text information is missing due to image occlusion, the model can still learn effective feature representation through contrast loss constraint. In the image feature extraction module, a triple enhancement module is designed to implement parallel enhancement of feature erasing, feature transformation and noise injection, and an anti-occlusion feature is generated by bottleneck layer projection.The present application constructs an end-to-end anti-occlusion feature learning framework through triple enhancement and step-by-step optimization, and realizes feature-level occlusion simulation.A step-by-step adversarial optimization mechanism is designed in the optimization process of the image feature extraction module, the classifier parameters are updated with a negative weight loss in the first stage, and the whole network is optimized with a positive weight regularization in the second stage.The present application decouples the complex joint optimization task from the simple classification task through the step-by-step optimization strategy, and breaks through the limitation of the separation of feature enhancement and model optimization in traditional occlusion ReID methods.

[0016] In summary, the present application can overcome the occlusion interference in pedestrian re-identification, improve the robustness of the model, and further improve the accuracy of occluded pedestrian re-identification and the generalization ability of the model in the occlusion scene. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and are meant to explain the present application and are not intended to limit the application.

[0018] Figure 1 A flow chart of the method of the present application.

[0019] Figure 2 An experimental comparison chart of the method of the present application and a baseline model. DETAILED DESCRIPTION

[0020] In order to clearly illustrate the technical features of the present application, the present application will be described in detail below with reference to the specific embodiments and the accompanying drawings.

[0021] Example 1 A occlusion-aware pedestrian re-identification method based on region occlusion and prompt distribution cooperation, comprising the following steps: S1. Collecting pedestrian images and performing text description, while labeling the real identity information of the pedestrian images, then pre-processing the pedestrian image data, and then dividing the pre-processed pedestrian image data into a training set and a test set; S2, constructing an occlusion pedestrian re-identification model, including a text feature extraction module and an image feature extraction module arranged in parallel, inputting the pedestrian image data in the training set into the model to train the model; The text feature extraction module is constructed based on the CLIP-ReID model combining the visual language model and the pedestrian re-identification technology, and the text extraction module includes a learnable text label of a specific identity ID, a unified sentence template, a text encoder, a double prompt enhancement module and a text feature projection; The image feature extraction module includes a three-enhancement module, an image encoder, a bottleneck layer and a classifier; S3. In the training process, the text feature extraction module and the image feature extraction module in the occlusion pedestrian re-identification model are optimized by the loss function respectively until the change of the loss function tends to be stable, the training of the model is ended, and then a trained model is obtained; S4. Verify the performance of the image encoder in the trained model with the pedestrian images in the test set, input the pedestrian images into the image encoder for feature extraction, obtain the image feature vector, and then project the image feature vector through the linear mapping layer to the feature space for normalization processing to obtain the retrieval feature; S5. Similarity matching is performed between the retrieval feature and the features of the pedestrian images in the retrieval library, the pedestrian images are sorted in order from high to low according to the similarity score, and the sorting result is output to obtain the pedestrian re-identification result.

[0022] In a specific implementation, S1 is as follows: The system uses a multi-scene camera system to capture pedestrian images from different perspectives and adds corresponding text descriptions to each image. The pedestrian images are preprocessed, including size unification, horizontal random flipping, random cropping, and region-priority erasure. Specifically, region-priority erasure is achieved by analyzing real-world occlusion data and designating areas less than 30% of the human body as key erasure areas. An erasure probability of p=0.7 is used to control the erasure area, while retaining 30% of the probability to preserve the model's generalization ability. The preprocessed pedestrian images have corresponding text description information and classification labels for real pedestrian identity information used for detection; Finally, the preprocessed pedestrian image dataset was divided into a training set and a test set in a 7:3 ratio.

[0023] In a specific implementation, the operations in the text feature extraction module are as follows: The text feature suggestion module includes two text feature extraction methods: the original text feature extraction method and the enhanced text feature extraction method. (1) Original text feature extraction method: Input the text description corresponding to the pedestrian image in the training set into the text encoder of the text feature extraction module to obtain the text embedding, perform text feature projection on the text embedding, and output the original text features; (2) Enhanced text feature extraction method: Input the text description corresponding to the pedestrian image in the training set into the text feature extraction module, introduce learnable text tags with specific identity IDs for the text description of the pedestrian image. Each identity ID is independent. Assign a set of token vectors to each identity ID, embed the token vectors into a unified sentence template, and obtain the text prompt words of the pedestrian image. The text prompts are input into the dual prompt enhancement module to obtain the enhanced prompts, and then the enhanced prompts are input into the text encoder to obtain the enhanced embedding; Finally, text feature projection is performed on the enhanced embedding to obtain the text enhancement features; The unified sentence template is "one". "Photos of people", among which, Indicates the first A learnable text tag This represents the total number of learnable text tags. The text encoder is the text encoder part in the CLIP-ReID model.

[0024] In a specific implementation, the dual-prompt enhancement module is as follows: The text prompt word is input into the dual prompt enhancement module, first through the prompt learner of the CLIP-ReID model to learn the prompt word, and a set of basic prompt vectors is generated according to the category label of each prompt word; Then, based on the preset enhancement type and enhancement parameter in the dual prompt enhancement module, the basic prompt vector set is subjected to double independent data enhancement operation to generate enhanced prompt words, the enhanced prompt words including a first enhanced prompt word set and a second enhanced prompt word set, Wherein, the preset enhancement type includes mask, exchange or mixing operation, and the preset enhancement parameter is the parameter setting in the enhancement type; The text contrast learning loss is calculated according to the first enhanced prompt word set and the second enhanced prompt word set, the dual prompt enhancement module is optimized, and the robustness of the occluded pedestrian re-identification model to the category feature representation is further optimized.

[0025] When the enhancement type is a mask, the double independent data enhancement operation is as follows: For each prompt vector in the basic prompt vector set, two different mask matrices are independently randomly generated, and the mask matrix determines the zero position according to the preset proportion threshold; The first mask matrix is used to perform a position-by-position zero operation on the basic prompt vector to generate the first enhanced prompt word set; The second mask matrix is used to perform a position-by-position zero operation on the basic prompt vector to generate the second enhanced prompt word set.

[0026] In the specific implementation, the image feature extraction module is as follows: The image feature extraction module has an image feature extraction method, one of which is the original image feature extraction method, and the other of which is the enhanced image feature extraction method; (1) The original image feature extraction method: input the pedestrian image in the training set into the image encoder of the text feature extraction module to output the original image feature; (2) Enhanced image feature extraction method: performing enhancement operation on the original image feature, and enhancing the original image feature through the three enhancement modules, the form of the original image feature being a feature sequence, the three enhancement modules including a feature erasing unit, a feature transformation unit and a feature noise adding unit, and the operation of the three enhancement modules being as follows: Feature erasing unit: randomly positioning the feature sequence of the original image feature, performing local feature erasing on the positioned continuous region, taking the lower half region as the priority erasing region, and controlling the erasing position selection through a probability threshold; then, a binary mask matrix is created, the feature values of the selected continuous region are set to zero, and an erasing enhanced feature is generated; Feature transformation unit: randomly select two non-overlapping continuous regions in the feature sequence of the original image features, replace the feature vector of the first region with the corresponding feature vector of the second region, perform feature space transformation through position exchange, and generate transformed enhanced features; Feature noise adding unit: randomly select a continuous region in the feature sequence of the original image features, inject random noise conforming to a Gaussian distribution into the region, control the disturbance amplitude through a noise intensity coefficient, and generate noise-enhanced features; The erasing enhanced features, the transformed enhanced features, and the noise-enhanced features constitute triple enhanced features. Then, the triple enhanced features are respectively input to the classifier after being reduced in dimension by the bottleneck layer, The triple enhanced features are respectively input to the classifier after being reduced in dimension by the bottleneck layer,

[0027] In the specific implementation, the occluded pedestrian re-identification model is optimized and trained by calculating the loss function, and the specific process is as follows: The total loss is composed of the text feature loss of the text feature extraction module , the comprehensive loss of the image feature extraction module , the image-text contrast loss , the adversarial loss function , and the triplet loss . The occluded pedestrian re-identification model is optimized by the total loss until the result change area of the total loss function is smooth each time the training is performed, the training of the occluded pedestrian re-identification model is ended, and the best performance occluded pedestrian re-identification model is obtained. The loss function of the text feature extraction module is composed of three parts, which are: image-text contrast loss, text-image contrast loss, and text-text contrast loss. The calculation of the loss function of the image feature extraction module adopts a step-by-step adversarial optimization strategy, including two stages, the first stage is classifier adversarial pre-optimization, and the second stage is global feature joint optimization.

[0028] The calculation formula of the total loss is as follows: .

[0029] In the specific implementation, the loss function calculation process of the text feature extraction module is as follows: According to the original image features and the original text features, the image-text contrast loss and the text-image contrast loss are calculated, and according to the text enhanced features corresponding to the first set of prompt words and the text enhanced features corresponding to the second set of prompt words, the text-text contrast loss is calculated. The similarity between the identity categories of the original image features and the original text features, as well as the similarity between the identity categories of the two text-enhanced features, is calculated. Furthermore, the image-text contrast loss, text-image contrast loss, and text-text contrast loss are calculated using category cross-entropy loss. The three losses are summed to obtain the text feature loss. The percentage of losses when they are added together is determined by their contribution.

[0030] The formula for calculating similarity is as follows: , in, This represents the vector dot product operation; Representing image features, Representing text features; Indicates the similarity between image features and text features; The formula for calculating image-text contrast loss is as follows: , in, Indicates identity label Image features, Indicates identity label Textual features, express and Similarity between Indicates batch size. Indicates batch indexing. express In batch Text features of an image express and Similarity between Indicates identity label Image-to-text contrast loss; The formula for calculating text-image contrast loss is as follows: , The formula for calculating text-to-text contrast loss is as follows: , in, Indicates in Features of the target text in a batch The set of all matched positive samples. Indicates in Features of the target text in a batch Any matched positive sample, Image features representing the matched elements. represents the text feature in the batch, represents the image feature of the matched element, represents the text feature in the batch, represents the contrastive loss of the text to image with the identity label ; represents the contrastive loss of the text to text with the identity label ; The three loss functions are added to obtain the text feature loss function : .

[0031] In the specific implementation, the loss function calculation process of the image feature extraction module is as follows: The first stage: The classifier parameter update is performed synchronously in the forward propagation process of the image feature extraction module, and the original classification loss of the original image feature and the original text feature is calculated through the cross-entropy loss ; The classification loss of the triple enhanced feature is calculated through the cross-entropy loss, and the erasing loss between the identity classification prediction result of the erasing enhanced feature and the actual identity , the transformation loss between the identity classification prediction result of the transformation enhanced feature and the actual identity and the noise added loss between the identity classification prediction result of the noise added enhanced feature and the actual identity , and the negative weight coefficients are set for each loss; The original classification loss and the classification loss of the triple enhanced feature with the set negative weight coefficient are added to obtain the adversarial loss function ; The classifier is updated through the gradient back propagation of the adversarial loss function , and the updated parameters are reserved for subsequent global optimization; The calculation formula is as follows: , wherein, , and represent the weight coefficients, and are set as , , ; The second stage: The comprehensive loss is calculated based on the multi-modal feature of the output of the first stage , and the comprehensive loss including an identity recognition loss , a triplet loss and an enhanced feature regularization loss ; wherein the identity recognition loss and the triplet loss are respectively obtained by calculating the cross-entropy loss between the features of the original image features after different processing and the original text features; the enhanced feature regularization loss is composed of the original classification loss and the classification loss of the triple-enhanced features with a set positive weight coefficient.

[0032] The calculation of the loss function of the second stage is as follows: , .

[0033] Embodiment 2 To prove the significant progress of the method proposed in the present application, the method of the present application and the baseline model (i.e. only using the CLIP-ReID network) are compared on the Occluded-Duke (Occluded Duke is a data set proposed in “Pose-Guided Feature Alignment for Occluded Person Re-Identification”) data set, and the retrieval is performed on the Occluded-Duke data set using the method of the present application and the baseline method respectively, as shown in Figure 2 , the left side is the retrieval result of the baseline method, and the right side is the retrieval result of the method of the present application, the first image of each row in the left and right sides is a query sample, the baseline method and the method of the present application use the same query sample, for each query sample, the similarity distance with it is calculated in the retrieval library, and the top ten retrieval results (i.e. Rank-1 to Rank-10) are returned, in the visualization result, the correct retrieval result is marked with a green footnote, and the incorrect retrieval result is marked with a red footnote; According to Figure 2 it can be observed that, compared with the baseline model, the correct results retrieved by the method of the present application are not only more in the ranking, but also more in the overall correct retrieval quantity, which fully shows that the proposed occlusion-aware person re-identification method based on region occlusion and prompt distribution cooperation can effectively improve the retrieval performance under the occlusion condition, and the extracted features have good discriminability and robustness.

[0034] In addition, the experimental results of the method of the present application and the existing method on the public unoccluded data set and the occluded data set can also be compared, and Table 1 is as follows: Table 1 Comparison results of the method of the present application and the existing method on multiple data sets The method in Table 1 is the latest existing method in the field from 2022 to 2025, the data sets Market1501 and DukeMTMC are two public data sets without occlusion, the method of the application is an occlusion-aware pedestrian re-identification method, and the significance of selecting two non-occluded data sets for comparison here requires that the premise of each method for occlusion-aware pedestrian re-identification is that the performance index when performing non-occluded pedestrian re-identification needs to reach a certain standard; the data sets Occluded-Duke, Occluded-ReID and Partial-ReID are three public data sets with occlusion; the evaluation index is mAP (a key index for measuring the overall performance of the model) and rank-1 (rank-1 indicates the first ranked search result, and the data in the table is the accuracy of the first ranked search result); As can be seen from Table 1, the average precision mAP of the method of the application on the three occluded data sets achieves an effect of 80.6, and the average rank-1 accuracy achieves an effect of 87.1, both of which are better than other existing methods, which can prove that the application can realize more reliable pedestrian re-identification in real scenes, and can greatly improve the effect of pedestrian re-identification in the occlusion scene, and while focusing on improving the effect of occluded pedestrian re-identification, the overall pedestrian re-identification effect can still be in a good position.

[0035] Although the specific embodiments of the application have been described above with reference to the drawings, it is not a limitation on the protection scope of the application, and various modifications or variations made by those skilled in the art on the basis of the technical solutions of the application without creative labor are still within the protection scope of the application.

Claims

1. A method for occlusion-aware pedestrian re-identification based on region occlusion and cue distribution collaboration, characterized in that, The method comprises the following steps: S1. Collecting pedestrian images and text descriptions, labeling the real identity information of the pedestrian images, preprocessing the pedestrian image data, and then dividing the preprocessed pedestrian image data into a training set and a test set; S2, constructing a blocked pedestrian re-identification model, including a text feature extraction module and an image feature extraction module arranged in parallel, inputting the pedestrian image data in the training set into the model to train the model; The text feature extraction module is constructed based on a CLIP-ReID model combining a visual language model and a pedestrian re-identification technology, and the text extraction module includes a learnable text label of a specific identity ID, a unified sentence template, a text encoder, a double prompt enhancement module, and a text feature projection; The image feature extraction module includes a three-enhancement module, an image encoder, a bottleneck layer, and a classifier; S3. In the training process, the text feature extraction module and the image feature extraction module in the blocked pedestrian re-identification model are optimized by the loss function respectively until the change of the loss function tends to be stable, the training of the model is ended, and then a trained model is obtained; S4. Verify the performance of the image encoder in the trained model with the pedestrian images in the test set, input the pedestrian images into the image encoder for feature extraction, obtain the image feature vector, and then project the image feature vector through the linear mapping layer to the feature space for normalization processing to obtain the retrieval feature; S5. Similarity matching is performed between the retrieval feature and the features of the pedestrian images in the retrieval library, the pedestrian images are sorted in descending order of similarity score, and the sorting result is output to obtain the pedestrian re-identification result.

2. The method of claim 1, wherein the method is characterized in that, S1 is as follows: Different angle pedestrian images are collected by a multi-scene camera system, and corresponding text description information is labeled for each image; The pedestrian image is preprocessed, and the preprocessing operation includes size unification, horizontal random flip, random cropping, and region priority erasing, wherein the region priority erasing is specifically performed by analyzing the real blocked data, setting the human body region as the key erasing region, and using erasing probability to control the erasing region; The preprocessed pedestrian image has corresponding text description information and real pedestrian identity information classification label for detection; Finally, the preprocessed pedestrian image dataset is divided into a training set and a test set.

3. The method of claim 2, wherein the text The operation in the feature extraction module is as follows: There are two text feature extraction methods in the text feature prompt module, one is the original text feature extraction method, and the other is the enhanced text feature extraction method; (1) The original text feature extraction method: input the text description corresponding to the pedestrian image in the training set into the text encoder of the text feature extraction module to obtain text embedding, perform text feature projection on the text embedding, and output the original text feature; (2) Enhanced text feature extraction method: input the text description corresponding to the pedestrian image in the training set into the text feature extraction module, introduce a learnable text label of a specific identity ID for the text description of the pedestrian image, each identity ID is independent, assign a group of token vectors to each identity ID, embed the token vectors into a unified sentence template to obtain a text prompt word of the pedestrian image; input the text prompt word into the dual prompt enhancement module to obtain an enhanced prompt word, and then input the enhanced prompt word into the text encoder to obtain an enhanced embedding; Finally, the enhanced embedding is projected to obtain a text enhanced feature.

4. The method of claim 3, wherein the method is characterized in that, The dual prompt enhancement module is as follows: input the text prompt word into the dual prompt enhancement module, first pass through the prompt learning of the CLIP-ReID model to learn the prompt word, generate a set of basic prompt vectors according to the category label of each prompt word; Then, based on the preset enhancement type and enhancement parameter in the dual prompt enhancement module, the basic prompt vector set is subjected to double independent data enhancement operation to generate an enhanced prompt word, the enhanced prompt word includes a first enhanced prompt word set and a second enhanced prompt word set, Wherein, the preset enhancement type includes mask, exchange or mixing operation, and the preset enhancement parameter is the parameter setting in the enhancement type; According to the first enhanced prompt word set and the second enhanced prompt word set, the text contrast learning loss is calculated, and the dual prompt enhancement module is optimized.

5. The method of claim 4, wherein the image is obtained by a camera mounted on a vehicle. The feature extraction module is as follows: There are image feature extraction methods in the image feature prompt module, one is the original image feature extraction method, and the other is the enhanced image feature extraction method; (1) Original image feature extraction method: input the pedestrian image in the training set into the image encoder of the text feature extraction module to output the original image feature; (2) Enhanced image feature extraction method: on the basis of the original image feature, the original image feature is enhanced by the three enhancement modules, the form of the original image feature is a feature sequence, and the three enhancement modules include a feature erasing unit, a feature transforming unit and a feature adding noise unit, and the operation of the three enhancement modules is as follows: Feature erasing unit: randomly positioning the feature sequence of the original image feature, performing local feature erasing on the positioned continuous region, taking the lower half region as the priority erasing region, and controlling the erasing position selection by a probability threshold; then, a binary mask matrix is created, the feature values of the selected continuous region are set to zero, and an erasing enhanced feature is generated; Feature transformation unit: randomly select two non-overlapping continuous regions in the feature sequence of the original image feature, replace the feature vectors of the first region with the corresponding feature vectors of the second region, perform feature space transformation by position exchange, and generate a transformed enhanced feature; Feature adding noise unit: randomly select a continuous region in the feature sequence of the original image feature, inject random noise conforming to a Gaussian distribution into the region, control the disturbance amplitude by a noise intensity coefficient, and generate a noise-added enhanced feature; The erasing enhanced feature, the transformed enhanced feature and the noise-added enhanced feature constitute a triple enhanced feature; Then, the triple enhanced features are respectively input to the classifier after dimension reduction by the bottleneck layer, The triple enhanced features are respectively input to the classifier after dimension reduction by the bottleneck layer, to generate corresponding identity classification prediction results.

6. The method of claim 5, wherein the method is characterized by, The loss function is used to optimize and train the occluded pedestrian re-identification model, and the specific process is as follows: The total loss is composed of a text feature loss of the text feature extraction module , a comprehensive loss of the image feature extraction module , an image-text contrast loss , an adversarial loss function , and a triplet loss . The occluded pedestrian re-identification model is optimized through the total loss until the result change area of the total loss function is gentle each time the training is performed, the training of the occluded pedestrian re-identification model is ended, and the best performance occluded pedestrian re-identification model is obtained. The loss function of the text feature extraction module is composed of three parts, which are image-text contrast loss, text-image contrast loss and text-text contrast loss. The calculation of the loss function of the image feature extraction module adopts a step-by-step adversarial optimization strategy, including two stages, the first stage is classifier adversarial pre-optimization, and the second stage is global feature joint optimization.

7. The method of claim 6, wherein the method is characterized by, The loss function calculation process of the text feature extraction module is as follows: According to the original image features and the original text features, the image-text contrast loss and the text-image contrast loss are calculated, and according to the text enhanced features corresponding to the first prompt word set and the text enhanced features corresponding to the second prompt word set, the text-text contrast loss is calculated; The similarity between the identity categories of the original image features and the original text features, and the similarity between the identity categories of the two text enhanced features are calculated, and the image-text contrast loss, the text-image contrast loss and the text-text contrast loss are further calculated through the class cross-entropy loss; The three losses are added together to get the text feature loss The proportion of each loss when added together is determined by its contribution.

8. The method of claim 7, wherein the method is characterized by, The loss function calculation process of the image feature extraction module is as follows: First stage: The classifier parameter updating is executed synchronously in the forward propagation process of the image feature extraction module, and the original classification loss of the original image feature and the original text feature is calculated through cross-entropy loss ; The classification loss of the triple enhanced features is calculated through a cross-entropy loss, and specifically, an erasing loss between an identity classification prediction result of the erasing enhanced features and an actual identity is calculated , a transformation loss between an identity classification prediction result of the transformation enhanced features and the actual identity , and a noise adding loss between an identity classification prediction result of the noise adding enhanced features and the actual identity , and negative weight coefficients are set for each loss. Original classification loss The adversarial loss function is obtained by adding the classification loss of the triple-enhanced features with negative weights to the sum of the classification loss and ... ; by an adversarial loss function Gradient backpropagation updates the classifier and keeps the updated parameters for subsequent global optimization; Second stage: Computing a comprehensive loss based on the output of the first stage , the comprehensive loss includes an identity loss , a triplet loss , and an enhanced feature regularization loss ; Wherein, the identity recognition loss and the triple loss are obtained by calculating the cross-entropy loss between the features obtained by processing the original image features and the original text features respectively. regularization loss from the original classification loss is added to the classification loss of the triple enhanced features with the positive weight coefficient set.

Images