Pedestrian re-identification method based on dual token decoupling and soft strip part feature extraction

Abstract

The present application relates to a kind of pedestrian re-identification method based on double Token decoupling and soft strip site feature extraction, belong to computer vision and deep learning technical field.The method introduces double Token countermeasure decoupling mechanism in global feature level, and fuses soft strip space bias in local feature level, effectively breaks through the performance bottleneck of existing visual language model in complex scene feature coupling, local alignment difficult.In the premise of not increasing the modal input burden of inference stage, the present application significantly improves the robustness and re-identification precision of model to real monitoring interference factors such as occlusion, illumination and posture change, provides strong technical support for efficient landing of cross-modal pedestrian re-identification technology in real scene such as intelligent security.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision and deep learning technology, and relates to the field of pedestrian re-identification, specifically to a pedestrian re-identification method based on dual-token decoupling and soft strip feature extraction. Background Technology

[0002] Person re-identification (ReID) aims to retrieve specific pedestrians across camera scenes, and this technology is widely used in intelligent surveillance and security. Due to variations in background, lighting, pose, and even occlusion, ReID has always been a highly challenging task. Traditional ReID models primarily rely on convolutional neural networks (CNNs) or visual transformers (ViTs) to extract visual features and are heavily dependent on pre-training on datasets with specific, fixed labels, such as ImageNet. However, conventional pre-training methods only utilize predefined category labels in the dataset, failing to leverage the richer and more nuanced semantic information within the images.

[0003] In recent years, vision-language models have demonstrated outstanding multimodal understanding and powerful zero-shot classification capabilities. For example, CLIP, through contrastive learning on large-scale image-text pairs, aligns visual and linguistic representations into a shared embedding space, providing a novel approach to extracting semantically rich pedestrian features. ReID methods utilizing vision-language models can be broadly categorized into two types: one relies entirely on images and self-generated pseudo-descriptions without requiring manual text annotation throughout the training process; the other involves manually annotating text labels or using multimodal large-scale language models to generate image-text pairs for large-scale pedestrian datasets.

[0004] In current research, the first type of method, which achieves image-text alignment without relying on artificial text and through self-generated semantics, is gradually becoming one of the mainstream directions in ReID due to its low cost, easy scalability, and strong robustness. CLIP-ReID, one of the most representative methods, proposes a two-stage strategy to perform cross-modal learning without text labels, matching visual features with their linguistic descriptions. The core idea is to assign a set of learnable text tokens to each pedestrian ID to generate a fuzzy description. In the first stage, the encoder is fixed, optimizing only the text tokens; in the second stage, both the text tokens and the text encoder are fixed, providing cross-modal constraints for fine-tuning the image encoder.

[0005] Despite the progress made in existing technologies, the following major problems still exist in practical applications: (1) Lack of decoupling ability for interference factors: Existing methods usually encode pedestrian identity features and interference features (such as occlusion, lighting changes, and pose differences) in the same feature vector, which leads to a decrease in feature discrimination in complex scenes.

[0006] (2) Loss of local fine-grained features: The visual encoder in the existing visual language model has difficulty accurately capturing the prior vertical structure of human body parts, resulting in insufficient attention of the model to human body parts and loss of local fine-grained features. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a pedestrian re-identification method based on dual-token decoupling and soft strip feature extraction, which improves upon the shortcomings of existing pre-trained CLIP models and specifically addresses the following technical problems: (1) Solving the problem of coupling between identity features and interference factors: Overcoming the defect that existing pre-trained CLIP models are difficult to distinguish between identity information and interference information. By introducing interference tokens and combining them with adversarial training strategies, the image encoder is driven to actively strip away and forget non-identity features, thereby achieving cross-modal semantic decoupling of dual tokens.

[0008] (2) Solve the problem of local fine-grained feature loss: overcome the problem that existing networks lack prior knowledge of human spatial structure. Without destroying the global receptive field, the model improves its ability to extract fine-grained features of human body parts (such as head, upper body, etc.) by using soft strip bias and vertical order prior guidance.

[0009] To achieve the above objectives, the present invention provides the following technical solution: Solution 1: A pedestrian re-identification method based on dual-token decoupling and soft-strip feature extraction. This method introduces interference tokens, assigning a set of learnable interference tokens to each ID to capture the semantics of various interference factors in the image. By introducing text decoupling loss and adversarial training based on gradient backward layer (GRL), the model is driven to actively forget interference factors and retain only stable identity features. A feature extraction module is designed to extract K feature parts from ViT patch tokens. By adding vertical order priors and soft-strip bias, attention is guided towards specific feature parts rather than hard constraints, achieving efficient extraction of fine-grained local features.

[0010] The method specifically includes the following steps: S1. Obtain the occluded pedestrian re-identification dataset, perform data augmentation (such as cropping, flipping, etc.) and segmentation; initialize the deep learning environment and load a pre-trained visual language model (such as CLIP) as the basic feature extractor; S2. Perform the first stage of training (i.e., dual-token text semantic decoupling): Freeze the image encoder and text encoder, and construct an identity description and interference description for each pedestrian ID; by comparing the loss and text decoupling loss, optimize only the learnable text tokens, including the identity token and interference token, to complete the dual-token semantic dictionary learning. S3. Perform the second stage of training, including simultaneous visual feature adversarial decoupling and soft stripe feature extraction; the visual feature adversarial decoupling involves freezing the learned text token and text encoder, unfreezing the image encoder, and introducing an identity projection head, a gradient inversion layer (GRL), and a non-identity projection head to enable the image encoder to actively reduce interference information; the soft stripe feature extraction utilizes vertical structure priors and soft stripe bias to extract part features, and combines joint loss to optimize the image encoder; S4. Joint Reasoning Test: Discard the text branch, input the test set image to be queried into the optimized image encoder, extract the cls token output by the image encoder as a global feature, and concatenate it with the local aggregated global vector; complete the pedestrian re-identification retrieval by calculating the Euclidean distance or similarity with features of other image libraries.

[0011] S5. Model Evaluation and Comparison: Based on the matching results of the test set, calculate evaluation metrics such as Rank-k accuracy and mean average precision (mAP), and verify the robustness and accuracy improvement of the model in complex scenarios by combining visualization results.

[0012] Furthermore, in step S2, the contrast loss includes: Image-to-text contrast loss for identity token The expression is:

[0013] in, B Indicates batch size. Represents cosine similarity. Indicates the first The semantic prototype of an identity. Indicates the first A semantic prototype that interferes with an identity; The global visual features output by the image encoder; Text-to-image contrast loss for identity tokens The expression is:

[0014] in, This indicates all pairs of elements in the current training batch. A set of image indexes belonging to the same identity.

[0015] Furthermore, in step S2, the text decoupling loss Used to separate identity tokens and interference tokens in semantic space, its expression is:

[0016] in, N To train the total number of pedestrian identity categories, Represents cosine similarity. The text embedding represents the identity of the c-th pedestrian. This indicates the embedding of interfering text. This is the decoupling interval threshold.

[0017] Furthermore, in step S3, the visual feature adversarial decoupling specifically includes: for the input image The image encoder outputs global visual features. :

[0018]

[0019]

[0020] in, Indicates an image encoder. Z This is the output of the last layer of ViT. The CLStoken output from the last layer of ViT. These are the patch tokens output from the last layer of ViT. W It is a linear projection matrix. for Z The output after projection for The output after projection for The output after projection; ViT stands for Visual Transformer, CLS Token is the classification token, and Patch Token is the image patch token.

[0021] Then, identity features are obtained through the identity projection head. for:

[0022] in, Identification projection head; Persistent prediction features are obtained through a gradient inversion layer and a non-identity projection head:

[0023] in, Indicates a non-identified projection head. It is a gradient inversion layer; Gradient-backward layers keep the input constant during forward propagation:

[0024] During backpropagation, the gradient is multiplied by a negative coefficient:

[0025] in, λ To counteract the strength coefficient, I It is the identity matrix; The identity branch continues to apply cross-modal constraints through the identity text prototype:

[0026] in, For the first i Image-to-identity text cross-entropy loss for each sample; N To train the total number of pedestrian identity categories; For the first k The unique hot tags for each category are: 1 for the "Real Identity" category and 0 for the rest. Non-identity branches perform perturbation semantic prediction by perturbing text prototypes:

[0027] in, The cross-entropy loss is the image-to-interference text cross-entropy loss for the i-th sample. Embed the corresponding category of distracting text. Embed candidate interference text; because The input passes through a gradient inversion layer (GRL) and a non-identity projection head. It was optimized to identify interfering semantics, while the image encoder During backpropagation, it is subjected to the opposite gradient, thus forcing a reduction in global features. Interference information that can be identified by non-identified projection heads. The optimization objective of enabling the image encoder to actively reduce interference information:

[0028] To ensure the ability to re-identify pedestrians, a conventional identity classification loss is also incorporated. and triplet loss :

[0029]

[0030] in, For classifiers based on the first i Global features of each sample The probability of predicting the true identity category. To and Positive sample features belonging to the same identity Features of negative samples with different identities. m The interval is a triplet. d ( , ) represents the Euclidean distance.

[0031] Furthermore, in step S3, the soft stripe feature extraction specifically includes: assuming the patch tokens output by the last layer of the image encoder are: , ,in, , H and W These represent the height and width of the patch grid, respectively. d Indicates the feature dimension; No. n The normalized coordinates of each patch in the vertical direction are:

[0032] in, = indicates the first n The row index where each patch is located. ; Divide the human body along the vertical direction into K Soft tissues, such as the head, upper body, lower body, and feet. k The center position of each part Defined as: ,

[0033] Set an independent learnable query vector for each part Calculate the first k The query for the first part and the first n The basic attention score between each patch token :

[0034] in, and These are the linear projection matrices of the query and the key, respectively; To incorporate prior knowledge of the human vertical structure, a soft-strip bias in the form of a Gaussian superposition of attention scores is applied. :

[0035] in, Indicates the bias strength of the soft strip. Indicates the first k The width parameter of the stripe for each part; unlike hard segmentation, this bias does not force a patch to belong to only a certain part, but increases the attention probability of the corresponding vertical region in the form of continuous weights, allowing overlap between adjacent parts, thereby enhancing the fault tolerance in the case of pose changes, detection box offsets and occlusion.

[0036] Attention weights after adding soft strip bias for:

[0037] The patch tokens are weighted and aggregated according to attention weights to obtain the first... k Features of each part To further form a locally aggregated global vector, K Features of each part Perform splicing and linear projection:

[0038] Where [;] denotes vector concatenation operation. Represents the local aggregation projection matrix. This represents the local aggregation feature of the part branch output.

[0039] Furthermore, in step S3, in order to enhance the identity discrimination capability of each part feature, a part triplet loss is introduced for each part feature. :

[0040] in, This represents the first sample of the same identity. k Features of each part The first represents the different identity samples k Features of each part; At the same time, for local aggregation features Apply identity classification loss:

[0041] in, To represent the local aggregated features of the identity classifier based on the i-th sample The sample is predicted to be its true identity category. The probability of.

[0042] Furthermore, in step S4, the model inference test specifically includes: recording the global visual features output by the image encoder. for Record the local aggregated features output by the soft strip region feature extraction module. for global features and local aggregation features By splicing the components together, the final fused feature is obtained:

[0043] Perform final fusion features Normalization: for query images and candidate library images Normalized features were obtained respectively. and Calculate the Euclidean distance between the two. for:

[0044] Finally, the candidate images are sorted according to their distance from smallest to largest, and the sorted results are output as the pedestrian re-identification results.

[0045] Solution 2: A pedestrian re-identification system based on dual-token decoupling and soft strip feature extraction, comprising: The visual and language encoder employs a pre-trained visual-language model, including an image encoder and a text encoder, to extract global visual features from the input pedestrian image and patch tokens from the last layer output of the image encoder.

[0046] The dual-token decoupling learning module is used to achieve semantic separation of identity features and interference features. This module configures a set of learnable identity tokens and a set of learnable interference tokens for each pedestrian identity in the training set, and constructs identity text prompts and interference text prompts based on preset prompt templates. The text encoder then obtains the identity text prototype and interference text prototype. The module is trained in two stages: In the first stage, the image encoder and text encoder are frozen, and only the identity tokens and interference tokens are optimized. Identity image to text contrast loss, identity text to image contrast loss, and text decoupling loss that constrains the orthogonality of identity text prototypes and interference text prototypes are used to decouple identity semantics from interference semantics. In the second stage, the text encoder and the learned dual tokens are frozen, the image encoder is unfrozen, and an identity projection head, a gradient inversion layer, and a non-identity projection head are introduced. When the image encoder is backpropagated through the gradient inversion layer, it receives the opposite gradient, actively weakening the interference information in the global visual features and achieving adversarial decoupling of visual features.

[0047] The soft strip feature extraction module is used to extract fine-grained local features of the human body without disrupting the global receptive field. This module divides the patch tokens output by the image encoder along the vertical direction.K For each soft body part, a learnable part query vector is set. A base attention score is calculated between the part query and each patch token, and a Gaussian soft strip bias is superimposed to guide the model to focus on vertical regions of the human body such as the head, upper body, lower body, and feet through soft attention. The attention weights are then weighted and aggregated to obtain... K Features of each part, K The features of each part are spliced ​​together and linearly projected to obtain local aggregated features.

[0048] The joint inference module is used to concatenate and fuse global visual features and local aggregated features during the testing phase. After normalization, it calculates the Euclidean distance between the query image and the candidate images and completes the ranking and matching to achieve pedestrian re-identification. During the inference phase, only the trained image encoder, global feature branch and soft strip feature branch are retained, without relying on the text encoder and dual tokens.

[0049] The beneficial effects of this invention are as follows: (1) Significantly enhances robustness to interference factors such as lighting, occlusion, and pose. Traditional methods cannot distinguish between useful identity information and harmful interference information in images. This invention introduces independent, learnable interference tokens and performs adversarial optimization in the second stage using a gradient inversion layer, forcing the image encoder to strip away interference semantics in the feature space. This allows the model to actively forget non-identity features such as occlusions and lighting changes, greatly improving the model's generalization ability in complex scenes.

[0050] (2) Enhancing the model's ability to extract local fine-grained features. Existing ViT-based methods lack structural guidance when extracting local features. This invention introduces a soft strip bias mechanism, which, without destroying the global receptive field of the Transformer, endows the network with prior knowledge of the vertical structure of the human body. At the same time, combined with Part Triplet Loss and local aggregated ID supervision, the model's ability to discriminate local details is greatly enhanced.

[0051] (3) Inference is efficient and requires no additional text input. Since the complex double token construction and text alignment are completed during the training phase, inference only requires extracting the global features of the image's cls token and the part branch aggregation features and concatenating them to complete the matching. High-performance person re-identification is achieved without adding extra modal input burden.

[0052] In summary, this invention effectively overcomes the performance bottleneck of existing visual language models in complex scenarios by introducing a dual-token adversarial decoupling mechanism at the global feature level and fusing soft-strip spatial bias at the local feature level. This addresses the challenges of feature coupling and local alignment in complex scenes. Without increasing the modal input burden during the inference stage, this invention significantly improves the model's robustness and re-identification accuracy against real-world monitoring interference factors such as occlusion, illumination, and pose changes. This provides strong technical support for the efficient implementation of cross-modal pedestrian re-identification technology in real-world scenarios such as intelligent security.

[0053] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0054] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a diagram illustrating the overall architecture of the pedestrian re-identification system based on dual-token decoupling and soft strip feature extraction according to the present invention. Figure 2 This is a flowchart of the pedestrian re-identification method based on dual-token decoupling and soft strip feature extraction of the present invention. Detailed Implementation

[0055] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0056] Example 1: Please see Figure 1 This embodiment provides a pedestrian re-identification system based on dual-token decoupling and soft strip feature extraction, whose structure mainly consists of the following core parts: (1) Visual and speech encoders: As the basic feature extractor, an image encoder and a text encoder are used, which are based on a visual language model (a CLIP model based on ViT) pre-trained on ImageNet.

[0057] (2) Dual-Token Decoupled Learning Module: This module is used to separate identity features from interference features in two-stage training.

[0058] Suppose there are N pedestrian identity categories in the training set, for the input image Its identity label is , No. The identity token corresponding to each identity is denoted as:

[0059] No. c The interference token corresponding to each identity is denoted as:

[0060] in, M This indicates the number of learnable tokens in each prompt template.

[0061] Based on the aforementioned Token, construct identity text prompts and distracting text prompts:

[0062]

[0063] Input the two types of text prompts into the CLIP text encoder separately. This yields the identity text prototype and the interference text prototype:

[0064]

[0065] in, Indicates the first The semantic prototype of an identity. Indicates the first A semantic prototype of interference with identity.

[0066] 1) First stage: Decoupling of dual-token textual semantics The first stage freezes the image encoder and text encoder, optimizing only the identity token and interference token. For the input image... Its identity label is The image encoder outputs global visual features:

[0067] Identity tokens are aligned with corresponding identity image features through image-text comparison learning, enabling identity text prototypes to represent stable identity information.

[0068] The image-to-text contrast loss for identity tokens is defined as:

[0069] in, B Indicates batch size. This represents the cosine similarity.

[0070] Since multiple images in a training batch may belong to the same pedestrian identity, for the text-to-image direction, multiple images of the same identity need to be treated as positive samples. Let... Indicates batch p All of the above For a set of image indexes belonging to the same identity, the text-to-image comparison loss of the identity token is:

[0071] in, This represents the number of samples in the positive sample set. The loss is based on identity text features. Use it as an anchor point to simultaneously approximate the features of multiple images with the same identity.

[0072] To separate identity tokens and distractor tokens in the semantic space, a text decoupling loss is introduced. This loss constrains the identity text prototype and the distractor text prototype to be as orthogonal as possible, preventing them from learning repetitive semantics.

[0073] in, This is the decoupling interval threshold. When the similarity between the identity text prototype and the interfering text prototype is below the threshold, the penalty is stopped; when the similarity is too high, the loss is increased to force them to separate.

[0074] 2) Second stage: Visual feature adversarial decoupling The second stage fixes the text encoder and the learned dual tokens, unfreezes the image encoder, and introduces an identity projection head, a gradient inversion layer (GRL), and a non-identity projection head, enabling the image encoder to actively reduce interference information.

[0075] For the input image The image encoder outputs global visual features. :

[0076]

[0077]

[0078] Where Z is the output of the last layer of ViT. The CLS token output from the last layer of ViT. These are the patch tokens output from the last layer of ViT. W It is a linear projection matrix. This is the output after projection.

[0079] Next, identity characteristics are obtained through the identity projection head:

[0080] in, The projector head indicates identity.

[0081] Persistent prediction features are obtained through a gradient inversion layer and a non-identity projection head:

[0082] in, Indicates a non-identified projection head. This is a gradient inversion layer.

[0083] Gradient-backward layers keep the input constant during forward propagation:

[0084] Multiply the gradient by a negative coefficient during backpropagation:

[0085] in, λ To counteract the strength coefficient, I It is an identity matrix.

[0086] The identity branch continues to apply cross-modal constraints through the identity text prototype:

[0087] Non-identity branches perform perturbation semantic prediction by perturbing text prototypes:

[0088] because The input passes through a gradient inversion layer (GRL) and a non-identity projection head. It was optimized to identify interfering semantics, while the image encoder During backpropagation, it is subjected to the opposite gradient, thus forcing a reduction in global features. Interference information that can be identified by non-identified projection heads. This process forms the following minimum-maximum optimization objective:

[0089] To ensure the ability to identify pedestrians again, both conventional identity classification loss and triplet loss are added:

[0090]

[0091] in, To and Positive sample features belonging to the same identity represents the negative sample features for different identities, and m is the triplet margin. d ( , ) represents the Euclidean distance.

[0092] (3) Soft strip feature extraction module: The soft strip feature extraction module addresses the issues of ViT models lacking prior knowledge of the human body's vertical structure and insufficient local feature extraction. Instead of rigid segmentation, this module introduces learnable part queries and Gaussian soft strip biases into the patch tokens output by ViT, enabling the model to focus on human body regions such as the head, upper body, lower body, and feet using soft attention.

[0093] Suppose the patch tokens output by the last layer of the image encoder are: , ,in, , H and W These represent the height and width of the patch grid, respectively, and d represents the feature dimension.

[0094] No. n The normalized coordinates of each patch in the vertical direction are:

[0095] in, = indicates the row index of the nth patch. .

[0096] The human body is divided vertically into K soft tissue parts, such as the head, upper body, lower body, and feet. The center position of the kth part is defined as: ,

[0097] Set an independent learnable query vector for each part Calculate the base attention score between the k-th part query and the n-th patch token:

[0098] in, and These are the linear projection matrices of the query and the key, respectively.

[0099] To incorporate prior knowledge of the human vertical structure, a soft-strip bias in the form of a Gaussian is applied to the attention scores:

[0100] in, Indicates the bias strength of the soft strip. This represents the width parameter of the strip at the k-th region. Unlike hard segmentation, this bias does not force a patch to belong only to a certain region. Instead, it increases the attention probability of the corresponding vertical region in the form of continuous weights, allowing overlap between adjacent regions, thereby enhancing the fault tolerance under pose changes, bounding box offsets, and occlusion conditions.

[0101] The attention weights after adding soft strip bias are:

[0102] The patch tokens are weighted and aggregated based on attention weights to obtain... k Features of each part:

[0103] To further form a local-to-global vector, the features of K regions are concatenated and linearly projected:

[0104] Where [;] denotes vector concatenation operation. Represents the local aggregation projection matrix. This represents the local aggregation feature of the part branch output.

[0105] To enhance the identity discrimination capability of each feature part, a feature part triplet loss is introduced for each feature part:

[0106] in, This represents the k-th feature of a sample with the same identity. This represents the k-th part feature of samples with different identities.

[0107] At the same time, for local aggregation features Apply identity classification loss:

[0108] (4) Joint Reasoning Module: The joint inference module is used to generate the final pedestrian re-identification features during the testing phase and to perform similarity matching between the Query image and the Gallery image. Unlike the training phase, the inference phase no longer uses the text encoder, identity token, and interference token; it only retains the trained image encoder, global feature branch, and soft stripe feature branch, thus avoiding additional text input overhead.

[0109] Record the global visual features output by the image encoder. for Record the local aggregated features output by the soft strip region feature extraction module. for global features and local aggregation features By splicing the components together, the final fused feature is obtained: ; Perform final fusion features Normalization: for query images and candidate library images Normalized features were obtained respectively. and Calculate the Euclidean distance between the two: .

[0110] Finally, the test set images are sorted in ascending order of distance, and the sorting result is output as the pedestrian re-identification result.

[0111] Example 2: Please see Figure 2 This embodiment provides a pedestrian re-identification method based on dual-token decoupling and soft strip feature extraction, specifically including the following steps: Step 1: Dataset Preparation and Preprocessing Obtain pedestrian re-identification datasets (such as Market1501, Occluded-Duke, etc.), perform basic data augmentation on the image data (such as random cropping, horizontal flipping, etc.), and divide the data into training set, query set, and candidate set (Gallery) according to standard protocols. Initialize the deep learning environment on the computing platform and load the pre-trained CLIP model (such as ViT-B / 16).

[0112] Step 2: Dual-Token Decoupling Learning (First Stage Training) Step 2.1: Fix the CLIP image and text encoder Step 2.2: For each image in the training set, construct two Prompt texts. Identity description: "A photo of a [ID1] [ID2]... [IDm] person"; Disturbance description: "A photo of a person with [DIS1][DIS2]...[DISm] disturbance".

[0113] Step 2.3: Calculate the contrast loss between the image and the text, and at the same time apply the text decoupling loss to optimize the learnable [ID] and [DIS] tokens to ensure that identity semantics and interference semantics are orthogonally isolated.

[0114] Step 3: Adversarial Fine-tuning of Image Features (Second Stage Training) Step 3.1: Freeze all text tokens and text encoders learned in the first stage, and begin fine-tuning the image encoder.

[0115] Step 3.2: After the global feature g1 output by the image encoder, connect the non-identity feature projection head that has passed through the gradient inversion layer (GRL).

[0116] Step 3.3: Calculate the adversarial alignment loss. Using a min-max optimization framework, the image encoder updates in the direction of removing interfering features, while the projector updates in the direction of extracting interfering features, driving the image encoder to actively forget occlusion and other interfering factors in the feature space.

[0117] Step 4: Feature extraction and joint supervision of soft strip regions (performed simultaneously in step 2): Step 4.1: Preset K body part queries (e.g., Q head, Q upper body, etc.).

[0118] Step 4.2: Generate a spatial bias matrix based on the y-coordinate of the patch grid, calculate the bias and add it to the attention score to achieve soft strip bias.

[0119] Step 4.3: Aggregate the extracted K Part features and generate a local aggregated global vector g2 through linear projection.

[0120] Step 4.4: Calculate the location feature average triplet loss and the local aggregated ID loss to supervise network convergence.

[0121] Step 5: Model Inference and Matching Input the image to be queried, and extract the cls token output by the image encoder as the global feature g1.

[0122] Extract the local aggregation vector g2 output after the part branch.

[0123] By concatenating g1 and g2 along the channel dimension, the Euclidean distance to features in other image databases is calculated to complete the re-identification retrieval.

[0124] Step Six: Model Evaluation and Performance Verification To verify the superiority of the present invention, a comparative experiment was conducted between the model of the present invention and the baseline model (CLIP-ReID) under the same hardware and software environment and dataset.

[0125] The results show that the model of this invention achieves the best performance in both Rank-1 accuracy and mAP (mean average precision), significantly outperforming the original baseline model. Especially in complex occlusion scenarios, this model exhibits strong robustness, effectively reducing false positives and false negatives caused by interference from lighting, pose, background changes, and occlusion. This demonstrates that the dual-token decoupling and soft-strip feature mechanism introduced in this invention can efficiently and accurately complete cross-modal feature alignment and re-identification tasks.

[0126] Example 3: Alternatives to the image and text encoding modules: This invention is not limited to specific visual language pre-trained models. Although the basic embodiment may use standard OpenAI CLIP (such as ViT-B / 16), in scenarios with higher requirements for feature granularity and recognition accuracy, it can be completely replaced by CLIP variants with larger parameters and stronger alignment capabilities (such as EVA2-CLIP-L) or other pre-trained large visual language models such as ALIGN.

[0127] Example 4: Alternatives to decoupling mechanisms: Simple orthogonal constraints can be used to force identity features and interfering features to be unrelated. However, orthogonal constraints are only static geometric restrictions, while the GRL adversarial training used in this scheme can achieve dynamic semantic stripping, and the robustness improvement is more significant in complex occlusion scenarios.

[0128] Example 5: Alternatives to site extraction methods: Option A (Hard Strip Cutting): Rigidly cuts the feature map like the PCB method. Option A performs poorly when pedestrian pose changes drastically or alignment is inaccurate; while the soft strip offset of this option allows attention to overlap between regions, providing higher fault tolerance.

[0129] Solution B (Mask-Guided): Utilizes human semantic segmentation maps to guide attention. Alternative Solution B requires additional labeled data or a pre-trained segmentation model; this solution utilizes soft constraints achieved through vertical order priors, requiring no additional labeling and offering greater versatility.

[0130] Example 6: Alternatives to the Prompt format: Hand-designed fixed templates can be used. However, fixed templates have limited expressive power and cannot adapt to the unique distribution of pedestrian IDs; using learnable tokens can more accurately capture the semantic prototypes of pedestrian identities and various interfering factors.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A pedestrian re-identification method based on dual-token decoupling and soft strip feature extraction, characterized in that, The method includes the following steps: S1. Obtain the occluded pedestrian re-identification dataset, perform data augmentation and segmentation; initialize the deep learning environment, and load the pre-trained visual language model as the basic feature extractor; S2. Perform the first stage of training, namely dual-Token text semantic decoupling: freeze the image encoder and text encoder, and construct an identity description and interference description for each pedestrian ID; by comparing the loss and text decoupling loss, only optimize the learnable text tokens, including identity tokens and interference tokens, to complete the dual-Token semantic dictionary learning. S3. Perform the second stage of training, including simultaneous visual feature adversarial decoupling and soft stripe feature extraction; the visual feature adversarial decoupling involves freezing the learned text token and text encoder, unfreezing the image encoder, and introducing an identity projection head, a gradient inversion layer, and a non-identity projection head to enable the image encoder to actively reduce interference information; the soft stripe feature extraction utilizes vertical structure priors and soft stripe bias to extract part features, and combines joint loss to optimize the image encoder. S4. Joint Reasoning Test: Discard the text branch, input the test set image to be queried into the optimized image encoder, extract the cls token output by the image encoder as a global feature, and concatenate it with the local aggregated global vector; complete the pedestrian re-identification retrieval by calculating the Euclidean distance or similarity with features of other image libraries.

2. The pedestrian re-identification method based on dual-token decoupling and soft strip feature extraction according to claim 1, characterized in that, In step S2, the contrast loss includes: Image-to-text contrast loss for identity token The expression is: in, B Indicates batch size. Represents cosine similarity. Indicates the first The semantic prototype of an identity. Indicates the first A semantic prototype that interferes with an identity; The global visual features output by the image encoder; Text-to-image contrast loss for identity tokens The expression is: in, Indicates batch p All of the above A set of image indexes belonging to the same identity.

3. The pedestrian re-identification method based on dual-token decoupling and soft strip feature extraction according to claim 1, characterized in that, In step S2, the text decoupling loss Used to separate identity tokens and interference tokens in semantic space, its expression is: in, N To train the total number of pedestrian identity categories, Represents cosine similarity. Indicates the first c Embedded personal identification text of each pedestrian This indicates the embedding of interfering text. This is the decoupling interval threshold.

4. The pedestrian re-identification method based on dual-token decoupling and soft strip feature extraction according to claim 2, characterized in that, In step S3, the visual feature adversarial decoupling specifically includes: for the input image The image encoder outputs global visual features. : in, Indicates an image encoder. Z This is the output of the last layer of ViT. The CLStoken output from the last layer of ViT. These are the patch tokens output from the last layer of ViT. W It is a linear projection matrix. for Z The output after projection for The output after projection for The output after projection; ViT stands for Visual Transformer, CLS Token is the classification token, and Patch Token is the image patch token; Then, identity features are obtained through the identity projection head. for: in, Identification projection head; Persistent prediction features are obtained through a gradient inversion layer and a non-identity projection head: in, Indicates a non-identified projection head. It is a gradient inversion layer; Gradient-backward layers keep the input constant during forward propagation: During backpropagation, the gradient is multiplied by a negative coefficient: in, λ To counteract the strength coefficient, I It is the identity matrix; The identity branch continues to apply cross-modal constraints through the identity text prototype: in, For the first i Image-to-identity text cross-entropy loss for each sample; N To train the total number of pedestrian identity categories; For the first k The unique hot tags for each category are: 1 for the "Real Identity" category and 0 for the rest. Non-identity branches perform perturbation semantic prediction by perturbing text prototypes: in, For the first i Image-to-interference-text cross-entropy loss for each sample Embed the corresponding category of distracting text. Embed candidate interference text; The optimization objective of enabling the image encoder to actively reduce interference information is: To ensure the ability to re-identify pedestrians, a conventional identity classification loss is also incorporated. and triplet loss : in, Predict the probability of the true identity category output by the classifier. To and Positive sample features belonging to the same identity Features of negative samples with different identities. m The interval is a triplet. d ( , ) represents the Euclidean distance.

5. The pedestrian re-identification method based on dual-token decoupling and soft strip feature extraction according to claim 4, characterized in that, In step S3, the soft stripe feature extraction specifically includes: assuming the patch tokens output by the last layer of the image encoder are: , ,in, , H and W These represent the height and width of the patch grid, respectively. d Indicates the feature dimension; No. n The normalized coordinates of each patch in the vertical direction are: in, = indicates the first n The row index where each patch is located. ; Divide the human body along the vertical direction into K The soft part, the first k The center position of each part Defined as: ， Set an independent learnable query vector for each part Calculate the first k The query for the first part and the first n The basic attention score between each patchtoken : in, and These are the linear projection matrices of the query and the key, respectively; Attention scores are superimposed with Gaussian-form soft strip biases : in, Indicates the bias strength of the soft strip. Indicates the first k Width parameters of the stripe in each part; Attention weights after adding soft strip bias for: The patch tokens are weighted and aggregated according to attention weights to obtain the first... k Features of each part ;Will K Features of each part Perform splicing and linear projection: Where [;] denotes vector concatenation operation. Represents the local aggregation projection matrix. This represents the local aggregation feature of the part branch output.

6. The pedestrian re-identification method based on dual-token decoupling and soft strip feature extraction according to claim 5, characterized in that, In step S3, to enhance the identity discrimination capability of each feature, a feature triplet loss is introduced for each feature. : in, This represents the first sample of the same identity. k Features of each part The first represents the different identity samples k Features of each part; At the same time, for local aggregation features Apply identity classification loss: in, The identity classifier is based on the first i Local aggregation features of individual samples The sample is predicted to be its true identity category. The probability of.

7. The pedestrian re-identification method based on dual-token decoupling and soft strip feature extraction according to claim 1, characterized in that, In step S4, the model inference test specifically includes: recording the global visual features output by the image encoder. for Record the local aggregated features output by the soft strip region feature extraction module. for global features and local aggregation features By splicing the components together, the final fused feature is obtained: Perform final fusion features Normalization: for query images and candidate library images Normalized features were obtained respectively. and Calculate the Euclidean distance between the two. for: Finally, the candidate images are sorted according to their distance from smallest to largest, and the sorted results are output as the pedestrian re-identification results.

8. A pedestrian re-identification system based on dual-token decoupling and soft strip feature extraction, characterized in that, include: The visual and language encoder employs a pre-trained visual language model, including an image encoder and a text encoder, to extract global visual features from the input pedestrian image and patch tokens from the last layer output of the image encoder. The dual-token decoupling learning module is used to achieve semantic separation of identity features and interference features. This module configures a set of learnable identity tokens and a set of learnable interference tokens for each pedestrian identity in the training set, and constructs identity text prompts and interference text prompts based on preset prompt templates. The text encoder then obtains the identity text prototype and interference text prototype. The module is trained in two stages: In the first stage, the image encoder and text encoder are frozen, and only the identity tokens and interference tokens are optimized. Identity image to text contrast loss, identity text to image contrast loss, and text decoupling loss that constrains the orthogonality of identity text prototypes and interference text prototypes are used to decouple identity semantics from interference semantics. In the second stage, the text encoder and the learned dual tokens are frozen, the image encoder is unfrozen, and an identity projection head, a gradient inversion layer, and a non-identity projection head are introduced. When the image encoder is backpropagated through the gradient inversion layer, it receives the opposite gradient, actively weakening the interference information in the global visual features and achieving adversarial decoupling of visual features. The soft strip feature extraction module is used to extract fine-grained local features of the human body without disrupting the global receptive field. This module divides the patch tokens output by the image encoder along the vertical direction. K For each soft region, a learnable region query vector is set. The base attention score between the region query and each patch token is calculated, and a Gaussian soft strip bias is superimposed to guide the model to focus on the vertical region of the human body through soft attention. The attention weights are then weighted and aggregated to obtain... K Features of each part, K Local aggregated features are obtained by splicing together the features of individual parts and then projecting them linearly. The joint inference module is used to concatenate and fuse global visual features and local aggregated features during the testing phase. After normalization, it calculates the Euclidean distance between the query image and the candidate images and completes the ranking and matching to achieve pedestrian re-identification. During the inference phase, only the trained image encoder, global feature branch and soft strip feature branch are retained, without relying on the text encoder and dual tokens.

Images