A pedestrian re-identification method based on a visual large model

Abstract

The application provides a pedestrian re-identification method based on a visual large model, comprising constructing an identification model based on a visual large model, wherein the identification model comprises a feature extraction module, a feature fusion module, a distribution feature learning module, and a soft and hard label joint training module; the feature extraction module is used for globally capturing image information; the feature fusion module is used for obtaining multi-level image features; the distribution feature learning module is used for converting features from a simple vector representation to a distribution feature; and the soft and hard label joint training module is used for training the identification model to obtain a final model; and pedestrian images are input into the final identification model to obtain an identification result; and the application improves the accuracy and robustness of feature extraction and the accuracy of pedestrian re-identification.

Description

Technical Field

[0001] This invention belongs to the field of pedestrian re-identification technology, and in particular relates to a pedestrian re-identification method based on a large visual model. Background Technology

[0002] Person re-identification (ReID), a core task in computer vision, aims to identify specific pedestrians across different camera scenarios and is widely used in smart security, intelligent transportation, and other fields. However, due to the influence of factors such as different camera perspectives, lighting, and occlusion, person re-identification faces significant challenges. Traditional methods are insufficient in terms of the accuracy and robustness of feature extraction, making it difficult to meet the practical application needs in complex scenarios. Existing technologies still face the following key challenges in practical applications:

[0003] 1. Insufficient feature expression ability

[0004] Traditional methods rely on manually designed features (such as LBP, HOG, and color histograms), making it difficult to capture fine-grained semantic information across different scenarios. For example, local details of pedestrians (such as backpacks and accessories) or dynamic features (such as gait) are easily affected by changes in lighting and posture, leading to insufficient feature discriminative power. While deep learning-based CNN models (such as ResNet and DenseNet) can extract global features, their robustness to occlusion and low-resolution images is limited. Furthermore, ViT (Vision Transformer) enhances global modeling capabilities through its self-attention mechanism, but its shallow networks may ignore local details, while the computational complexity of its deep networks limits real-time performance.

[0005] 2. Weak domain generalization ability

[0006] Existing methods heavily rely on fine-tuning within the target domain, making it difficult to directly generalize to unseen camera distributions. For example, domain differences (such as background style and resolution) between the Market-1501 and DukeMTMC-reID datasets can lead to a significant drop in model performance. This problem stems from the limited size of labeled data: mainstream datasets (such as CUHK03 and MSMT17) have fewer than 10,000 classes, far fewer than the data volume of face recognition (such as Glint360k with 360,000 classes) or object detection (such as Objects365 with 10 million samples). Furthermore, differences in pedestrian appearance across domains (such as clothing changes and occlusion) further exacerbate the difficulty of model transfer.

[0007] 3. Insufficient modeling of label noise sensitivity and intra-class differences.

[0008] Hard-label training (e.g., cross-entropy loss) is susceptible to annotation noise and cannot effectively model intra-class feature distribution differences. For example, pedestrians with the same identity may exhibit multimodal distributions due to pose variations, while traditional methods rely solely on single feature center constraints, leading to excessively large intra-class distances. Furthermore, existing methods struggle to achieve robust feature alignment when processing cross-modal data (e.g., visible-infrared images) due to significant inter-modal spectral differences (e.g., missing color information). Limitations of existing technologies include:

[0009] Traditional manual feature methods rely on manually designed features, have poor generalization ability, and achieve an accuracy of less than 20% in complex scenarios.

[0010] CNN-based deep learning methods can achieve Rank-1 accuracy of over 90% on public datasets (such as Market-1501), but their performance drops by more than 30% in cross-domain tests (such as MSMT17→DukeMTMC).

[0011] Pre-trained model methods: Self-supervised models such as ViT improve feature generalization, but when directly applied to ReID, they do not fully integrate multi-level semantic information and lack explicit modeling of distribution differences.

[0012] Multimodal fusion methods: Although the combination of visible light and infrared light can alleviate the illumination problem, the calculation of modal alignment loss (such as KL divergence) is complicated, and the cross-modal retrieval accuracy is still lower than that of single-modal scenes.

[0013] In summary, existing pedestrian re-identification technologies urgently need to overcome multiple bottlenecks, such as feature representation, domain generalization, and noise robustness, in order to achieve a leap from closed datasets to open scenarios. Summary of the Invention

[0014] In view of this, the present invention aims to overcome the shortcomings of the above-mentioned problems in the prior art and proposes a pedestrian re-identification method based on a large visual model to improve the accuracy and robustness of pedestrian re-identification in cross-domain scenarios.

[0015] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0016] A pedestrian re-identification method based on a large visual model includes...

[0017] A recognition model is constructed based on a large visual model. The recognition model includes a feature extraction module, a feature fusion module, a distributed feature learning module, and a soft and hard label joint training module. The feature extraction module is used to capture image information globally. The feature fusion module is used to obtain multi-level image features. The distributed feature learning module is used to convert features from simple vector representations into distributed features. The soft and hard label joint training module is used to train the recognition module to obtain the final model.

[0018] The pedestrian image is input into the final recognition model to obtain the recognition result.

[0019] Furthermore, the feature extraction module uses the distillation model of the visual large model DINOv2 as the feature extractor, and the model architecture is built based on Vision Transformer.

[0020] Furthermore, the feature extraction module implementation process includes:

[0021] Perform shape transformation on the pedestrian image to make its size 252×126;

[0022] The image is input into the feature extractor, which embeds the input image.

[0023] The input image is segmented into 18×9 image blocks, and an initial embedding vector is generated through linear projection. The vector is then input into the Transformer block.

[0024] Global contextual relationships are modeled using multi-head self-attention, and feature vectors of pedestrian images are obtained through attention layers.

[0025] Furthermore, each Transformer block consists of a multi-head self-attention mechanism and a feedforward neural network.

[0026] Furthermore, the feature fusion module implementation process includes:

[0027] The features are extracted sequentially from the output of the Transformer block to obtain a feature set.

[0028] Perform dimension alignment transformation on each layer of features obtained;

[0029] Perform convolution processing on the fused feature vectors

[0030] Operations on convolutional layers are handled using activation functions.

[0031] Furthermore, the convolutional processing uses 1D convolution to capture cross-layer dependencies, and performs convolution operations on the feature vector by sliding convolution kernels to dynamically combine shallow local features with deep semantic features.

[0032] Furthermore, the implementation process of the distribution feature learning module includes:

[0033] The features are passed through a Transformer encoder to transform the fused features;

[0034] The transformed features are used to predict the center and variance of the samples through a two-branch fully connected layer;

[0035] The feature centers and variances of all samples belonging to the same identity are aggregated to obtain the identity level distribution;

[0036] Soft tags are generated based on identity level distribution by calculating the similarity between identity level distributions.

[0037] Furthermore, the soft tag is calculated as follows:

[0038]

[0039] Where k is the ID of the current identity; It is the set of positive sample identities found based on distribution similarity, D(p k ,p j ) is the distribution distance between identity k and identity j, τ is the temperature parameter, which controls the smoothness of the soft label to 0.1, and α is the weight of the soft label, which controls the influence of the soft label and is set to 0.4.

[0040] Furthermore, the loss function of the joint training module for soft and hard labels is designed as follows:

[0041]

[0042] in, It's a soft tag, Y k λ is the hard label, and λ is the weight balancing parameter between the soft and hard labels, set to 0.35. It is the probability distribution of the model's prediction of global features.

[0043] Furthermore, the loss is calculated and backpropagation is performed to update the model parameters. After multiple rounds of training, the final recognition model is obtained.

[0044] Compared with existing technologies, the pedestrian re-identification method based on a large visual model described in this invention has the following advantages:

[0045] This invention employs the DINOv2 feature extractor based on the ViT architecture, which can globally capture image information. By fusing the features output from the 12-layer Transformer block of the model, it obtains multi-level image features, improving the accuracy and robustness of feature extraction and enabling it to better cope with various interference factors in different scenarios.

[0046] This invention transforms features from simple vector representations into distributed features through parallel convolution processing and distributed feature learning, enhancing the expressive power of features and enabling more accurate description of pedestrian features, thereby improving the accuracy of pedestrian re-identification.

[0047] This invention combines hard and soft labels for training, making full use of the information contained in the labels. Soft labels can provide more detailed distribution features and identity information, improving the model's cross-domain generalization ability and making the model's performance more stable and reliable in complex scenarios. Attached Figure Description

[0048] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0049] Figure 1 This is a schematic diagram of the overall method of a pedestrian re-identification method based on a large visual model according to the present invention;

[0050] Figure 2 This is a schematic diagram illustrating the implementation process of the feature fusion module of the present invention;

[0051] Figure 3 This is a schematic diagram illustrating the implementation process of the distribution feature learning module of the present invention. Detailed Implementation

[0052] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0053] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0054] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0055] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0056] like Figure 1 As shown, this invention provides a pedestrian re-identification method based on a large visual model, including...

[0057] A recognition model is constructed based on a large visual model. The recognition model includes a feature extraction module, a feature fusion module, a distributed feature learning module, and a soft and hard label joint training module. The feature extraction module is used to capture image information globally. The feature fusion module is used to obtain multi-level image features. The distributed feature learning module is used to convert features from simple vector representations into distributed features. The soft and hard label joint training module is used to train the recognition module to obtain the final model.

[0058] The pedestrian image is input into the final recognition model to obtain the recognition result.

[0059] In this invention, the feature extraction module uses the distillation model of the large visual model DINOv2 as the feature extractor. DINOv2 is a general visual model based on self-supervised learning. It learns general visual features on the LVD-142M dataset through self-supervised pre-training. The model architecture is based on Vision Transformer (ViT).

[0060] The feature extraction module performs shape transformation on the pedestrian image, making its size 252×126. The image is then input into the feature extractor for embedding. The input image is segmented into 18×9 image blocks, and an initial embedding vector is generated through linear projection. This vector is input into a Transformer block, where each Transformer block consists of a multi-head self-attention mechanism and a feedforward neural network. Global contextual relationships are modeled through multi-head self-attention, and the feature vector of the pedestrian image is obtained through attention layers. The output of each layer contains F... i ∈R d The feature vectors (where d is the embedding dimension) are generated by 12 Transformer blocks in this model, resulting in 12 feature vectors output by each block. Each feature vector output by a block contains semantic information at different levels.

[0061] In this invention, the feature set extracted from the outputs of the 12 Transformer blocks of ViT by the feature fusion module from layers 1 to 12 is as follows:

[0062] {F1,F2,…,F 12}

[0063] Where F i ∈R 768That is, the feature dimension is 768. To eliminate the difference in dimensions between layers, L2 normalization is performed on the features of each layer to make them satisfy ||F||. i ||2=1. Then, a dimension alignment transformation is performed on the obtained features of each layer, and the features output by each attention layer are mapped to a unified dimension space d′=256 through independent fully connected layers:

[0064] F′ i =W i ·F i +b i W i ∈R 256×768 ,b i ∈R 256

[0065] Among them W i b is a weight vector that varies in dimension. i As the bias vector, the 12 transformed features are fused. Specifically, the 12 feature vectors are first stacked in parallel sequentially to form a higher-dimensional fused input feature matrix.

[0066] F stack =[F′1;F′2;…;F′ 12 ]∈R 12×256

[0067] The fused feature vectors are then subjected to convolution processing. This invention designs a convolutional neural network module that uses 1D convolution to capture cross-layer dependencies. By using a sliding convolution kernel to perform convolution operations on the feature vectors, shallow local features are dynamically combined with deep semantic features. The convolution operation is as follows:

[0068]

[0069] Where W∈R 12×256 It is a learnable weight matrix, Y∈R 256 To output a tensor.

[0070] The convolution kernel design is as follows:

[0071] Number of input channels: 12 (corresponding to the number of attention layers)

[0072] Number of output channels: 256 (feature count)

[0073] Kernel size: k = 1

[0074] Step size: 1

[0075] Padding: 1 (Keep output length unchanged)

[0076] Finally, the operations on the convolutional layers are processed using activation functions:

[0077] F fusion =σ(F C )

[0078] Where σ(·) is the ReLU activation function.

[0079] In this invention, the distribution feature learning module first transforms the fused features through a Transformer encoder. The Transformer encoder contains two Transformer blocks and uses the GELU activation function. Then, the transformed features are passed through a two-branch fully connected layer to predict the center and variance of the samples.

[0080] The dual-branch fully connected layer is as follows:

[0081] Central prediction branch:

[0082]

[0083] in W t ∈R 256×256 d is the output dimension

[0084] Variance prediction branch:

[0085]

[0086] The Softplus function ensures that the variance is non-negative.

[0087] The feature centers c and variance v of all samples belonging to the same identity 2 Aggregate the data to obtain the identity level distribution. For each identity k, calculate the feature center μ of its identity level. k and variance

[0088]

[0089] Soft labels are generated based on identity level distributions. This is achieved by calculating the similarity between identity level distributions. The distributional distance D(p) between identity k and identity j is calculated using the KL divergence method. k ,p j ).

[0090] Based on the distribution distance, find the identity most similar to the current identity k (i.e., the positive sample identity), and generate soft labels based on these positive sample identities. The soft labels are calculated as follows.

[0091]

[0092] in:

[0093] k is the ID of the current identity;

[0094] It is a set of positive sample identities found based on distribution similarity.

[0095] D(p k ,p j ) is the distribution distance between identity k and identity j.

[0096] τ is a temperature parameter that controls the smoothness of the soft label by 0.1.

[0097] α is the weight of the soft label, which controls the degree of influence of the soft label, and is set to 0.4.

[0098] During the training phase, the generated soft labels are combined with traditional hard labels for model training. Soft labels provide additional information on identity level distribution similarity, while hard labels ensure the model can classify correctly. The loss function is designed as follows:

[0099]

[0100] It's a soft tag.

[0101] Y k It is a hard tag (one-hot encoded).

[0102] λ is the weight balancing parameter for soft and hard labels, set to 0.35.

[0103] It is the probability distribution of the model's prediction of global features.

[0104] Finally, the loss is calculated and backpropagation is performed to update the model parameters. After multiple rounds of training, the final model is obtained, and the query results are obtained.

[0105] The experimental data obtained by training on the MSMT17 dataset and testing on the Market, MSMT-17, and CUHK03-NP datasets are shown in Table 1.

[0106] Table 1 Experimental Results

[0107]

[0108]

[0109] Experimental results show that the present invention has good generalization ability on multiple datasets, especially for the CUHK03-NP dataset, where it achieves an improvement of 13.1% and 10.0% in Rank-1 and mAP evaluation metrics respectively compared with the existing PAT method. It also shows significant improvement on multiple datasets and has good robustness.

[0110] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A pedestrian re-identification method based on a visual large model, characterized in that: The application relates to a recognition model based on a visual large model, which comprises a feature extraction module, a feature fusion module, a distribution feature learning module and a soft and hard label joint training module. The feature extraction module uses a distillation model of a visual large model DINOv2 as a feature extractor, and the model architecture is based on a Vision Transformer. The feature extraction module comprises the following steps: The pedestrian image is subjected to shape transformation to have a size of 252*126; The image is input into the feature extractor to embed the input image; The input image is divided into 18*9 image blocks, an initial embedding vector is generated through linear projection, and the vector is input into a Transformer block; The global context relationship is modeled through multi-head self-attention, and a feature vector of the pedestrian image is obtained through an attention layer; The feature fusion module comprises the following steps: The features of the output of the Transformer block are sequentially extracted to obtain a feature set; The dimensions of the obtained features of each layer are aligned and transformed; The fused feature vector is subjected to convolution processing The operation of the convolution layer is processed through an activation function; The distribution feature learning module comprises the following steps: The features are transformed through a Transformer encoder; The transformed features are input into a double-branch fully connected layer to predict the center and variance of the sample; The feature center and variance of all samples belonging to the same identity are aggregated to obtain an identity level distribution; Soft labels are generated based on the identity level distribution, and the soft labels are generated by calculating the similarity between the identity level distributions; The loss function of the soft and hard label joint training module is designed as follows: The pedestrian image is input into the final recognition model to obtain a recognition result. wherein, is a soft label, Y k is a hard label, λ is a weight balancing parameter of soft and hard labels, set to 0.35, is the predicted probability distribution of the model on the global features; Each Transformer block is composed of a multi-head self-attention mechanism and a feedforward neural network.

2. The pedestrian re-identification method based on a visual large model according to claim 1, characterized in that: The convolution processing uses 1D convolution to capture the cross-layer dependency relationship, and the convolution operation is performed on the feature vector through a sliding convolution kernel to dynamically combine the shallow local features and deep semantic features.

3. The pedestrian re-identification method based on a visual large model according to claim 1, characterized in that: The soft label calculation method is as follows:

4. The pedestrian re-identification method based on a visual large model according to claim 1, characterized in that: The loss is calculated and the model parameters are updated through back propagation, and the final recognition model is obtained through multiple rounds of training. wherein k is the ID of the current identity; is the positive sample identity set found based on distribution similarity, D(p k , j is the distribution distance between identity k and identity j, τ is a temperature parameter, controlling the smoothing degree of the soft label 0.1, and α is the weight of the soft label, controlling the influence degree of the soft label, and is set to 0.

4.

5. The pedestrian re-identification method based on a visual large model according to claim 1, characterized in that: ​

Images