A pedestrian re-identification method

Abstract

The present application belongs to the field of computer vision, and discloses a pedestrian re-identification method.A counterfactual data generation method of random patch clipping and random padding is proposed to generate counterfactual images; through counterfactual double-path network feature extraction, fact feature vectors and counterfactual feature vectors are generated.The fact feature vectors and the counterfactual feature vectors are input into a causal counterfactual inference module to generate counterfactual loss; the prediction result is evaluated through a loss function.The present application uses causal reasoning to construct a controllable counterfactual space, avoiding dependence on local features.In the counterfactual prediction process, explicit supplementary long-range dependency features can be added to improve the robustness of the model.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more particularly to a method for pedestrian re-identification. Background Technology

[0002] In recent years, pedestrian re-identification has played an increasingly important role in criminal investigation, security and prevention, and various public environments (such as shopping malls and streets). Pedestrian re-identification technology aims to accurately identify pedestrians in different scenarios by extracting and matching features from pedestrian images. The research background and significance of pedestrian re-identification technology is that accurate identification of pedestrians enables precise management and monitoring of people, thereby improving social security and public safety management. Currently, most pedestrian re-identification methods rely on local features to identify target individuals.

[0003] On the other hand, given the relatively small size of existing datasets, numerous counterfactual data generation methods have emerged during model training. These include methods such as random erasure and random cropping.

[0004] Most current pedestrian re-identification methods use local features to identify the target person, but this also has some drawbacks. First, feature-based local methods usually fuse many local features before recognition, resulting in a large number of parameters and model size, making deployment difficult in real-world scenarios. Second, over-reliance on and mining of local features can easily lead to recognition errors. Furthermore, the fusion of multiple local features can result in poor model robustness, failing to consider more contextual information.

[0005] Due to the limited size of existing datasets, overfitting is a significant problem in deep network models. Currently, most counterfactual data generation methods directly disrupt the contextual relationships within instances. For example, random erasure randomly selects rectangular regions in an image and replaces their pixels with random values. However, existing random erasure methods often erase connected regions, leading to the loss of local image features and reducing the model's robustness. Furthermore, filling the erased regions with fixed pixel values ​​alters the pixel value distribution of the original image. Summary of the Invention

[0006] The objective of this invention is to achieve concise and explicit supplementary long-range dependency features through controllable counterfactual prediction at the output layer. Utilizing causal counterfactual reasoning, we propose employing global features to reduce reliance on local detail features and additional semantic information, thereby lowering prediction bias for images of people. Furthermore, to effectively construct an efficient explicit counterfactual space, we propose a novel counterfactual data generation method that encourages the construction of counterfactual data that disrupts contextual relationships while preserving local features, without introducing additional counterfactual noise.

[0007] In the above introduction, we aim to achieve two goals through optimization methods: (1) the proposed model should explicitly and concisely supplement pedestrian long-range dependency features and extract more valuable feature information; (2) we penalize counterfactual predictions in a counterfactual framework, which forces the classifier to mitigate the influence of biased training sets due to insufficient information.

[0008] The technical solution of the present invention is as follows: A pedestrian re-identification method, comprising the following steps:

[0009] Step 1: Construct a counterfactual image;

[0010] For each person, K identities and M original fact images are randomly selected to form a training batch, which is set as B = K × M. Two sets of input data are constructed using the training batch. The first set of input data is the original fact images, and the second set of input data is the counterfactual images constructed by randomly cropping, patching and filling the original fact images with a certain probability.

[0011] Step 2: Generate fact feature vectors and counterfactual feature vectors through counterfactual dual-path network feature extraction;

[0012] Step 3: Input the fact feature vector and counterfactual feature vector into the causal counterfactual inference module to generate the counterfactual loss; use a directed acyclic graph. This is redefined and represented as a causal counterfactual inference module; we pass the factual feature vector and the counterfactual feature vector through a fully connected layer to obtain the factual prediction result and the counterfactual prediction result. Represents a directed acyclic graph In the node variables, ε represents Interactions between node variables; representing the original fact image, counterfactual image, and classification prediction result Y obtained through the causal counterfactual inference module as node variables; linking I→I e Represents the generated fact pixel patch block, (I,I) e →Y indicates that the classification prediction result is determined by the original fact image and fact pixel patch blocks; the causal relationship between nodes is encoded as →, and node I is called I in the causal counterfactual inference module. e The causal parent node, node Y is called I and I e causal child nodes;

[0013] Based on the causal relationships between variables in the causal counterfactual inference module, several causal sub-nodes are directly processed to perform causal counterfactual inference and analyze the causal relationships between variables. Counterfactual pixel patch blocks are used instead of fact pixel patch blocks for controllable counterfactual intervention. During the learning process, the counterfactual bias Y is used. effectThis represents the impact of the counterfactual pixel patch on the final prediction result;

[0014] Step 4: Evaluate the prediction results using loss functions, including center loss, ID classification loss, triplet loss, and counterfactual loss.

[0015] The specific generation process of the counterfactual image is as follows: Given a batch of original factual images X, the probability of random cropping and random padding is set to P; in an original factual image, several rectangular regions of the same size are randomly selected and saved as factual pixel patch blocks; the area ratio of the factual pixel patch blocks is denoted as r. e =S e / S, where S e Let S represent the area of ​​the fact pixel patch, and S represent the area of ​​the original fact image; the sequence of fact pixel patches is denoted as... By shuffling the extracted fact pixel patch blocks, a list of counterfactual pixel patch blocks is generated. Where x i ∈[1,n]; then, the counterfactual pixel patch blocks are randomly filled on the original factual image to generate a counterfactual image.

[0016] The counterfactual dual-path network is a feature extractor with two shared parameters, used to consider instance longitude dependencies; the counterfactual dual-path network allows for simultaneous modeling of a person's global features; in the counterfactual dual-path network, the original factual image x is input into the first path, and the corresponding counterfactual image x... * Enter it into the second path.

[0017] The feature extractor uses ResNet 50 as the backbone network to generate feature maps for the original factual image and the counterfactual image, respectively, as shown below. and Where X represents the fact feature vector obtained after feature extraction from the original fact image, X... * This represents the counterfactual feature vector obtained after feature extraction from the counterfactual image, where C, H, and W represent the channel size, height, and width of the feature map, respectively.

[0018] The counterfactual bias Y effect Represented as:

[0019]

[0020] Where I represents the original fact image, I e This represents a patch of fact pixels extracted from the original fact image. Y represents the counterfactual pixel patch generated from the factual pixel patch block, and Y represents the classification prediction result obtained through the causal counterfactual inference module.

[0021] The loss function Represented as:

[0022]

[0023] Where y is the category label. For cross-entropy loss, The original objective cost function includes ID classification loss, triplet loss, and center loss.

[0024] The beneficial effects of this invention: The pedestrian re-identification method proposed in this invention is based on causal reasoning. This method achieves concise and explicit supplementation of long-range dependency features through controllable counterfactual prediction at the output layer. Utilizing causal counterfactual reasoning, global reasoning can be performed, reducing reliance on local detailed features and additional semantic information, and lowering prediction bias for person images. Furthermore, to construct an effective explicit counterfactual space, we propose a counterfactual data generation method involving random cropping patches and random padding to encourage the construction of counterfactual data that disrupts contextual relationships while preserving local features, without introducing additional noisy counterfactuals.

[0025] Our proposed person re-identification method can be extended based on various approaches and achieves good results. The advantage of this method lies in constructing a controllable counterfactual space using causal reasoning, avoiding dependence on local features. During counterfactual prediction, explicit supplementary long-range dependency features can be added to improve the model's robustness. Our proposed counterfactual data generation method can also effectively construct an explicit counterfactual space, thereby further improving model performance. We conducted extensive experimental evaluations on several benchmarks, demonstrating the effectiveness and superiority of the proposed method. Furthermore, our method provides new insights into considering global information of instances and constructing counterfactual instances to obtain better feature representations, and can be applied to various computer vision tasks other than person re-identification. Attached Figure Description

[0026] Figure 1(a) shows the model framework of the pedestrian re-identification method;

[0027] Figure 1(b) is a schematic diagram of the method for generating counterfactual data;

[0028] Figure 2 A schematic diagram illustrating the generation method for counterfactual data;

[0029] Figure 3 This is a method for generating counterfactual data. Detailed Implementation

[0030] The key technical point of this invention is:

[0031] 1. We propose a counterfactual learning framework aimed at improving the accuracy of person re-identification by explicitly incorporating long-range dependency features. The proposed framework employs a dual-path counterfactual architecture with shared convolutional parameters, simultaneously accepting both factual and counterfactual instances as input to disrupt contextual relationships. The framework also implements controllable counterfactual operations in the prediction space, thereby interpreting global features without requiring additional parameters;

[0032] 2. We propose a method for generating counterfactual data at the instance level. This method utilizes a counterfactual strategy to directly construct counterfactual instances in the instance space. The main goal of our proposed method is to efficiently create an explicit counterfactual space, enabling reliable counterfactual predictions. Specifically, we randomly crop equal-sized patches from the original image and shuffle their order before infilling the image. This process breaks the contextual relationships in the image without introducing any additional noise.

[0033] This invention provides a pedestrian re-identification method that uses only global features. It performs counterfactual computation in the prediction space by constructing a parameter-shared counterfactual dual-path network structure. It takes actual instance images and counterfactual instance images that disrupt contextual relationships as input. The counterfactual framework allows for interpretable modeling of global features without introducing additional parameters. Figure 1(a) is a schematic diagram of the model structure of this invention, mainly including four steps:

[0034] Step 1: Construct images and labels as a training set;

[0035] The specific process is as follows: During data loading, we randomly select K identities and M images for each person to form a training batch. Finally, the training batch size is set to B = K × M.

[0036] We constructed two sets of input data: the first set used the original factual data as input, and the second set used the corresponding counterfactual data as input. During model training, we randomly cropped and patched the training batch with a certain probability. Specifically, given a batch of images X, we set the probability of recombination to P and the probability of remaining unchanged to 1-P. As shown in Figure 1(b), in the image, we randomly selected rectangular regions of the same size and saved them as factual pixel patches. The area ratio of the factual patches is denoted as r. e =S e / S, where S e Let S and S represent the area of ​​the fact pixel patch and the area of ​​the original image, respectively. The sequence of fact patches is denoted as... By shuffling the extracted fact patch blocks, a list of counterfactual patch blocks is generated. Where x i ∈[1,n]. Then, the original image is randomly filled with counterfactual patches to generate a counterfactual data image. Figure 2 This is a demonstration of the results of the counterfactual generation method of the present invention.

[0037] For factual image x and counterfactual image x * The original factual image sequence is denoted as Counterfactual image sequence is represented as

[0038]

[0039] Step 2: Remote Feature Extraction of Pedestrian Instances

[0040] The specific process is as follows: Our proposed counterfactual dual-path network adopts a parameter-sharing structure as a feature extractor that considers the long-range dependencies of instances. This counterfactual dual-path network allows for simultaneous modeling of a person's global features. In the counterfactual dual-path network, we input the original factual data into the first path, and the other path inputs the corresponding counterfactual data. We use two parameter-sharing ResNet50s as the backbone network to generate feature maps for the factual and counterfactual images, represented as follows: and Where X represents the fact feature vector obtained after feature extraction from the original fact image, X... * This represents the counterfactual feature vector obtained after feature extraction from the counterfactual image, where C, H, and W represent the channel size, height, and width of the feature map, respectively.

[0041] Step 3: Causal Counterfactual Inference Module

[0042] The specific process is as follows: We use a directed acyclic graph. Redefining and representing as Figure 3 The causal counterfactual structure module in the model. Each variable in the model is a directed acyclic graph. Each node represents the interaction between variables. For example... Figure 3 As shown, we represent the proposed Re-ID framework as nodes in a directed acyclic graph, including the original fact image used, the counterfactual image generated using random cropping patches and random padding, and the prediction result Y output by the causal counterfactual inference module. [Link I→I] e Represents the discovered fact pixel block, (I,I) e →Y indicates that the prediction result is determined by the original fact image and fact pixel patch blocks. The causal relationship between nodes is encoded as →, and in the causal graph, we call node I I. eThe causal parent node, node Y is called I and I e The causal child nodes.

[0043] Step 4: Evaluate the model's prediction results using the loss function.

[0044] The specific process is as follows: Based on the causal relationships between variables in the causal counterfactual inference module, we directly process several causal sub-nodes to perform causal counterfactual inference and analyze the causal relationships between variables. In our proposed method framework, we use counterfactual pixel patches instead of original factual pixel patches to perform controllable counterfactual intervention on the input image. During the learning process, we use the counterfactual bias Y... effect This represents the impact of the counterfactual pixel patch on the final human prediction result. According to... Figure 3 We can obtain the following formula:

[0045]

[0046] Where I represents the original fact image, I e This represents a patch of fact pixels extracted from the original fact image. Y represents the counterfactual pixel patch generated from the factual pixel patch block, and Y represents the classification prediction result obtained through the causal counterfactual inference module.

[0047] In this work, we introduce a strong supervision signal for factual and counterfactual features extracted from a dual-path structure and shared by the model parameters. This signal not only measures the distance between the generated counterfactual image and the original factual image but also provides a tool and method for analyzing the impact of counterfactual pixel patches on the final prediction. The counterfactual loss is expressed as follows:

[0048]

[0049] Where y is the category label. For cross-entropy loss, The original objective cost function includes ID classification loss, triplet loss, and center loss.

[0050] Ablation Experiment of Pedestrian Re-identification Method

[0051] We conducted ablation experiments on the Market-1501 and DukeMTMC-reID datasets to validate the effectiveness of our proposed counterfactual framework and counterfactual data generation method. Let B, S, and Table 1 shows the ablation experiments of our proposed network structure on Market-1501 and DukeMTMC-reID, representing the baseline network, randomized pruning and patching method, randomized padding method, and counterfactual framework, respectively.

[0052] Table 1 shows the ablation experimental results of our proposed pedestrian re-identification method.

[0053]

[0054] We add a counterfactual framework representation to the baseline network as follows: As shown in Table 1, the network framework that incorporates counterfactual feature learning improves the baseline recognition performance. The proposed method achieves a Rank-1 / mAP improvement of 0.3% / 0.7% over the baseline network on the Market1501 dataset and also performs well on Duke-MTMC. These results demonstrate that the counterfactual framework has a strong ability to simulate both local and global features. Furthermore, the effectiveness of the proposed method is validated, and the effectiveness of counterfactual learning in human Re-ID tasks is also confirmed.

[0055] We add a counterfactual data generation method, denoted as B+S, to the baseline network. During training, only the proposed counterfactual data generation method is incorporated into the baseline network, while maintaining the original training strategy. As shown in Table 1, our proposed counterfactual data generation method improves the evaluation metrics. Considering the relatively small data size of the pedestrian re-identification dataset, the counterfactual data generation method not only preserves the detailed local features of the original images but also achieves better anti-overfitting performance.

[0056] Finally, our proposed method is expressed as follows: Our proposed person re-identification method achieves state-of-the-art performance on the Market1501 dataset, with a Rank-1 score of 95.2% and an mAP of 87.3%. It also achieves good results on the DukeMTMC-reID dataset.

[0057] Comparison Experiment of Pedestrian Re-identification Methods

[0058] As shown in Table 2, we present the recognition accuracy results of different methods on two commonly used datasets. We considered four types of person re-identification methods for comparison, including methods based on local strip features; methods based on attention mechanisms; methods based on additional local semantic features; and methods based on global features. The proposed method is listed at the end of Table 1. The methods selected for comparison are all among the best in the field of person re-identification.

[0059] From the results in Table 2, we observe that the GCP network based on stripe features and the SCAL network using an attention mechanism achieve excellent results in the person re-identification task. However, when the GCP network uses only global features, its Rank-1 accuracy and mAP are not ideal. Furthermore, the improved attention mechanism method has proven effective, but it requires more training parameters and involves significant computational costs during training. For example, Transformer-based decoupled representation methods, such as DRL-Net, achieve similar recognition accuracy to our proposed method, but our method is more efficient and has a simpler loss function. Methods based on additional semantic features, such as InSTD, combine time-series models, spatial sequence models, and 3D visual feature extraction based on additional semantic features. In contrast, our proposed method only extracts visual features. Further extending our method by incorporating spatiotemporal features can achieve higher accuracy. Among all methods using only global features, our proposed network achieves the best results on the Market1501 and DukeMTMC-reID datasets, outperforming other methods.

[0060] Our proposed person re-identification method achieves state-of-the-art performance on the Market1501 dataset, with a Rank-1 score of 95.2% and an mAP of 87.3%. Our experimental results further demonstrate that our proposed method is more effective in improving the mAP metric, indicating its ability to significantly improve the matching ranking of target task images. On the DukeMTMC-reID dataset, individuals typically carry personal attributes such as backpacks and other items. Our proposed method extracts only global features, without relying on personal attributes for feature guidance, achieving Rank-1 and mAP of 87.1% and 76.9%, respectively. Compared to our proposed method, some methods based on additional attribute guidance exhibit lower competitiveness. Conversely, our counterfactual framework fully leverages latent, important person contextual relationships and robust features.

[0061] Table 2 shows the results of different methods on the Market1501 and DukeMTMC-reid datasets.

[0062]

[0063]

Claims

1. A pedestrian re-identification method, characterized in that, The steps include the following: Step 1: Construct a counterfactual image; For each person, randomly select K identities and M original fact images to form a training batch, and set the training batch as B = K × M; Two sets of input data are constructed using training batches. The first set of input data is the original factual image, and the other set of input data is a counterfactual image constructed by randomly cropping, patching and filling the original factual image with a certain probability. Step 2: Generate fact feature vectors and counterfactual feature vectors through counterfactual dual-path network feature extraction; Step 3: Input the fact feature vector and counterfactual feature vector into the causal counterfactual inference module to generate the counterfactual loss; use a directed acyclic graph. The module is redefined and represented as a causal counterfactual inference module; the fact feature vector and the counterfactual feature vector are passed through a fully connected layer to obtain the fact prediction result and the counterfactual prediction result; where Represents a directed acyclic graph In the node variables, ε represents Interactions between node variables; representing the original fact image, counterfactual image, and classification prediction result Y obtained through the causal counterfactual inference module as node variables; linking I→I e Represents the generated fact pixel patch block, (I, I e →Y indicates that the classification prediction result is determined by the original fact image and fact pixel patch blocks; the causal relationship between nodes is encoded as →, and node I is called I in the causal counterfactual inference module. e The causal parent node, node Y is called I and I e causal child nodes; Based on the causal relationship between variables in the causal counterfactual inference module, several causal sub-nodes are directly processed to perform causal counterfactual inference and analyze the causal relationship between variables. Use counterfactual pixel patch blocks instead of factual pixel patch blocks to perform controlled counterfactual intervention operations; During the learning process, counterfactual bias Y is used. effect This represents the impact of the counterfactual pixel patch on the final prediction result; Step 4: Evaluate the prediction results using loss functions, including center loss, ID classification loss, triplet loss, and counterfactual loss.

2. The pedestrian re-identification method according to claim 1, characterized in that, The specific generation process of the counterfactual image is as follows: Given a batch of original factual images X, the probability of random cropping and random padding is set to P; in an original factual image, several rectangular regions of the same size are randomly selected and saved as factual pixel patch blocks; the area ratio of the factual pixel patch blocks is denoted as r. e =S e / S, where S e Let S represent the area of ​​the fact pixel patch, and S represent the area of ​​the original fact image; the sequence of fact pixel patches is denoted as... By shuffling the extracted fact pixel patch blocks, a list of counterfactual pixel patch blocks is generated. Where x i ∈[1, n]; Then, counterfactual pixel patches are randomly filled onto the original factual image to generate a counterfactual image.

3. The pedestrian re-identification method according to claim 2, characterized in that, The counterfactual dual-path network is a feature extractor with two shared parameters, used to consider instance remote dependencies; the counterfactual dual-path network allows for the simultaneous modeling of a person's global features. In a counterfactual two-path network, the original factual image x is input into the first path, and the corresponding counterfactual image x is obtained. * Enter it into the second path.

4. The pedestrian re-identification method according to claim 3, characterized in that, The feature extractor uses ResNet 50 as the backbone network to generate feature maps for the original factual image and the counterfactual image, respectively, as shown below. and Where X represents the fact feature vector obtained after feature extraction from the original fact image, X... * This represents the counterfactual feature vector obtained after feature extraction from the counterfactual image, where C, H, and W represent the channel size, height, and width of the feature map, respectively.

5. The pedestrian re-identification method according to any one of claims 2-4, characterized in that, The counterfactual bias Y effect Represented as: Where I represents the original fact image, I e This represents a patch of fact pixels extracted from the original fact image. Y represents the counterfactual pixel patch generated from the factual pixel patch block, and Y represents the classification prediction result obtained through the causal counterfactual inference module.

6. The pedestrian re-identification method according to claim 5, characterized in that, The loss function Represented as: Where y is the category label. For cross-entropy loss, The original objective cost function includes ID classification loss, triplet loss, and center loss.

Images