Shielding pedestrian re-identification method based on combined data enhancement and feature interaction reconstruction

Abstract

The invention discloses an occluded pedestrian re-identification method based on combined data enhancement and feature interaction reconstruction, which relates to the technical field of pedestrian re-identification, and comprises the following steps: acquiring an occluded pedestrian re-identification data set containing different identities, and dividing the data set into a training set and a test set according to a preset proportion; designing a CAFIR model framework, wherein the framework structure comprises a combined data enhancement module, a feature interaction module and a feature reconstruction module; the training set is input into the CAFIR model in batches for training until the loss function converges, and an optimized CAFIR model is obtained; and performing pedestrian re-identification on the test set by using the optimized CAFIR model to obtain an identification result. According to the method, the problem of recognition of pedestrian shielding in different scenes can be solved, local detail features of visible parts of pedestrians are accurately extracted, shielding noise is weakened, and then pedestrian re-recognition is accurately carried out under the shielding condition.

Description

Technical Field

[0001] The present invention relates to the technical field of pedestrian re-identification, and more specifically, to an occluded pedestrian re-identification method based on combined data augmentation and feature interaction reconstruction. Background Art

[0002] The main objective of pedestrian re-identification (Re-ID) is to retrieve a specific pedestrian from images captured by non-overlapping surveillance cameras. With the continuous progress of deep learning technology, remarkable achievements have been made in the field of pedestrian re-identification. However, the occlusion problem remains a key factor restricting the performance of pedestrian re-identification. In practical application scenarios, occluders such as umbrellas, billboards, cars (referred to as object occlusion, OTP) or other pedestrians (referred to as pedestrian occlusion, PTP) often block part of the target pedestrian's body, which greatly increases the difficulty of effectively extracting pedestrian features from incomplete or partially occluded images. Although deep learning has made great progress in pedestrian re-identification tasks, existing methods still have difficulty in effectively dealing with large-area occlusion or diverse occluders, which has prompted a large number of research efforts.

[0003] The field of occluded pedestrian re-identification faces the following key problems: 1) Data distribution imbalance is a significant issue: as shown in (a) of Figure 1 , the training set mainly contains a large number of fully visible pedestrian images, while the query set in the test set is almost entirely occluded pedestrian pictures. In contrast, the gallery set is mainly composed of fully visible pedestrian pictures. This unbalanced data distribution poses a severe challenge to the training of the model. 2) There are significant differences between occluded pedestrian re-identification and traditional re-identification methods that assume pedestrians are fully visible: in occluded pedestrian re-identification, the main challenge comes from the noise brought by occluders, which can mask the key features of pedestrians, thus increasing the difficulty of feature extraction and matching.

[0004] Regarding the first problem, namely data distribution imbalance, current mainstream methods generally adopt data augmentation strategies aimed at enhancing data diversity. However, traditional data augmentation means, such as randomization processing or directly pasting occluders on images, may inadvertently change the basic details of the images, especially when these changes involve the key body areas of pedestrians, which may have an adverse impact on the accuracy of pedestrian recognition.

[0005] Regarding the second problem, namely the challenge of the noise caused by occlusion to pedestrian feature extraction, most solutions attempt to distinguish the visible part and the occluded part of pedestrians by introducing external models, such as pose estimation and human parsing. However, such methods usually face two major problems in practical applications: First, these methods mainly focus on solving the problem of complete pedestrian re-identification in the case of no occlusion. In the object occlusion (OTP) scenario, as shown in (b) of Figure 1 , even if the IDs of two pedestrian images are different, due to object occlusion, the visible information extracted only relying on external clues is often very limited. In fact, the occluded area often contains the key information that can distinguish different pedestrians. In addition, in the case of pedestrian occlusion (PTP), the generalization ability of the external model is also particularly limited, as shown in (c) of Figure 1 . Due to the restriction of the cross-domain problem on external clues, there is often a domain gap between the pre-training dataset of the external model and the target occluded image, which may lead to the model wrongly mixing and extracting the features of the target pedestrian and the occluded pedestrian in a severely occluded scenario, thus wrongly introducing the features of non-target pedestrians. Second, introducing an external model will undoubtedly increase the complexity of the network and bring additional computational costs. This not only increases the training difficulty of the model but also may affect the processing speed and efficiency in practical applications.

[0006] Therefore, how to solve the problem of pedestrian occlusion recognition in different scenarios, accurately extract the local detailed features of the visible part of pedestrians and weaken the occlusion noise, and then accurately perform pedestrian re-identification in the case of occlusion is a technical problem that those skilled in the art urgently need to solve. Summary of the Invention

[0007] In view of this, the present invention provides an occluded pedestrian re-identification method based on combined data augmentation and feature interaction reconstruction, which solves the problems existing in the background technology.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] An occluded pedestrian re-identification method based on combined data augmentation and feature interaction reconstruction, comprising the following steps:

[0010] Collect an occluded pedestrian re-identification dataset containing different identities and divide it into a training set and a test set according to a preset ratio;

[0011] Design a CAFIR model framework, the framework structure includes a combined data augmentation module, a feature interaction module, and a feature reconstruction module;

[0012] Input the training set into the CAFIR model in batches for training until the loss function converges to obtain an optimized CAFIR model;

[0013] Use the optimized CAFIR model to perform pedestrian re-identification on the test set to obtain the recognition results.

[0014] Optionally, the combined data augmentation module uses a three-branch input in the form of an image triplet to assist the network in learning the feature representation of occluded pedestrians in different scenarios. The specific content is as follows:

[0015] Given any pedestrian image where H, W, and C represent the height, width, and channel size of the image, respectively;

[0016] Retain the original image I base = BA(I), and perform center cropping I cropped = CA(I), regional random value erasing I erased = EA(I), and data augmentation based on prior knowledge I PKOS = PA(I);

[0017] Concatenate the basic augmentation BA(·), cropping augmentation CA(·), erasing augmentation EA(·), and prior knowledge-based augmentation PA(·) in pairs to form an image triplet [I 1 , I 2 , I 3 , and the expression is as follows:

[0018] I 1 = concat(I base , I PKOS ), I 2 = concat(I cropped , I PKOS ), I 3 = concat(I cropped , I erased )

[0019] In the formula: concat(·,·) represents the concatenation operation;

[0020] Use ViT as the backbone network to extract features, and send the feature Figure 3 tuple to the multi-branch backbone network with shared parameters. Divide I 1 into N patches through the sliding window method The expression is:

[0021]

[0022] In the formula: N H and N W represent the number of patches in the vertical and horizontal directions after segmentation, S represents the stride, P represents the size of each patch, represents the floor function;

[0023] After linear projection, a learnable classification token I is pre-added cls Aggregate information from image patches; before feeding into the Transformer block, add learnable position embeddings to the patch embeddings to preserve position information respectively, expressed as:

[0024]

[0025] In the formula: Z 0 represents the input sequence, F(·) represents the linear projection operation, and D is the channel dimension;

[0026] Z 0 Pass through the Transformer Layer of the l-th layer to obtain the final output of ViT Among them, represents the classification token of the input image, the patch embeddings corresponding to each patch;

[0027] Let f cls Pass through a shared Transformer Layer to get I 1 The final global feature representation Similarly, input I 2 and I 3 into ViT to obtain the global feature representation

[0028] Optionally, the feature interaction module uses a two-way interaction modeling method between global features and local features to assist the network in extracting local detail features of the visible part of the pedestrian and weakening occlusion noise. The specific content is as follows:

[0029] Extract the global feature representation from the image I 1 and extract the local feature representation [f from the same image, f l 1 , f l 2 , f l 3 , …, f l i ; Input the global feature into the GLFI module to enhance the local feature f l i , and this process is represented by the following formula;

[0030]

[0031] In the formula: Obtained through the conv1×1 operation n represents the number of times passing through the GLFI module, which is the same as the number of local features; GLFI(·) represents the global-to-local feature interaction strategy, and the specific operation is expressed by the formula:

[0032]

[0033] In the formula: Softmax(·) is the Softmax function, represents the enhanced local feature; fl represents the local feature;

[0034] The enhanced local feature is incorporated into the global feature, and the expression is as follows:

[0035]

[0036] In the formula: represents the enhanced global feature; is obtained through the conv1×1 operation n represents the number of times passing through the LGFI module, which is the same as the number of local features; LGFI(·) represents the local-to-global feature interaction strategy, and the specific operation is expressed by the formula:

[0037]

[0038] Add to to fuse the global feature, and get:

[0039]

[0040] In the formula: f G represents the fused feature, and α represents the hyperparameter.

[0041] Optionally, the feature reconstruction module uses the first k most similar pictures in the picture library to reconstruct the occluded features of the pedestrian. The specific content is:

[0042] Adopt occlusion simulation based on prior knowledge, and use the upper half of the previous photo of the input photo as the occluder for the lower half of the current photo; set an occlusion label l v for the replaced part to judge whether occlusion simulation is performed. The specific operation is as follows:

[0043]

[0044] In the formula: represents the part of the pedestrian simulated as occluded in the PKOS input branch, represents the original visible part of the pedestrian not simulated as occluded;

[0045] Input I PKOS into the sub-module with a fully connected layer and softmax to predict I PKOS The visible scores of each part The expression is as follows:

[0046]

[0047] In the formula: W fc represents the fully connected layer;

[0048] Retain the unoccluded pedestrian local features, retrieve the positive samples from the gallery and select the unoccluded features of the same pedestrian from them, and reconstruct the occluded features using the visibility weighted average method; the reconstructed local feature f re is expressed as:

[0049]

[0050] In the formula: respectively represent the visible score and local feature of the i-th part corresponding to the pedestrian image in the query set; respectively represent the visible score and local feature of the corresponding part in the k-th sorted list in the gallery;

[0051] After feature reconstruction, update the visibility scores of the query image parts and recalculate the distances between the complete query image and gallery images for retrieval. The expression is:

[0052]

[0053] In the formula: dist(·,·) represents the Euclidean distance, n represents the number of local features, argmax represents finding the maximum value, and ‖·‖ represents the L2 norm; represents the enhanced local features of the gallery image.

[0054] Optionally, the expression of the loss function is:

[0055]

[0056] Among them,

[0057]

[0058]

[0059] In the formula: represents the visibility loss, 2B represents the number after image stitching, l v represents the visibility label, W v is the classifier of the visible score f v ; represents the identity loss, denotes the triplet loss, y denotes the ground truth, P(·) denotes the probability prediction, and W G 、W g and W L are the prediction classifiers of f G 、 and respectively, and are the positive sample pairs, and are the negative sample pairs, and η represents the margin; denotes the main loss function of the combined data augmentation module, denotes the loss function of the fused features, denotes the loss function of the global features, denotes the loss function of the enhanced local features, denotes the enhanced global features, denotes the global features, denotes the enhanced local features.

[0060] Optionally, the CAFIR model is implemented based on the PyTorch platform and is trained and tested in a single RTX 3090 GPU environment.

[0061] Optionally, during the training process of the CAFIR model, a stochastic gradient descent optimizer with a learning rate of 0.002 is adopted, with a momentum of 0.9 and a weight decay of 1e-8, and the training period is 100.

[0062] Optionally, the evaluation metrics of the optimized CAFIR model are: the cumulative match characteristic curve and the mean average precision.

[0063] From the above technical solutions, it can be seen that compared with the prior art, the present invention discloses and provides an occluded pedestrian re-identification method based on combined data augmentation and feature interaction reconstruction, which has the following beneficial effects:

[0064] (1) The present invention proposes the CAFIR model, which uses data augmentation strategies to assist the network in learning the feature representations of occluded pedestrians in different scenarios, and uses the feature interaction mechanism to extract potential and discriminative detailed information. On this basis, the most similar pictures are selected according to these detailed information to reconstruct the occluded area of the pedestrian, which can effectively solve the problem of identifying occluded pedestrians in different scenarios;

[0065] (2) The combined data augmentation method proposed by the present invention not only significantly increases the quantity of the dataset and enriches its diversity, but also revolutionizes the traditional single-branch image input mode; specifically, a three-branch input in the form of an image triple is adopted to replace the single-branch input; in this setting, one branch is set as the main branch, and the other two branches serve as auxiliary branches, which jointly learn the feature representation of pedestrians in a complementary manner;

[0066] (3) The feature interaction module proposed by the present invention adopts a two-way interaction modeling method of global features and local features, which can assist the model in accurately extracting the local detail features of the visible part of pedestrians and weakening the occlusion noise. Description of the Drawings

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the provided drawings.

[0068] Figure 1 Schematic diagram of the challenges faced by occluded person re-identification provided by the present invention;

[0069] Figure 2 Flowchart of the occluded person re-identification method based on combined data augmentation and feature interaction reconstruction provided by the present invention;

[0070] Figure 3 Overall framework diagram of the CAFIR model provided by the present invention;

[0071] Figure 4 Feature interaction model provided by the present invention;

[0072] Figure 5 Feature reconstruction model provided by the present invention;

[0073] Figure 6 Schematic diagram for analyzing the influence of hyperparameters with different values on performance provided by the present invention;

[0074] Figure 7 Comparison chart of the retrieval results of the Baseline and CAFIR methods provided by the present invention;

[0075] Figure 8 Heatmap visualization processing result provided by the present invention. Detailed Embodiments

[0076] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0077] An embodiment of the present invention discloses an occluded pedestrian re-identification method based on combined data augmentation and feature interaction reconstruction, as Figure 2 shown, which includes the following steps:

[0078] Collect an occluded pedestrian re-identification data set containing different identities, and divide it into a training set and a test set according to a preset ratio; design a CAFIR model framework, and the framework structure includes a combined data augmentation module, a feature interaction module, and a feature reconstruction module; input the training set into the CAFIR model in batches for training until the loss function converges to obtain an optimized CAFIR model; use the optimized CAFIR model to perform pedestrian re-identification on the test set to obtain the recognition result.

[0079] To solve the problems pointed out in the background art, this embodiment proposes a new method called CAFIR. In response to the challenge of the unbalanced distribution of occluded images and non-occluded images in the data set, a combined data augmentation (Composite Data Augmentation, CA) strategy is designed to mitigate the negative impact brought by data imbalance. In response to the problem of the loss of key pedestrian information that may be caused by complex occluders, a feature interaction module (Feature Integration Module, FI) and a feature reconstruction module (Feature Reconstruction Module, FR) are proposed. In the training stage, the feature interaction module can combine the context relationship of pedestrians to effectively extract the local distinguishable detail features of pedestrians and weaken the occlusion noise.

[0080] It is recognized in FI that different individual pedestrians with similar appearances pose an obstacle to the recognition efficiency. Therefore, relying solely on the features extracted by the backbone network cannot achieve effective discrimination. For this reason, the complementary relationship between global features and local features is deeply considered, and this relationship is fully utilized to enhance the network's discriminative ability for pedestrians with similar appearances. However, for the recognition challenge in the PTP (Pedestrian Occlusion) scenario, FI alone is not sufficient. Since the features extracted by the backbone network ViT have long-range dependence relationships, the target pedestrian and non-target pedestrians may be wrongly regarded as a whole. To solve this problem, a reconstruction method is adopted in the feature reconstruction module, where the visibility score predictor is used to predict whether the local features are occluded. Subsequently, the visible parts of pedestrians in the feature sets of the k nearest neighbors in the gallery are used to replace the occluded local features, thereby restoring the complete pedestrian feature representation.

[0081] Next, Figure 2 the occluded pedestrian re-identification method shown is elaborated in detail for a deeper understanding.

[0082] I. Related Work

[0083] 1.1 Occluded Pedestrian Re-identification

[0084] In the occluded person re-identification task, matching an occluded query image with full gallery images from different camera views is a highly challenging task. This task not only requires precise extraction of fine-grained features but also effective handling of large intra-class distances. The real-world scenario is complex and full of uncertainties. The diversity of surveillance images is jointly affected by multiple factors, including internal factors such as pedestrian pose and clothing changes, and external factors such as view changes, occlusion by other objects or pedestrians. To address the occlusion problem, there are currently two mainstream methods: the method based on external cues and the method based on hard strip segmentation. The method based on external cues assists feature extraction by introducing external models, such as pose estimation and human parsing. For example, Miao et al. use a human pose estimator to generate human semantic key points, and then expand high-order semantic features to achieve effective alignment of the visible parts of pedestrians; Cioppa et al. use a human parsing model trained on the COCO dataset to generate pseudo-pixel labels for each part of the ReID image and focus on the most representative part information through a part attention module to achieve accurate inter-part matching. The method based on hard strip segmentation manually divides the image or feature map into vertical or horizontal strips for fine-grained feature learning. The part-based convolutional baseline (PCB) network proposed by Sun et al. provides a powerful baseline for person re-identification. This method enhances the internal consistency of local features by uniformly dividing local features through refined partial pooling (RPP); Huang et al. further divide the feature map into several horizontal strips to compensate for the missing overall pedestrian features due to occlusion by exploring the semantic relationship between self-mined local features and global features; Yan et al. design a reasonable segmentation strategy according to the information characteristics of different parts of the human body to prompt the model to learn a more stable pedestrian feature representation under the feature map with occlusion noise.

[0085] 1.2 Occluded Person Re-identification Based on Data Augmentation

[0086] Data augmentation is a direct and effective method to improve the performance of occluded pedestrian re-identification, which has been verified in numerous studies. The fundamental reason is that data augmentation can significantly increase the number of training set images and enrich their diversity. Generally speaking, data augmentation methods can be divided into two categories: operations at the image pixel level and strategies based on the fusion of occluding objects. In the first category of data augmentation methods, a rectangular area is usually selected in the image, and its pixels are replaced with random values or background noise. For example, Yan et al. adopted two data augmentation techniques, Random Erasing (RE) and Batch-constant Erasing (BcE), in their research; Random Erasing enhances the model's robustness to partial occlusion by randomly removing small regions in the image, enabling the model to learn to ignore irrelevant information or occluded parts; while Batch-constant Erasing deletes larger continuous regions to simulate the scenario where pedestrians are partially occluded by objects in reality. The second category of data augmentation methods usually constructs a container to store the occluding objects that often appear in the dataset, then takes an image or feature as input, assigns the occluding object to the image or feature, and outputs it to the subsequent network for training. For example, Jia et al. proposed an Occlusion Sample Augmentation (OSA) strategy, which achieves data augmentation by randomly covering pedestrian images with predefined obstacles. However, the drawback of this method is that it requires manual cropping and pasting of occluding objects, and the operation is relatively cumbersome; in addition, Wang et al. proposed a Feature Diffusion Module, which simulates the fusion of features of multiple pedestrians through cross-attention operations between pedestrian features in the memory bank, but this method also has certain limitations, that is, the module requires additional storage space to save ID representations. This embodiment is dedicated to exploring a more superior data augmentation strategy to further improve the performance of occluded pedestrian re-identification.

[0087] 1.3 Occluded Pedestrian Re-identification Based on ViT

[0088] In recent years, Vision Transformer (ViT) has achieved remarkable results in multiple vision task areas such as image classification, image segmentation, and visual question answering. He et al. were the first to introduce Vision Transformer into the field of person re-identification and proposed the TransReID model. This is the first person re-identification model completely based on the Transformer architecture, completely abandoning the traditional method of using convolutional neural networks (CNNs) for feature extraction. This innovative measure not only broadens the design ideas of person re-identification models but also fully demonstrates the great potential of the pure Transformer architecture in complex vision tasks, while effectively making up for the defect that CNNs cannot obtain long-range related features due to their limited receptive fields. Subsequently, many researchers have conducted more in-depth research and improvement based on TransReID, actively exploring more application scenarios and technological innovations of the Transformer architecture in the field of person re-identification. Among them, Wang et al. used the Transformer as the backbone network and cleverly introduced a pose-guided matching and distribution mechanism, which can accurately identify the visible parts of pedestrians and further enhance the feature representation of the visible parts of pedestrians by introducing semantic views. In addition, Tan et al. also used Vision Transformer as the backbone network and proposed a dynamic prototype mask (DPM) strategy, which can accurately focus on the key parts of the image through dynamic masking without relying on additional models, thus significantly improving the model's ability to handle occluded person re-identification tasks. And Xu et al. took a different approach, using graph matching technology to focus on the visible regions in the image group and cleverly using the Transformer to construct the nearest neighbor feature set to effectively recover the lost information.

[0089] II. CAFIR Model

[0090] This section will introduce the CAFIR model framework for occluded person re-identification, specifically as Figure 3 shown. This framework mainly includes three core modules: the combined data augmentation module, the feature interaction module, and the feature reconstruction module. The combined data augmentation module splices the data of three data augmentations and an original input to form a triple data form to enhance the learning of pedestrian representations; the feature interaction module strengthens the local features and global features successively, and obtains the context relationship features of the pedestrian global features and the overall correlation feature representations of the local features through the way of global and local interactions, strengthening the pedestrian detail features while suppressing some occlusion noises; the feature reconstruction module uses the top k most similar images in the gallery to reconstruct the occluded features of the pedestrian.

[0091] 2.1 Combined Data Augmentation Module

[0092] Data augmentation plays a crucial role in the occluded person re-identification task. It can not only effectively expand the quantity of the dataset but also significantly enhance the diversity of the data. Through data augmentation techniques, pedestrian image samples under various occlusion conditions can be generated, which is crucial for improving the generalization ability and robustness of the model. However, when implementing data augmentation, the possible impact on the basic image details must be carefully considered to ensure that the integrity and stability of the data are not compromised. In this section, a combined data augmentation method will be introduced in detail, which aims to balance the improvement of data diversity and the protection of image details.

[0093] In this embodiment, the combined data augmentation module uses a three-branch input in the form of an image triple to assist the network in learning the feature representation of occluded pedestrians in different scenarios. The specific content is as follows:

[0094] Given any pedestrian image where H, W, and C represent the height, width, and channel size of the image, respectively;

[0095] Retain the original image I base = BA(I), and perform center cropping I cropped = CA(I), regional random value erasing I erased = EA(I), and data augmentation based on prior knowledge I PKOS = PA(I);

[0096] Concatenate the basic augmentation BA(·), cropping augmentation CA(·), erasing augmentation EA(·), and prior-knowledge-based augmentation PA(·) in pairs to form an image triple [I 1 , I 2 , I 3 , and the expression is as follows:

[0097] I 1 = concat(I base , I PKOS ), I 2 = concat(I cropped , I PKOS ), I 3 = concat(I cropped , I erased )

[0098] In the formula: concat(·,·) represents the concatenation operation;

[0099] Use ViT as the backbone network to extract features, and send the feature Figure 3 tuple to a multi-branch backbone network with shared parameters. Divide I 1 into N patches through the sliding window method The expression is:

[0100]

[0101] Where: N H and N W respectively represent the number of slices in the vertical and horizontal directions after segmentation, S represents the step size, P represents the size of each patch, represents the floor function;

[0102] After linear projection, by prepending a learnable classification token I cls aggregate the information from the image patches; before feeding into the Transformer block, add the learnable position embedding to the patch embedding to preserve the position information respectively, and the expression is:

[0103]

[0104] Where: Z 0 represents the input sequence, F(·) represents the linear projection operation, and D is the channel dimension;

[0105] On this basis, Z 0 passes through the Transformer Layer of the l-th layer to obtain the final output of the ViT Among them, represents the classification token of the input image, corresponding to the patch embedding for each patch;

[0106] Let f cls pass through a shared Transformer Layer to obtain I 1 the final global feature representation Similarly, input I 2 and I 3 into the ViT to obtain the global feature representation

[0107] The combined data augmentation strategy improves the adaptability and flexibility of pedestrian feature representation by converting the input of a single branch into a three-branch input containing an image triple and adopting a complementary learning method. This method complements and learns pedestrian feature representation during the training phase, significantly enhancing the model's anti-interference ability to OTP and PTP occlusion noises.

[0108] 2.2 Feature Interaction Module

[0109] The relationships between local human body features and the long-range dependencies between pedestrians and other objects are crucial for constructing robust pedestrian feature representations. However, due to the presence of occlusion noise, many existing methods often focus only on the globally and locally extracted features at the shallow layer during the training phase. Although Vision Transformer (ViT) can effectively capture the long-range dependencies of pedestrians, the features it extracts may ignore the local detailed features, thus misleading the network in accurately extracting pedestrian features in the Person-to-Person (PTP) occlusion pedestrian re-identification task.

[0110] Specifically, ViT is good at extracting globally correlated features and thus obtaining highly correlated global pedestrian representations. However, it has deficiencies in the ability to perceive local details, which may introduce additional noise. To address this challenge, this embodiment designs an enhancement strategy based on feature interaction. This strategy aims to promote in-depth interaction between global and local features, thereby activating the detailed information in pedestrian features.

[0111] Compared with the Person-to-Person (PTP) occlusion problem, the Object-to-Person (OTP) occlusion problem can be better solved in this module. This is because OTP occlusion usually does not cause serious deformation or overlap of the pedestrian area like PTP occlusion. In the PTP occlusion scenario, since both pedestrians have similar shapes and features, it is difficult for the network to accurately distinguish which part is the real pedestrian and which part is the occluded pedestrian. In contrast, for OTP occlusion, because the occluders (such as vehicles, buildings, etc.) have significant differences in shape and features from pedestrians, the network can more easily distinguish them.

[0112] The feature interaction module proposed in this embodiment can not only activate the detailed features that are easily overlooked in the network but also effectively weaken the noise introduced by OTP occlusion. In this way, this module can better balance global and local information and provide a more accurate and robust feature representation for the occlusion pedestrian re-identification task.

[0113] For each image, the global feature and patch embedding derived from ViT are represented as The embedded patches are divided into i groups, and the patch embeddings of each group are concatenated with the f cls classification token. Subsequently, each group of features is processed through a shared Transformer layer to generate local feature representations To further solve the occlusion problem, the feature interaction module is used to learn the relationship between global features and local features, guiding the network to focus on the unoccluded and noise-free pedestrian details in the global features. Referring to Figure 4 , the specific operations are as follows:

[0114] (1) Global-to-local feature interaction

[0115] The fusion of global features and local features realizes effective interaction between the two. Global features provide an overall information overview of the image, while local features focus on the detailed parts of the image. By introducing global features, local features not only rely on the information they carry themselves but are also constrained and guided by the overall information of the entire image. This interaction mechanism enables the model to refer to and utilize the overall information when processing local details, thus significantly enhancing the robustness and accuracy of feature representation.

[0116] Specifically, extract the global feature representation 1 from the image I and extract the local feature representations [f l 1 , f l 2 , f l 3 , …, f l i from the same image, as shown in (a) of Figure 4 ; input the global feature into the GLFI module to enhance the local feature f l i , and this process is expressed by the following formula;

[0117]

[0118] In the formula: Obtained through the conv1×1 operation n represents the number of times passing through the GLFI module, which is the same as the number of local features; GLFI(·) represents the global-to-local feature interaction strategy, and the specific operation is expressed by the formula as:

[0119]

[0120] In the formula: Softmax(·) is the Softmax function, represents the enhanced local feature; fl represents the local feature.

[0121] (2) Local-to-global feature interaction

[0122] After the enhancement of the GLFI module, the obtained local features have incorporated global information, thus demonstrating a high correlation between local and global features. However, the presence of occlusion noise still poses a challenge to the model's recognition ability, requiring the network to focus more on those key detailed features that can accurately distinguish pedestrian identities. This is because in the global features, these key discriminative features may be weakened or even lost due to occlusion or noise. To address this challenge, in this embodiment, the enhanced local features are further fused and interacted with the global features, aiming to improve the expressive ability of the global features so that they can more robustly reflect the unique features of pedestrians.

[0123] As Figure 4 shown in (b) of , the local features in turn enhance the global features, and the enhanced local features

[0124]

[0125] In the formula: represents the enhanced global feature; is obtained through the conv1×1 operation n represents the number of times passing through the LGFI module, which is the same as the number of local features; LGFI(·) represents the local-to-global feature interaction strategy, and the specific operation is expressed by the formula:

[0126]

[0127] Adding to to fuse the global features, we get:

[0128]

[0129] In the formula: f G represents the fused feature, and α represents a hyperparameter.

[0130] (3) Loss function

[0131] To optimize the measurement target and learn more stable feature representations, in this embodiment, triplet loss is used to describe the similarity and difference between pedestrian features, and identity loss is selected to narrow the distance between the model result and the true label. The learning of all global and local features is carried out under constraints. The model is trained using the following loss:

[0132]

[0133] In the formula: represents the identity loss, denotes the triplet loss, y denotes the ground truth, P(·) denotes the probability prediction, and W G 、W g and W L are the prediction classifiers of f G 、 and respectively. and are positive sample pairs, and and are negative sample pairs, and η denotes the margin; denotes the main loss function of the combined data augmentation module, denotes the loss function of the fused features, denotes the loss function of the global features, denotes the loss function of the enhanced local features, denotes the enhanced global features, denotes the global features, denotes the enhanced local features.

[0134] 2.3 Feature Reconstruction Module

[0135] In previous studies on occluded person re-identification, the impacts of OTP and PTP on occluded person re-identification have rarely been concerned. The existence of PTP not only destroys the integrity of the person but also contaminates the true feature representation in the feature space. To reduce the interference of noise, in this embodiment, the visible key features of the complete persons in the gallery are used to reconstruct the features of the occluded persons in the query image, as specifically shown in Figure 5 where the blue dashed box is the retrieved correct sample image, the red dashed box is the retrieved incorrect sample image, the top k most similar gallery images retrieved in the first-round rank sorting are used to reconstruct the occluded part of the query image, and the rank sorting in the last round is the result of the retrieval after the query set images are reconstructed.

[0136] (1) Data Augmentation Based on Prior Knowledge

[0137] This embodiment introduces a powerful data augmentation strategy, namely, PriorKnowledge-based Occlusion Simulation. Specifically, for almost all occluded images in the dataset, the occluders are located in the lower half of the image. Based on this prior knowledge, the upper half of the previous photo of the input photo is used as the occluder for the lower half of the current photo. This method can effectively simulate the two situations where a person is occluded by an object or another person. It is spliced with the original image to form the main branch input, and at the same time, two other branch inputs are set to augment the data and perform complementary learning, so as to ensure that the features of the person will not be completely lost. At the same time, an occlusion label l is set for the replaced part.​​​​​​​​​v , used to determine whether occlusion simulation has been performed. The specific operations are as follows:

[0138]

[0139] In the formula: represents the part of the pedestrian simulated as occluded in the PKOS input branch; represents the original visible part of the pedestrian not simulated as occluded;

[0140] Input I PKOS into the sub-module with a fully connected layer and softmax to predict the visibility score of each part of I PKOS The expression is as follows: In the formula:

[0141]

[0142] In the formula: W fc represents the fully connected layer;

[0143] Set a visibility loss:

[0144]

[0145] In the formula: represents the visibility loss, 2B represents the number after image stitching, l v represents the visibility label, W v is the classifier of the visibility score f v of.

[0146] (2) Feature reconstruction

[0147] After obtaining the visibility score of the pedestrian, retrieve high-quality complete pedestrians from the image library to reconstruct the occluded pedestrian components. Use a ranking list to select the top images to reconstruct the occluded part of the pedestrian. The initial ranking list contains more positive images in the first k positions, which can provide corresponding reliable unoccluded part features for the occluded query image. Retain the unoccluded local features of the pedestrian, and select the unoccluded features of the same pedestrian from the retrieved positive sample pairs in the image library to reconstruct the occluded features of the pedestrian. Use the visibility weighted average method to reconstruct the occluded features; the reconstructed local feature f re is expressed as:

[0148]

[0149] In the formula: respectively represent the visibility score and local feature of the i-th part corresponding to the pedestrian image in the query set; respectively represent the visibility score and local feature of the corresponding part in the k-th sorted list in the image library;

[0150] After feature reconstruction, update the visibility scores of the query image parts and recalculate the distances between the complete query images and gallery images for retrieval, with the expression:

[0151]

[0152] In the formula: dist(·,·) represents the Euclidean distance, n represents the number of local features, argmax represents finding the maximum value, and ‖·‖ represents the L2 norm; represents the enhanced local features of the gallery images.

[0153] The total training loss can be formulated as:

[0154]

[0155] III. Experiments

[0156] To verify the effectiveness of the method proposed in the present invention, comprehensive experimental verifications were carried out on seven datasets in this embodiment. Specifically, three datasets focusing on occlusion situations were selected: Occluded-DukeMTMC, Occluded-reID, and Partial-REID, as shown in Table 1. In addition, two partial datasets, namely P-DukeMTMC-reID and P-ETHZ, were also adopted. Finally, for comprehensive evaluation, two complete datasets: Market-1501 and DukeMTMC were included. Through these diverse datasets, it is aimed to comprehensively and deeply evaluate the performance of the method proposed in the present invention under occlusion, partial occlusion, and complete view conditions.

[0157] 3.1 Experiments and Evaluations

[0158] Occluded-DukeMTMC is currently the largest occluded pedestrian re-identification dataset, which is extremely challenging due to the large number of occluded images it contains. This dataset is subdivided into 15,681 training images (covering 702 identities), 2,210 query images (involving 519 identities), and 17,661 test images (containing 1,110 identities). It is particularly noteworthy that both the query images and the gallery images have occlusion situations, which undoubtedly adds additional difficulty to the occluded pedestrian re-identification task.

[0159] The Occluded-reID dataset was obtained by mobile phone shooting and includes 2,000 images of 200 occluded individuals, with each identity consisting of five full-body photos and five images with different types of severe occlusions.

[0160] P-DukeMTMC-reID is also an important occluded pedestrian re-identification dataset. Its training set contains 12,927 images of 665 identities, the query set contains 2,163 images of 634 identities, and the gallery contains 9,053 images of 634 identities. It should be noted that all query images are in an occluded state, while gallery images show the complete view of the individuals.

[0161] The P-ETHZ dataset is an occluded pedestrian re-identification dataset modified based on ETHZ, containing a total of 3,897 images of 85 identities. Each identity is depicted by an unequal number of images from 1 to 30, including both full-body and occluded images. In this experiment, following the practices of Jia et al. and Zhuo et al., randomly half of the identities' images are selected for training, and the remaining part is used for testing.

[0162] Partial-REID is a partial re-identification dataset whose images cover a variety of perspectives, backgrounds, and occlusion types. The dataset consists of 600 images of 60 identities, with each identity containing 5 full-body images and 5 partial images.

[0163] Market-1501 is a popular identity recognition benchmark dataset composed of 1,501 identities captured by six cameras in front of a supermarket. The dataset is divided into two subsets: the training set contains 12,936 images of 751 identities, and the test set contains 19,732 images of 750 identities. In the test phase, 3,368 query images are selected from the test set of 750 identities, and these query images are used to retrieve the corresponding individuals in the gallery.

[0164] The DukeMTMC-reID dataset contains 36,411 images of 1,404 pedestrians, specifically divided into 16,522 training images, 17,661 gallery images, and 2,228 query images.

[0165] To comprehensively evaluate the performance of various pedestrian re-identification models and existing methods, this experiment uses the Cumulative Matching Characteristic (CMC) curve and the mean Average Precision (mAP) as evaluation metrics, and all experiments are strictly carried out under the single-query setting.

[0166] Table 1 Statistics of the 7 datasets used in the experiment

[0167]

[0168] 3.2 Implementation details

[0169] The proposed CAFIR model is implemented based on the PyTorch platform and is trained and tested in a single RTX 3090 GPU environment. Vision Transformer (ViT) is selected as the backbone network of the model, and this network has been pre-trained on the ImageNet dataset. In the experimental settings, the stride of ViT is set to 16×16, and all input images are uniformly adjusted to the standard resolution of 256×128 pixels. In addition, the batch size is set to 64, where each batch contains 8 different identities and each identity contains 8 images. In terms of optimization, a Stochastic Gradient Descent (SGD) optimizer with a learning rate of 0.002 is adopted, and a momentum of 0.9 and a weight decay of 1e-8 are set. The entire model undergoes 100 training epochs, where the hyperparameters α is set to 0.7 and k is set to 5.

[0170] To ensure the fairness of the evaluation, for datasets without a specific training set, the same test strategy as other scholars is adopted. Specifically, for the Occluded-reID and Partial-ReID datasets, the training set of Market-1501 is used as their training data. For the P-ETHZ dataset, following the methods of Jia et al. and Zhuo et al., images of half of the identities are randomly selected from the entire dataset as the training set, and the remaining part is used as the test set.

[0171] 3.3 Experimental Results

[0172] (1) Experimental Results on Occluded Datasets

[0173] As shown in Table 2, the performance of the method proposed in the present invention is compared with existing methods. These existing methods can be divided into three groups according to the different backbone networks they adopt.

[0174] The first group uses CNN as the backbone network. Among the CNN-based methods, CAAO achieved the best results on the Occluded-Duke dataset, but there is still a certain gap compared with the present invention. In the Occluded-reID dataset, PGFL-KD performed well. In the Partial-REID dataset, PRE and PCB respectively achieved the best Rank-1 and mAP performances.

[0175] The second group are the methods with Transformer as the backbone network. In recent years, Transformer has been increasingly widely used in the field of person re-identification due to its excellent performance. As can be seen from Table 2, FCFormer achieved the best results in terms of Rank-1 and mAP metrics on the Occluded-Duke dataset. In the Occluded-reID dataset, DCG and FCFormer performed well in terms of Rank-1 and mAP metrics respectively. In the Partial-REID dataset, PAT and DCG achieved the best performance in terms of Rank-1 and mAP respectively.

[0176] The third group are the methods that mix Transformer and CNN, applying these two architectures simultaneously to the field of person re-identification. Although this combination can make up for their respective deficiencies to a certain extent, it also leads to an increase in the complexity of the architecture, and thus increases the burden of hyperparameter tuning. Therefore, a large amount of time is required for research to ensure the best performance of the network. As can be seen from Table 2, in the Occluded-Duke dataset, EIE and SPH in the hybrid methods achieved the best performance in terms of Rank-1 and mAP respectively. However, the method proposed in the present invention is 1.9% higher than them both in terms of Rank-1 and mAP. In the Occluded-REID dataset, EIE and SPH achieved the best results in terms of Rank-1 and mAP respectively.

[0177] From the overall comparison, the method proposed in the present invention achieved a Rank-1 accuracy of 74.7% and an mAP accuracy of 65.6% on the Occluded-DukeMTMC dataset, which is better than all existing methods and significantly better than the baseline. In the Occluded-REID dataset, the method proposed in the present invention achieved a Rank-1 accuracy of 86.5% and an mAP accuracy of 80.7%, being better than the existing methods in terms of Rank-1. In the Partial-REID dataset, the method proposed in the present invention achieved a Rank-1 accuracy of 84.8% and an mAP accuracy of 84.0%, being better than some past methods in terms of mAP accuracy.

[0178] Table 2 Performance test results of the CAFIR model and existing methods on three datasets focusing on occlusion situations, namely Occluded-DukeMTMC, Occluded-REID, and Partial-REID

[0179]

[0180]

[0181] (2) Experimental results on the overall dataset

[0182] On the Market-1501 and DukeMTMC-ReID overall datasets, the current existing algorithms are divided into two categories and compared with the algorithm of the present invention. One category is for solving overall pedestrian re-identification, and the other is for occluded pedestrian re-identification. The comparison results are shown in Table 3. The method of the present invention also achieves good performance on the overall dataset. In the Market-1501 dataset, the method proposed in the present invention is higher than the method based on overall pedestrian re-identification in terms of mAP, achieving the best result, improving the mAP accuracy by 0.5%, and performing equivalently in terms of Rank-1 accuracy. In the DukeMTMC dataset, the performance of the method proposed in the present invention is basically consistent with the existing methods. In addition, among the methods based on occluded pedestrian re-identification, the method of the present invention also achieves competitive results. This achievement benefits from the combined data augmentation technology, which not only simulates the scenarios occluded by various obstacles but also ensures the rationality of data augmentation, enabling the network to better learn richer and more comprehensive local features. At the same time, the feature interaction module enables the network to focus on more discriminative features. The good performance on the dataset indicates that the method of the present invention does not overfit and achieves excellent performance in both overall and occluded cases.

[0183] Table 3 Performance test results of the CAFIR model and existing methods on the two overall datasets of Market-1501 and DukeMTMC-reID

[0184]

[0185]

[0186] (3) Experimental results on partial datasets

[0187] On the two partial datasets of P-ETHZ and P-DukeMTMC-reID, the model of this embodiment is compared with the existing methods. In the P-ETHZ dataset. The method proposed in the present invention is significantly better than the existing methods. As shown in Table 4, it achieves an 81.8% Rank-1 accuracy, an 89.4% Rank-5 accuracy, and a 93.5% Rank-10 accuracy. It is worth noting that when compared with the MoS method, the method of the present invention shows good performance in terms of mAP accuracy, with the accuracy improved by 9.5%. On the P-DukeMTMC-reID dataset, the method of the present invention is compared with the existing methods, as shown in Table 5. CAAO achieves a 92.5% Rank-1 accuracy and an 81.4% mAP on P-DukeMTMC-reID, which is better than other previous baselines. And the method of the present invention achieves the best performance with an 82.3% mAP.

[0188] Performance test results of the CAFIR model and existing methods on the P-ETHZ partial dataset

[0189]

[0190] Performance test results of the CAFIR model and existing methods on the P-DukeMTMC-reID partial dataset

[0191]

[0192]

[0193] 3.4 Ablation experiments

[0194] (1) Effectiveness of the proposed modules

[0195] As shown in Table 6, ablation experiments were conducted by adding different modules to the baseline to analyze the effectiveness of the modules proposed in the present invention. The ablation experiments were carried out on the Occluded-Duke dataset. The combined data augmentation (CA) module, feature interaction module (FI), and feature matching (FR) module were successively added to the baseline. When the model only combines the CA module proposed in this embodiment, the Rank-1 and mAP of Index-1 reach 72.2% and 61.0% respectively, which are 8.8% and 5.2% higher than the Rank-1 and mAP of the Baseline. It can be seen that the combined data augmentation can achieve good results in the field of occluded person re-identification. Index-2 only adds the FI module and obtains 70.5% and 61.0% of Rank-1 and mAP respectively, which are 7.1% and 4.1% higher than the baseline method. Index-3, the combination of CA and FI, obtains Rank-1 and mAP of 72.4% and 62.0% respectively, which are 0.2% and 1.0% higher than Index-1 in terms of Rank-1 and mAP. Index-4 is the set of all modules and obtains the best performance, achieving 74.7% and 65.6% in Rank-1 and mAP respectively, with a Rank-1 accuracy 11.3% higher and an mAP accuracy 9.8% higher than the Baseline. The overall results show that the three modules proposed in the present invention are very effective for occluded person Re-ID and can improve the recognition performance.

[0196] Table 6 Ablation experiment results

[0197]

[0198]

[0199] (2) Analysis of hyperparameters

[0200] Figure 6 The left figure in shows the influence of different α values on the recognition performance. The performance was tested by increasing the value of α in steps of 0.1 (ranging from 0 to 1). The results show that the performance reaches its best when the hyperparameter α is set to 0.7. When the occlusion degree is different, the model's dependence on the initial local features and the enhanced global features will vary: The initial local features provide some information on local texture details (such as distinguishable schoolbags, clothes, etc.), while the enhanced global features contain richer global information but ignore some local texture features. To find a balance under different occlusion conditions, a weight α was designed. When α is small, the model relies more on the basic local information, which is more beneficial for unoccluded or slightly occluded situations. However, the dataset as a whole contains many pedestrian images with severe occlusion. As the occlusion degree increases, the initial local features are difficult to provide sufficient effective information. Therefore, a higher α value is needed to make the enhanced global features

[0201] Figure 6 dominant in the fusion. In addition, too small an α value will introduce a large amount of noise, while too large an α value will lose some texture detail features. Therefore, by appropriately adjusting the α value, the model can find the best feature fusion method under different occlusion conditions, thereby improving the recognition accuracy and robustness.

[0202] (3) Visualization

[0203] Such as Figure 7As shown, this figure compares the actual performance of the baseline model and the method proposed in the present invention in the pedestrian re-identification task. The occluded pedestrian images are used as query images, and the top ten most matching images returned by the Baseline model and the CAFIR method are respectively shown. In the comparison figure, the single column of images in the middle shows the occluded query pedestrian image, with the retrieval results of the Baseline model on the left and the retrieval results of the CAFIR method on the right. Specifically, on the Occluded-Duke dataset, the figure details the top 10 retrieval results (Rank list) of each query image under two different models. The green boxes represent the correctly identified positive samples (i.e., the real images of the same pedestrian as the query image), while the red boxes indicate the mis-identified negative samples (images of non-same pedestrians). In the part of the baseline model shown on the left, a large number of red boxes are included in its retrieval results, which means that the baseline model generates more mis-identifications of negative samples during the recognition process. Especially when the pedestrian is occluded, it is difficult for the baseline model to accurately identify the positive samples that match the query image. Especially when pedestrians have similar appearances and lack significant discriminative local detail features, the baseline model can hardly effectively identify the target pedestrian. While in the part of the method proposed in the present invention shown on the right, more green boxes are included in the top 10 retrieval results under the same query conditions, which fully indicates that the method of this embodiment retrieves more positive samples. Compared with the baseline model, its recognition accuracy has been significantly improved. This result shows that when dealing with the occluded pedestrian re-identification task, the present invention demonstrates more excellent performance and can more effectively identify the pedestrians that match the query image. Generally speaking, the method of the present invention shows obvious advantages in the occluded pedestrian re-identification task, can significantly improve the recognition performance, and its overall performance is more excellent than the baseline model.

[0204] As Figure 8 shown, the heatmap visualization processing is respectively performed on the occluded pedestrian pictures in two different scenarios. Among them, on the left side of the dashed line is the pedestrian image occluded by an obstacle, and on the right side is the pedestrian image occluded by other pedestrians. (a) is the heatmap of Baseline, (b) is the heatmap with the combined data augmentation module added, and (c) is the heatmap with the combined data augmentation and feature interaction module added. In the visualization results, red circles are used to mark the occluders to clearly observe the influence of the occluders on pedestrian feature extraction; yellow circles are used to mark the pedestrian features to highlight the performance of the model in feature extraction. Specifically, in Figure 8 subfigure (a), when the baseline model (Baseline) extracts pedestrian features, it extracts some noise information. The existence of this noise information may interfere with the model's accurate capture and recognition of pedestrian features. When the Figure 8After the combined data augmentation module shown in (b), through training with a large amount of augmented data, the network gradually learned to ignore the noise information brought by occlusion. This improvement enabled the network to focus more on the effective extraction of pedestrian features, thereby improving the recognition performance of the model. Further, in Figure 8 In (c), both the combined data augmentation module and the feature interaction module were added. The interaction of these two modules enabled the network to not only ignore occlusion noise when extracting pedestrian features but also notice some more subtle and distinguishable key information of pedestrians. This precise capture of key information and effective separation of noise further improved the recognition accuracy and robustness of the model.

[0205] IV. Conclusion

[0206] In this embodiment, aiming at the problem of various occlusion situations commonly existing in the occluded pedestrian re-identification (Re-ID) task, a network architecture named CAFIR was innovatively proposed. This network architecture utilized the combined data augmentation technology to effectively expand the pedestrian image dataset under different occlusion scenarios, thereby improving the efficiency and accuracy of the backbone network in extracting pedestrian features. In addition, CAFIR introduced a feature interaction module, which played a crucial role in capturing and distinguishing key detail information in pedestrian features. In the feature reconstruction module, the network further utilized these extracted key information to select complete pictures from the image library to reconstruct the occluded part of the pedestrian, thereby enhancing the robustness and generalization ability of the model. To comprehensively verify the performance of the CAFIR model, extensive experiments were conducted on three occluded datasets, two partial datasets, and two whole datasets. The experimental results show that compared with existing methods, the CAFIR model exhibits significant superiority, not only being able to accurately perform pedestrian re-identification under occlusion but also performing well when dealing with non-occluded data. CAFIR has achieved remarkable results in the field of occluded pedestrian Re-ID, but there is still room for further improvement in designing a more lightweight and simplified model to address the deployment challenges on low-resource devices. In view of this, subsequent research can focus on in-depth exploration of lightweight technologies, aiming to develop a more streamlined and efficient CAFIR model by reducing the computational load and shortening the training time to meet the diverse needs in practical applications.

[0207] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts among the embodiments can be referred to each other.

[0208] The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for re-identifying occluded pedestrians based on combined data enhancement and feature interactive reconstruction, characterized in that: The following steps are involved: Collect a dataset of occluded pedestrian re-identification with different identities and divide it into training set and test set according to the preset ratio; Design the CAFIR model framework, which includes a combined data enhancement module, a feature interaction module, and a feature reconstruction module; The training set is input into the CAFIR model in batches for training until the loss function converges to obtain the optimized CAFIR model; The optimized CAFIR model is used to perform person re-identification on the test set to obtain the recognition results.

2. The method for re-identifying occluded pedestrians based on combined data enhancement and feature interactive reconstruction according to claim 1, characterized in that: The combined data augmentation module uses a three-branch input in the form of image triplets to assist the network in learning the feature representation of occluded pedestrians in different scenarios. The specific contents are as follows: Given any pedestrian image Among them, H, W and C represent the height, width and channel size of the image respectively; Keep the initial image I base =BA(I), and perform center cropping I on the initial image cropped =CA(I), regional random value erase I erased =EA(I) and data augmentation based on prior knowledge I PKOS =PA(I); The basic enhancement BA(·), cropping enhancement CA(·), erasing enhancement EA(·) and prior knowledge-based enhancement PA(·) are concatenated in groups of two to form an image triplet [I1, I2, I3], which is expressed as follows: I1=concat(I base ,I PKOS ),I2=concat(I cropped ,I PKOS ),I3=concat(I cropped ,I erased ) Where: concat(·,·) represents the concatenation operation; ViT is used as the backbone network to extract features, and the feature map triples are sent to the parameter-sharing multi-branch backbone network. I1 is divided into N patches by the sliding window method. The expression is: Where: N H and N W Respectively represent the number of vertical and horizontal slices after segmentation, S represents the step size, and P represents the size of each patch. represents the floor function; After the linear projection, a learnable classification label I is prepended cls Aggregate information from image patches; embed learnable positions before feeding into Transformer blocks Added to the patch embedding to preserve the position information separately, the expression is: Where: Z0 represents the input sequence, F(·) represents the linear projection operation, and D is the channel size; Z0 passes through the Transformer Layer of the lth layer to obtain the final output of ViT in, represents the classification token of the input image, Patch embedding corresponding to each patch; f cls After a shared Transformer Layer, the final global feature representation of I1 is obtained Similarly, input I2 and I3 into ViT to obtain the global feature representation 3. The method for re-identifying occluded pedestrians based on combined data enhancement and feature interactive reconstruction according to claim 2, characterized in that: The feature interaction module uses a two-way interactive modeling method of global features and local features to assist the network in extracting local detail features of the visible part of pedestrians and weakening occlusion noise. The specific contents are as follows: Extracting global feature representation from image I1 And extract local feature representation from the same image Global Features Input into the GLFI module to enhance the local features f l i , this process is expressed by the following formula; Where: f l i After conv1×1 operation, we get n represents the number of times it passes through the GLFI module, which is the same as the number of local features; GLFI(·) represents the global to local feature interaction strategy, and the specific operation is expressed by the formula: Where: Softmax(·) is the Softmax function, represents the enhanced local features; fl represents the local features; The enhanced local features Integrating into the global features, the expression is as follows: Where: Represents the enhanced global features; After conv1×1 operation, we get n represents the number of times the LGFI module is passed, which is the same as the number of local features; LGFI(·) represents the local to global feature interaction strategy, and the specific operation is expressed by the formula: Will Add to By integrating the global features, we get: Where: f G represents fusion features, and α represents a hyperparameter.

4. The method for re-identifying occluded pedestrians based on combined data enhancement and feature interactive reconstruction according to claim 1, characterized in that: The feature reconstruction module uses the first k most similar images in the library to reconstruct the obscured features of pedestrians. The specific contents are as follows: Using occlusion simulation based on prior knowledge, the upper half of the previous photo of the input photo is used as the occlusion of the lower half of the current photo; an occlusion label l is set for the replaced part v , used to determine whether occlusion simulation has been performed. The specific operations are as follows: Where: represents the pedestrian part simulated as occluded in the PKOS input branch, Represents the original visible part of the pedestrian that is not simulated as occluded; Will I PKOS Input to the submodule with fully connected layer and softmax, predict I PKOS Visible score for each section The expression is as follows: Where: W fc represents a fully connected layer; The unobstructed local features of pedestrians are retained, positive samples are retrieved from the image library and unobstructed features of the same pedestrian are selected from them, and the obstructed features are reconstructed using the visibility weighted average method; the reconstructed local features f re It is expressed as: Where: They represent the visible score and local features of the i-th part corresponding to the pedestrian image in the query set respectively; They represent the visible score and local features of the corresponding part of the kth sorted list in the gallery; After feature reconstruction, the visibility score of the query image part is updated and the distance between the full query image and the gallery image for retrieval is recalculated as: Where: dist(·,·) represents the Euclidean distance, n represents the number of local features, argmax represents the maximum value, and ‖·‖ represents the L2 norm; Represents the enhanced local features of the gallery image.

5. The method for re-identifying occluded pedestrians based on combined data enhancement and feature interactive reconstruction according to claim 1, characterized in that: The expression of the loss function is: in, Where: represents the visibility loss, 2B represents the number of images after splicing, and l v Indicates visible label, W v is the visible score f v Classifier of Indicates identity loss, represents the triple loss, y represents the true value, P(·) represents the probability prediction, and W G , W g and W L They are f G , and The prediction classifier, and is a positive sample pair, and is a negative sample pair, η represents the margin; represents the main loss function of the combined data enhancement module, represents the loss function of fusion features, Represents the loss function of global features, represents the enhanced local feature loss function, represents the enhanced global features, Represents global features, Represents the enhanced local features.

6. The method for re-identifying occluded pedestrians based on combined data enhancement and feature interactive reconstruction according to claim 1, characterized in that: The CAFIR model is implemented based on the PyTorch platform, and training and testing are completed in a single RTX 3090 GPU environment.

7. The method for re-identifying occluded pedestrians based on combined data enhancement and feature interactive reconstruction according to claim 1, characterized in that: During the training process of the CAFIR model, a stochastic gradient descent optimizer with a learning rate of 0.002 was used, a momentum of 0.9 and a weight decay of 1e-8 were set, and the training cycle was 100.

8. The method for re-identifying occluded pedestrians based on combined data enhancement and feature interactive reconstruction according to claim 1, characterized in that: The evaluation indicators of the optimized CAFIR model are: cumulative matching characteristic curve and average precision mean.

Images