An occluded pedestrian re-identification method based on pose guidance and progressive feature repair

Abstract

The application discloses an occluded pedestrian re-identification method based on pose guidance and progressive feature repair, and comprises the following steps: obtaining an original pedestrian re-identification dataset, and performing occlusion simulation and enhancement processing on the dataset; using a pre-trained pose estimation model to extract human key points and performing a many-to-one selection strategy; constructing a skeleton-contour image and generating a visibility pseudo-label; constructing a feature extraction backbone network based on a Vision Transformer architecture; implementing attention guidance based on skeleton information; constructing a visibility estimation branch network containing a full connection layer; constructing a hybrid loss function containing an identity loss, a triplet loss and a visibility loss, and performing hybrid supervision joint training; performing inference and feature repair based on a progressive strategy, and outputting final pedestrian re-identification retrieval results. The application effectively solves the problems of semantic interference and feature loss in the occlusion scene, and improves the recognition robustness and accuracy.

Description

Technical Field

[0001] This invention relates to the fields of computer graphics and deep learning technology, and specifically to a method for re-identifying occluded pedestrians in complex occluded scenes, particularly based on pose guidance and progressive feature repair. Background Technology

[0002] Conventional pedestrian re-identification methods typically rely on the presupposition that "the pedestrian image is fully visible." However, in real-world surveillance scenarios, target pedestrians are inevitably obscured by objects (such as vehicles, trees, and walls) or other pedestrians, leading to missing or corrupted information about their appearance. Addressing this issue, occluded pedestrian re-identification technology has become a research hotspot. Existing common strategies for solving the occlusion problem include using artificially occluded images to enhance training data and extracting discriminative features of the visible region through feature alignment. These methods are generally effective when dealing with object-to-person (OTP) occlusion (i.e., the occluding object is a static object such as a vehicle or wall).

[0003] However, in handling more complex "person-to-person" (PTP) occlusion scenarios, where the target pedestrian is partially occluded by another pedestrian, the recognition performance of existing technologies is severely limited. On one hand, PTP occlusion introduces a high degree of semantic interference between the foreground occluder and the target pedestrian (i.e., the primary retrieval target). Unlike static occlusions such as vehicles and trees, the foreground pedestrian (the occluder) and the target pedestrian (the occluded) share a high degree of semantic similarity visually, both containing similar human structural features such as heads and limbs. Without an effective differentiation mechanism, existing feature extraction models are prone to attention drift, incorrectly focusing on the foreground occluder rather than the target in the background, resulting in a mismatch between the extracted feature representation and the true target.

[0004] Existing methods typically rely solely on manually defined overlapping regions of PTP-occluded images during data augmentation to generate visibility labels, guiding the model to focus on features of the unoccluded parts. However, this generation method is mechanical and simplistic, ignoring the natural PTP occlusion present in the original dataset. This results in mismatched part-level visibility labels, failing to provide accurate supervision signals for model training. Furthermore, existing feature inpainting strategies employ a single, global weighted average approach to fill in occluded regions using library features. This coarse approach not only easily introduces irrelevant feature noise but also ignores prior knowledge of human anatomy (e.g., in PTP scenarios, the probability of a pedestrian's lower body being occluded is typically significantly higher than their upper body), making it difficult for the model to achieve high-precision feature reconstruction while preserving fine-grained local details. Summary of the Invention

[0005] The purpose of this invention is to provide an occluded person re-identification method based on pose guidance and progressive feature repair. It utilizes human pose estimation results as prior knowledge, accurately locates the main target through a many-to-one selection strategy, guides the network to focus on the effective region using skeleton-silhouette images (SSI), and refines the occluded features during the inference stage through a progressive feature repair mechanism, thereby improving the retrieval performance of the model in complex occluded scenarios and solving the problems mentioned in the background art.

[0006] The specific technical solution is as follows: An occluded pedestrian re-identification method based on pose guidance and progressive feature repair, including the following steps:

[0007] Step S1: Obtain the original pedestrian re-identification dataset, perform occlusion simulation enhancement processing on the dataset, and construct the enhanced training sample dataset.

[0008] Preferably, the specific implementation process includes:

[0009] Step S11: Obtain the original pedestrian re-identification dataset. For each pedestrian image in the dataset, select another pedestrian as the occlusion source image.

[0010] Step S12: Set a probability threshold to achieve probability-based hybrid occlusion data enhancement;

[0011] Step S13: Combine the enhanced image with the original unenhanced image to form the final training sample set output.

[0012] Step S2: Use the pre-trained pose estimation model to extract human key points from the images in the training sample dataset, and execute a many-to-one selection strategy to automatically filter out the main retrieval targets in the images.

[0013] Preferably, this step includes two stages: the attitude key point extraction stage and the many-to-one selection algorithm execution stage.

[0014] Pose key point extraction stage: The input image is processed using a pre-trained pose estimation model to extract the pose information of all detected pedestrians in the image and output the set of key point coordinates for each pedestrian.

[0015] The many-to-one selection algorithm stage: For each detected pedestrian, calculate the geometric centroid of all its visible key points, and calculate the absolute distance between the horizontal coordinate of the centroid and the horizontal center of the image. Set a horizontal distance threshold, and use a hierarchical strategy to determine the main search target based on the distribution of pedestrians detected in the image.

[0016] Based on the filtering results of the many-to-one selection algorithm, only the pose key point data of the locked main search targets are retained, and the key point information that is judged to be interfering with pedestrians is filtered out.

[0017] Step S3: Construct a skeleton-contour image and generate visibility pseudo-labels.

[0018] Preferably, constructing the skeleton-contour image includes: using the data-enhanced image as a base, connecting adjacent key points based on the extracted key point coordinates and the human pose topology, directly drawing colored skeleton lines on the enhanced image to generate an enhanced image with superimposed skeleton information; performing pixel-by-pixel difference calculation between the superimposed image and the original enhanced image, and calculating the sum of the absolute differences in the three RGB channels for each pixel position in the image to obtain a single-channel cumulative difference map; setting a binarization threshold and performing binarization processing on the cumulative difference map to obtain a preliminary mask; finally, using morphological dilation operation to process the preliminary mask to generate the final binarized skeleton-contour image.

[0019] Generating visibility pseudo-labels involves: uniformly dividing the final binarized skeleton-contour image vertically into... For each horizontal strip region, the sum of the pixels of each horizontal strip is calculated. A "top-down" scanning strategy is used to locate the occlusion boundary. The first strip with a pixel sum of 0 is selected as the cutoff strip. After determining the cutoff strip, the visibility pseudo-labels of each horizontal strip are generated based on the prior knowledge of the continuity of human structure.

[0020] Step S4: Construct a feature extraction backbone network based on the Vision Transformer architecture.

[0021] Preferably, the specific implementation process includes:

[0022] Step S41: Divide the input image into non-overlapping image blocks of fixed size. Through a learnable linear projection layer, map each two-dimensional image block into a one-dimensional embedding vector to form the original image block token sequence.

[0023] Step S42: The learnable location is embedded and superimposed into the tile token sequence, and then the sequence is input into the Transformer encoder for interaction and extraction of global context information, and the output includes a category token for classification and a tile token sequence containing image content.

[0024] Step S43: Based on the spatial distribution of the original image, the tile token sequence output by the encoder is spatially rearranged to restore it to a two-dimensional feature map with the original spatial structure;

[0025] Step S44: Perform multi-granularity feature extraction based on the two-dimensional feature map, extract the global feature vector through global average pooling, and utilize horizontal uniformity. The equal-division pooling strategy extracts multiple local feature vectors.

[0026] Step S5: Implement attention guidance based on skeleton information.

[0027] Preferably, the specific implementation process includes:

[0028] Step S51: Perform max pooling on the generated binarized skeleton-contour image to generate an attention mask sequence that corresponds in number to the tile token sequence;

[0029] Step S52: Two attention computation paths are executed in parallel at the first layer of the Transformer encoder. One path performs standard self-attention computation to capture global contextual information, and the other path uses attention mask sequences to remove background interference and focus on target human features.

[0030] Step S53: Introduce a learnable balancing parameter to weight and fuse the outputs of the two attention calculation paths to obtain the final first-layer output features.

[0031] Step S6: Construct a visibility estimation branch network containing fully connected layers.

[0032] Preferably, the visibility estimation branch network includes:

[0033] The first fully connected layer receives local feature vectors as input and connects to a ReLU activation function at the output, which is used to introduce nonlinear transformation and enhance the discriminative representation ability of features;

[0034] The second fully connected layer receives the feature vector output from the first fully connected layer, further reduces its dimensionality and maps it, and the output is also connected to the ReLU activation function.

[0035] The third fully connected layer serves as the classification output layer. It receives the feature vector output from the second fully connected layer and maps it to a 2-dimensional classification prediction vector. The two dimensions of this vector correspond to the two state categories of "occlusion" and "visibility".

[0036] The output vector of the third fully connected layer is normalized using the Softmax function to obtain the predicted probability distribution of the local feature belonging to the "occluded" and "visible" categories.

[0037] Step S7: Construct a hybrid loss function that includes identity loss, triplet loss, and visibility loss, and perform hybrid supervised joint training.

[0038] Preferably, the hybrid loss function includes:

[0039] Identity recognition loss: Using the real pedestrian identity labels provided by the dataset, calculate the cross-entropy loss for the global feature vector and each local feature vector of each input image output by the backbone network;

[0040] Triple loss metric: Construct triple samples in the feature space based on identity labels, including anchor samples, positive samples of the same class and negative samples of different classes, and calculate triple loss;

[0041] Visibility estimation loss: Using the generated fine-grained visibility pseudo-labels as supervision ground truth, the cross-entropy loss is calculated on the prediction score output by the visibility estimation branch.

[0042] Step S8: Perform inference and feature repair based on a progressive strategy, and output the final pedestrian re-identification retrieval results.

[0043] Preferably, the specific implementation process includes:

[0044] Step S81: Obtain the global and local feature representations and visibility scores of the query image and the images in the library; calculate the standard Euclidean distance based on the global features; and obtain the initial K-nearest neighbor sample set of the query image.

[0045] Step S82: Perform progressive feature repair iteration: In each iteration, a reference feature vector is obtained by weighted aggregation of K-nearest neighbor features based on visibility score. The reference feature vector is used to perform weighted fusion repair on the local features of the query image. Based on the repaired local features, the visibility-weighted Euclidean distance is calculated to re-retrieve and update the K-nearest neighbor sample set. The repair continues in the next round until the iteration is completed.

[0046] Step S83: Calculate and sort the visibility-weighted Euclidean distance between the query image and the image in the library using the local features of the finally repaired query image, and output the final pedestrian re-identification retrieval results.

[0047] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0048] (1) Solved the problem of target locking and anti-interference in multi-person scenes: By introducing a many-to-one selection algorithm based on pose prior, this invention can accurately distinguish between targets and occlusions in complex images with multiple people, significantly improving the target localization accuracy of the model in the "person-to-person" (PTP) scene and overcoming the defect of attention drift in traditional methods.

[0049] (2) Provides accurate occlusion perception guidance and supervision signals. This invention uses skeleton-contour images as strong attention masks to force the network to suppress noise in the background and occluded areas at the pixel level, focusing on the target human body structure. At the same time, the visibility pseudo-labels generated based on skeleton-contour images can truly reflect the original physical occlusion state of the image, correcting the problem of inaccurate labels caused by relying solely on data augmentation in existing technologies, and providing high-quality supervision information for the training of the visibility estimation branch.

[0050] (3) Achieving refined progressive feature repair. To address the problem that traditional feature repair methods are prone to introducing noise, this invention proposes a progressive feature repair strategy based on prior knowledge of human structure. Through top-down iterative logic, high-visibility clear features are used to gradually guide low-visibility occluded areas for feature repair. During the inference process, the retrieved features are continuously optimized, thereby generating a more complete and accurate target feature representation, which significantly improves the recognition accuracy under complex occlusion.

[0051] (4) It has good generalization performance and robustness. Experimental results on several mainstream occluded pedestrian re-identification datasets show that the method of the present invention has strong robustness and high practical application value. Attached Figure Description

[0052] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0053] Figure 1 A flowchart illustrating the steps of the occluded pedestrian re-identification method based on pose guidance and progressive feature repair provided in this embodiment of the invention.

[0054] Figure 2 A schematic diagram illustrating the principle of the many-to-one selection algorithm provided in an embodiment of the present invention;

[0055] Figure 3 This is a diagram illustrating the skeleton-contour image and the process of generating visibility pseudo-labels provided in an embodiment of the present invention.

[0056] Figure 4 This is an overall network architecture diagram provided for embodiments of the present invention;

[0057] Figure 5 A flowchart of progressive feature repair reasoning provided in an embodiment of the present invention;

[0058] Figure 6 A comparison chart of visual search results provided in an embodiment of the present invention;

[0059] Figure 7A schematic diagram showing the effect comparison of the skeleton-contour image-guided attention mechanism provided in the embodiments of the present invention. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] Combination Figure 1 As shown, the present invention provides a technical solution: a method for re-identifying occluded pedestrians based on pose guidance and progressive feature repair, comprising the following specific steps:

[0062] Step S1: Data acquisition and augmentation processing.

[0063] In this embodiment, to construct an augmented dataset for training the occluded pedestrian re-identification model, the present invention employs an occlusion simulation strategy based on prior knowledge. This strategy utilizes existing pedestrian datasets as a foundation and constructs training samples containing occlusion scenarios such as OTP and PTP through synthetic means. Specifically:

[0064] Step S11: Sample pair selection and formal definition.

[0065] For example, the original pedestrian re-identification dataset is defined as follows: ,in Representing the Zhang Xingren's image, This represents the identity tag corresponding to the image. This represents the total number of images in the dataset. During the training phase, for each original image to be processed in a batch... Another image with a different identity is randomly selected from the same batch as the occlusion source (denoted as the occlusion source image). ),satisfy .

[0066] Step S12: Hybrid Occlusion Simulation Based on Probabilistic Control

[0067] For example, a probability threshold is set. (In this embodiment, the following is set) This is used to control the generation of occlusion types, in order to balance the model's learning ability for different occlusion modes:

[0068] (1) Simulate the "Object-Opped Person" (OTP) scenario (using probability) implement):

[0069] Simulation objective: To simulate the occlusion of pedestrians by static objects such as cars, trees, and walls. Since the visual semantic features of pedestrians differ significantly from those of static objects, directly overlaying unprocessed images of the occluding pedestrians would introduce human semantic interference instead of the static object interference required for the OTP (Optical Time of Pedagogy) scenario, leading to training bias.

[0070] Specific steps: Select the occlusion source image Spatial structure disruption processing is performed, specifically by swapping the upper and lower regions of the image, thereby destroying its original human body structure and making its texture features semantically closer to a disordered, static object. Subsequently, a rectangular region is randomly cropped from the processed image and overlaid onto the target image. Generate samples simulating OTP occlusion at any position.

[0071] (2) Simulate the "person-to-person" (PTP) scenario (using probability) implement):

[0072] Simulation Objective: To simulate the complex situation where a foreground pedestrian occludes a target pedestrian in a surveillance scenario. Based on prior knowledge of the spatial distribution of the surveillance viewpoint, this type of occlusion typically manifests as the upper body of a foreground pedestrian obscuring the lower body or torso of a target pedestrian in the background.

[0073] Specific steps: Maintain the image of the occluded source. The complete semantic structure of the human body remains unchanged, from the occluded source image A rectangular block is randomly cropped from the upper body area and then overlaid onto the target image. The operation involves the lower body or torso area. This operation introduces occlusion while preserving the "human" attribute of the occluded object, thereby guiding the model to learn to distinguish the semantic features between the target pedestrian and the foreground occluded pedestrian during training.

[0074] Step S13: Data output.

[0075] For example, the image after the above enhancement process is denoted as... This image inherits from the original target image. Identity tags This, along with the original unenhanced images, forms the final training sample set, which serves as the input data for subsequent neural networks.

[0076] Step S2: Attitude estimation and many-to-one selection strategy.

[0077] In this embodiment, the present invention utilizes human posture information as a priori to solve the problems of blurred localization of the main retrieval target and interference from occluded pedestrians in overlapping images of multiple people. The specific implementation process is divided into the following two stages.

[0078] Pose keypoint extraction stage: A pre-trained pose estimation model (OpenPose model chosen in this embodiment) is used to process the input image and extract the pose information of all detected pedestrians. The model outputs a set of keypoint coordinates for each pedestrian, denoted as... ,in, An index for pedestrian instances. For key point indexing, specifically Representing the The first pedestrian The height and width coordinates of each key point in the image coordinate system.

[0079] Execution of the many-to-one selection algorithm phase: such as Figure 2 As shown, the core logic of the many-to-one selection algorithm is based on the prior observation that "the geometric center of the main target's body is usually closer to the geometric center of the overall image." The specific calculation and judgment process is as follows:

[0080] (1) Centroid calculation: For the first For each pedestrian, calculate the geometric centroid of all visible keypoints. :

[0081]

[0082] in, This indicates the number of valid (visible) keypoints detected for the pedestrian. The height and width coordinates represent the centroid.

[0083] (2) Center distance measurement: Calculate the horizontal coordinates of the centroid. With the horizontal center of the image absolute distance between :

[0084]

[0085] in The width of the input image.

[0086] (3) Target locking and filtering: Set a horizontal distance threshold (In this embodiment, for a resolution of Image settings To determine the distribution of pedestrians detected in the image, a hierarchical strategy is employed:

[0087] Centrality-based determination: If the calculation result shows and This indicates that the first pedestrian is significantly off-center from the image, while the second pedestrian is located in the central area. Therefore, the second pedestrian is directly identified as the primary retrieval target.

[0088] Determination based on vertical position: If the calculation result shows and This indicates that both pedestrians are located in the central region of the image, meaning there is significant occlusion between them. Based on the prior knowledge of the monitoring perspective—that the occluded main target is usually higher than the foreground occluder on the image plane—the height centroid coordinates of the two are compared. In the image coordinate system (origin at the top left corner), select... Pedestrians with smaller values ​​(i.e., those in higher geometric positions) are the primary search targets.

[0089] For determining three or more pedestrians: If three or more pedestrians are detected in the image, to reduce the computational complexity of the algorithm in extremely crowded scenes and ensure accurate locking of the core target, the minimum distance criterion is directly adopted, selecting the distance from the geometric center of the image in the horizontal direction. The smallest pedestrian is the primary search target.

[0090] Based on the filtering results of the many-to-one selection algorithm, the system retains only the attitude key point data of the locked main search targets and filters out the key point information that is judged to interfere with pedestrians, laying the foundation for generating accurate guidance signals in subsequent steps.

[0091] Step S3: Construct skeleton-contour image (SSI) and generate visibility pseudo-labels.

[0092] In this embodiment, as Figure 3 As shown, after filtering out the main search targets, the present invention generates auxiliary information to guide network attention and provide visibility supervision signals. The specific implementation process includes two steps: constructing a skeleton-contour image and generating visibility pseudo-labels.

[0093] Skeleton-contour image generation: In order to transform discrete skeleton key points into binary semantic masks that are strictly aligned with the pedestrian spatial position of the original input image and guide the network's attention focus, this embodiment adopts a generation strategy based on "image overlay difference".

[0094] First, with data-enhanced images Using the extracted keypoint coordinates as a base, adjacent keypoints are connected according to the human pose topology in the image. Colored skeleton lines are drawn directly on top, thus generating an enhanced image with skeleton information superimposed, denoted as . .

[0095] Secondly, in order to extract clean skeletal regions and eliminate background texture interference, overlay images were used. Compared with the original enhanced image Perform pixel-by-pixel difference calculation. Since the pixel values ​​in the non-skeleton regions of the two images are completely identical, their difference is 0; however, in the skeleton regions, the pixel values ​​differ. Therefore, for each pixel position in the image... Calculate the sum of the absolute differences across the three RGB channels to obtain the cumulative difference map for a single channel. :

[0096]

[0097] in, , , Represents the image height. Represents the image width. Indicates the color channel index.

[0098] Next, set the binarization threshold. (Settings in this embodiment) ), for cumulative difference plot Binarization is performed to obtain the initial mask. :

[0099]

[0100] Finally, to compensate for the sparseness of the skeletal lines and obtain more robust human body region coverage, a morphological dilation operation was used to adjust the initial mask. Perform processing. Define structural elements. (This embodiment uses) (A matrix of all 1s) is used to generate the final binary skeleton-contour image. This operation can be represented as:

[0101]

[0102] In the generated In the image, areas with a pixel value of 1 precisely correspond to the area where the target human skeleton is located, while areas with a pixel value of 0 correspond to the background or occluded areas.

[0103] For example, the generation of visibility pseudo-labels: In order to supervise the training of the visibility estimation branch in subsequent networks, this invention utilizes The characteristics generate fine-grained visibility pseudo-label vectors. First, the binarized image is... Divide the vertical height evenly into A horizontal strip region (e.g.) For the first... Horizontal stripes Calculate the total number of pixels. :

[0104]

[0105] in, This is a strip index. Considering that in "person-to-person" (PTP) scenes, human occlusion typically exhibits a physical distribution pattern of "visible at the top, occluded at the bottom," this embodiment employs a "top-down" scanning strategy to locate the occlusion boundaries. Specifically, it finds the first pixel sum. The band with a value of 0 is used as the cutoff band. :

[0106]

[0107] in, This is the index of the cutoff strip. Once the cutoff strip is determined... Based on the prior knowledge of the continuity of human body structure, the following logic is used to generate the first... Visibility pseudo-labels of local features :

[0108]

[0109] Step S4: Construct the feature extraction backbone network.

[0110] In this embodiment, a neural network based on the Vision Transformer architecture is used as the backbone for feature extraction, receiving data-enhanced input images. (In this embodiment, the input size is normalized to) The process involves processing the data and outputting a feature vector containing semantic information. The specific processing flow is as follows:

[0111] Step S41: Image segmentation and sequence embedding.

[0112] For example, the input image Divided into A fixed-size, non-overlapping image patch. Let the patch size be... (In this embodiment) Then the total number of blocks The calculation is as follows: Subsequently, each 2D image block is mapped to a dimensionality of [dimensionality missing] through a learnable linear projection layer. One-dimensional embedding vector (in this embodiment) This forms the original tile token sequence.

[0113] Step S42: Encoder processing and feature extraction.

[0114] For example, to preserve the spatial location information of the image, learnable position embeddings are overlaid onto the tile token sequence. This sequence is then fed into a Transformer encoder containing multiple layers of self-attention mechanisms for deep feature interaction.

[0115] Specifically, to guide the model to focus on the main target using the generated skeleton-contour image (SSI), the multi-head self-attention module in the encoder is configured to support masked input. After processing by the encoder, the output is a feature sequence. The sequence consists of one class token for classification and... It consists of a tile token containing image content.

[0116] Step S43: Feature Reshape.

[0117] For example, unlike traditional Vision Transformers which only use category tokens for classification, this embodiment discards category tokens and retains only the category tokens in order to preserve the spatial structure of the original image so that local features of the corresponding locations in the original image can be obtained later. Each tile token is rearranged according to the spatial distribution of the original image to repair the spatial relationships of the features and reshape it into a feature tensor. .in, , representing the spatial resolution of the feature map.

[0118] Step S44: Multi-granularity feature generation.

[0119] For example, based on the reshaped feature map Global and local features are extracted separately to construct multi-granularity feature representations:

[0120] Global Feature Extraction: To obtain the overall semantic representation of a pedestrian, the entire feature map is processed. Apply global average pooling (GAP) to generate a global feature vector. :

[0121]

[0122] Local Feature Extraction: To capture fine-grained body part features (such as head, torso, legs, etc.), a horizontally uniform... Equal-division pooling strategy. Specifically, using an output size of... The adaptive average pooling layer divides the feature map vertically into... Horizontal stripes (in this embodiment) (This is consistent with the number of regions divided in step S3).

[0123] The calculated local feature matrix Represented as:

[0124]

[0125] The local features obtained through this operation are: , where each local feature vector It aggregates the semantic information of the corresponding horizontal regions in the feature map.

[0126] Step S5: SSI-based attention guidance mechanism.

[0127] In this embodiment, the present invention utilizes the generated skeleton-contour image (SSI) as prior information to guide the backbone network to focus on the main retrieval target in the early stages of feature extraction, while suppressing interference from background and occlusion noise. This embodiment introduces a mask-guided attention module in the first layer of the backbone network's Transformer encoder, employing a "dual-path parallel and adaptive fusion" strategy.

[0128] Step S51: Patch mapping and sequence alignment of the mask.

[0129] For example, in order to inject pixel-level skeleton information into a patch-based Transformer, the binarized skeleton-contour image generated in step 3 needs to be processed. Convert to input image The sequence of tile tokens corresponds to the mask sequence in terms of quantity. The specific processing flow is as follows:

[0130] Max pooling downsampling: Given the binarized pixel-level mask generated in step 3 (In this embodiment, it is) The kernel size and step size are both [missing information]. The max pooling operation. This operation ensures that as long as... The image area contains any target human pixel (i.e. The maximum value in this region is 1, and the corresponding mask value for this tile is 1. This process yields a tile-level mask sequence. ,in , with input image Maintain consistency.

[0131] Category Token Alignment: Since the feature sequence of the input Transformer contains a Category Token (CLS Token) for aggregating global information, in order to maintain dimensionality consistency and not affect the functionality of the CLS Token, in The first part is appended with a scalar value of 1 (indicating that the CLS Token is always visible). This concatenation yields the final mask sequence. (That is, the length is 197).

[0132] Step S52: Parallel computation of dual-path attention.

[0133] For example, in the first layer of the Transformer encoder, two attention computation paths are executed in parallel to capture global context information and target human focus information, respectively:

[0134] (1) Standard self-attention path: This path is used to preserve the global contextual information of the image, without imposing any restrictions on the attention matrix, and generates a query matrix given the input feature sequence. Key matrix Sum matrix Calculate standard attention output :

[0135]

[0136] (2) Mask-guided attention path: This path utilizes the mask tensor generated in step S51. Forced background information filtering. The specific calculation logic is as follows: First, calculate the original attention score matrix. Then, a masked fill operation is applied: for The position with a median value of 0 (corresponding to the background area) will The corresponding fraction in the equation is replaced with negative infinity. ).for The position with a median value of 1 (corresponding to the human body area and CLS Token) is reserved. The original scores in the matrix are then calculated. Finally, Softmax normalization is performed on the padded matrix. Since the scores at the background positions are... The weights after Softmax will become 0. This process can be represented as:

[0137]

[0138] Through this operation It aggregates only the features of the human skeleton area and CLS Token, completely shielding the interference of background and occlusion noise.

[0139] Step S53: Adaptive fusion of features.

[0140] For example, in order to find the optimal balance between "focusing on the target" and "preserving necessary context," this embodiment introduces a learnable balance parameter. (In this embodiment, (Initial value: 0.6). Final first-layer output features. The weighted fusion of the outputs from the two paths above yields:

[0141]

[0142] in, These are the automatically learned weight coefficients. This mechanism allows the model to dynamically adjust the focus on target features based on the occlusion complexity of specific samples. With global awareness features The degree of dependence is determined to achieve robust feature extraction. The output features are then fed into subsequent layers for further feature extraction and refinement, ultimately yielding the global and local feature outputs of step S4.

[0143] Step S6: Construction of the visibility estimation branch.

[0144] In this embodiment, to achieve a quantitative assessment of the occlusion state of local feature regions, a lightweight visibility estimation branch is constructed in parallel at the output of the feature extraction backbone network. This branch is configured to evaluate each local feature vector generated in step S4. (i.e., the first) Each body part's feature representation is processed independently, and a binary classification architecture is used to determine whether the part is in a "visible" state.

[0145] The specific network structure of this branch consists of three cascaded fully connected layers (FC) and a non-linear activation function, with the specific parameter settings as follows:

[0146] First fully connected layer (FC1): Receiver dimension is The local feature vectors are used as input (in this embodiment) This is mapped to a 256-dimensional hidden layer space. The output of this layer is connected to a ReLU activation function to introduce a nonlinear transformation and enhance the discriminative representation of the features.

[0147] The second fully connected layer (FC2) receives the 256-dimensional feature vector output from FC1 and further reduces its dimensionality to a 32-dimensional feature space. The output of this layer is also connected to the ReLU activation function.

[0148] The third fully connected layer (FC3) serves as the classification output layer. It receives the 32-dimensional feature vector output by FC2 and maps it to a 2-dimensional classification prediction vector. The two dimensions of this vector correspond to the two state categories of "occlusion" and "visibility".

[0149] Finally, the output of the third layer is normalized using the Softmax function to obtain the probability distribution of the local feature belonging to each category. In this embodiment, the probability value corresponding to the "visible" category is selected as the final visibility prediction score for the feature slice. This score is used to indicate the reliability of the current part and will be used as a weighting coefficient to guide feature fusion in the subsequent step S8.

[0150] Step S7: Construction of hybrid loss function and joint training.

[0151] In this embodiment, a multi-task hybrid supervision strategy is used to jointly train the model, and the total loss function is constructed. The loss consists of three weighted components: identity recognition loss, triplet metric loss, and visibility estimation loss. The definitions and calculation methods of each component are as follows:

[0152] Identity recognition loss ( ): Utilizing real pedestrian identity labels provided by the dataset For each input image output by the backbone network, the global feature vectors are respectively... and each local feature vector Calculate the cross-entropy loss. This loss aims to optimize the feature extraction network's ability to classify and distinguish pedestrian identities, ensuring that the extracted features possess clear category information.

[0153] Triplet measure loss ( The algorithm constructs triplet samples in the feature space based on identity labels, including anchor samples, positive samples from the same class, and negative samples from different classes. The triplet loss is then calculated. This loss improves the discriminability of features by bringing similar samples closer together in the feature space while simultaneously increasing the distance between dissimilar samples.

[0154] Visibility estimation loss ( ): Utilizing the fine-grained visibility pseudo-labels generated in step S3 As the supervised ground truth, the prediction score of the visibility estimation branch output. Calculate the cross-entropy loss. This loss aims to update the network parameters of the visibility estimation branch, enabling it to accurately determine the visibility of local image features, thereby providing a reliable basis for subsequent feature inpainting.

[0155] Total loss function The calculation formula is as follows:

[0156]

[0157] in, and These are the balance coefficients for the identity recognition loss and the triplet metric loss, respectively. In this embodiment, to give equal importance to the classification ability and metric properties of the features, both are set to 1.

[0158] Step S8: Progressive feature repair.

[0159] In this embodiment, as Figure 5 As shown, in the inference testing phase, to address the feature loss problem caused by occlusion, this embodiment designs a progressive feature repair module. This module is based on the idea of ​​"using unoccluded image library samples to repair occluded query samples," and gradually restores the interfered feature representation in the query image through multiple iterations. The specific implementation process is as follows:

[0160] Step S81: Define the retrieval process and the distance metric used.

[0161] For example, this step aims to establish a retrieval benchmark for the inference phase and define a distance metric suitable for occluded scenarios. To ensure the accuracy of overall features during the initial retrieval and to leverage the occlusion-resistant capabilities of local features in subsequent retrievals, this embodiment employs two retrieval strategies, the specific implementation process of which is as follows:

[0162] (1) Feature extraction and initialization:

[0163] First, retrieve all query images. and stock images Input the trained backbone network and extract the following initial feature information:

[0164] Initial global features: This is used to capture the overall semantic features of queries and gallery images;

[0165] Initial local features: (in ), used to capture fine-grained features of individual body parts in queries and gallery images;

[0166] Visibility Prediction Score: Predicts the visibility score of each local feature using the visibility estimation branch. .

[0167] (2) Initial search (based on standard Euclidean distance):

[0168] In the initial stage before restoration begins, due to the lack of reliable local feature guidance, robust global features are preferentially relied upon for coarse localization. Global features are calculated between the query image and all images in the library. , The standard Euclidean distance between them, and based on this, in the library. In this embodiment, the samples are sorted and the Top-K nearest samples are selected (in this embodiment) ), denoted as Although global features lack detail, they can quickly identify candidate targets with similar appearances in large-scale searches, providing a basic reference for subsequent refined repair.

[0169] (3) Subsequent retrieval (based on visibility-weighted Euclidean distance):

[0170] In subsequent iterative repair processes, to better utilize local features, this embodiment defines a visibility-weighted Euclidean distance based on local features and their corresponding visibility scores. For any two images... and Features and Its visibility-weighted Euclidean distance The calculation formula is as follows:

[0171]

[0172] in, Number of local feature blocks (in this embodiment) ); = Indicates the first Standard Euclidean distance between corresponding local features; This uses a joint visibility weighting. Therefore, when calculating distance, feature differences in the same area are only heavily weighted in the total distance if both the query image and the library image have high visibility (i.e., neither is occluded). Conversely, if either image is occluded in that area (visibility approaches 0), the feature difference weight in that area is significantly reduced or even ignored. This mechanism forces the model to automatically avoid occluded areas in subsequent searches, using only reliable common visible areas for matching, thus significantly improving the search accuracy for occluded targets.

[0173] Step S82: Perform feature iterative repair.

[0174] For example, the total number of repair rounds is set to (In this embodiment) That is, each round repairs one local feature slice), in the first round Round iteration ( In ), perform the following operations:

[0175] (1) Construction of reference features based on visibility weighting: High-quality reference features for repairing query images are constructed using the Top-K image library images retrieved in the previous round. For the query image... A local feature, represented by the visibility score of the image in that region from the library. As a weighting coefficient, for Top-K graph features Perform weighted aggregation to generate reference features. :

[0176]

[0177] By assigning greater weight to high-visibility samples, occlusion noise in the image library samples is effectively suppressed, and pure and effective features are extracted for subsequent restoration.

[0178] (2) Adaptive fusion update of query features: for the query image, the first feature is updated by the second feature update. Each local feature utilizes the visibility score from the previous round. As a weighting coefficient, the query features are weighted and fused for repair. Following a top-down repair strategy, in the first... Wheels are only designed for specific body parts. (In this embodiment, the first) Round of repair A slice, that is Update the formula as follows:

[0179]

[0180] This strategy prioritizes updating upper body features that are usually more visible, and then gradually guides the repair of severely obscured lower body areas.

[0181] (3) Dynamic improvement of visibility scores: After completing the local feature update, in order to ensure that the repaired features play a role in the next round of retrieval, the visibility scores corresponding to each local feature must be updated synchronously. Updated visibility scores It is determined by both the original visibility score and the maximum visibility score of the Top-K image library samples:

[0182]

[0183] This formula indicates that if the source of the library features used for repair is very clear (i.e., This significantly improves the visibility score of that part of the query image. This makes the previously occluded area considered a valid region in the next round of visibility-weighted distance retrieval, thus enabling the retrieval of more accurate image samples from the library.

[0184] Step S83: Output the final retrieval.

[0185] In this embodiment, after After rounds of iterative repair, the occluded region features of the query image have been effectively filled in and enhanced. The final repaired set of local features is then utilized. Based on the features in the image library, the visibility-weighted Euclidean distance is recalculated and sorted to output the final list of pedestrian re-identification retrieval results.

[0186] Step S9: Experimental verification.

[0187] In this embodiment, in order to objectively verify the effectiveness and advancement of the pose-guided and progressive feature repair method proposed in this invention, this embodiment conducted rigorous comparative tests on three mainstream publicly available occluded pedestrian re-identification datasets.

[0188] The experimental setup for this embodiment is as follows: During the training phase, the dimensions of all input images are uniformly adjusted to [size missing]. The optimization strategy employed was a stochastic gradient descent (SGD) optimizer, with a momentum factor set to 0.9 and a weight decay set to [value missing]. The initial learning rate was set to 0.008 and dynamically adjusted using a cosine annealing strategy.

[0189] Step S91: Set evaluation indicators.

[0190] In this embodiment, two core quantitative metrics commonly used in the field of person re-identification are employed to evaluate model performance:

[0191] Rank-1 Accuracy: This metric is derived from the Cumulative Matching Characteristic (CMC) curve. It represents the probability that the image ranked first (Rank-1) in the search results correctly matches the target image. Rank-1 is a key indicator of the algorithm's search accuracy under ideal matching conditions, directly reflecting the system's ability to achieve a perfect match.

[0192] Mean Average Precision (mAP): This metric comprehensively considers the precision and recall of retrieval results. mAP reflects the overall quality of the model across the entire retrieval ranking list. A higher mAP indicates that the model not only correctly retrieves the target but also ranks most of the correct images belonging to that target in the library at the top, reflecting the robustness and completeness of the retrieval.

[0193] Step S92: Compare the experimental results.

[0194] In this embodiment, Table 1 shows the performance comparison results of the method of the present invention with the current mainstream and advanced pedestrian re-identification methods on three benchmark datasets: Occluded-Duke, Occluded-REID, and Partial-REID.

[0195] Table 1 shows the performance comparison of different methods on three datasets ("-" indicates that the original method did not test this item).

[0196]

[0197] For example, Figure 6 The effectiveness of the progressive feature repair module in this invention is intuitively verified. A comparison shows that under the "no repair" and "one-step repair" strategies, severe occlusion easily leads to the model matching the wrong target (red box). However, this invention, through top-down iterative repair, successfully guides feature reconstruction using unoccluded areas, not only correcting the correct target (green box) back to the top position (Rank-1), but also significantly improving the overall accuracy and continuity of the retrieval list, fully demonstrating its accuracy and robustness in complex occlusion scenarios.

[0198] Figure 7 This paper visually demonstrates the significant effect of the skeleton-contour image-guided attention mechanism in this invention. A comparison of the attention heatmaps with and without SSI guidance reveals that in complex scenarios with occlusion by handbags (Example 1) or overlapping multiple people (Example 4), the traditional model's attention easily diverts to background noise or interfering pedestrians. However, by introducing the skeleton-contour image generated by this invention as a mask guide, the model's attention is successfully corrected and focused on the torso and limb regions of the main retrieval target, effectively suppressing occlusion interference. This demonstrates the accuracy and robustness of this mechanism in guiding attention focus.

[0199] Step S93: Result analysis and output.

[0200] In this embodiment, based on experimental data, the performance advantages of the present invention are analyzed as follows:

[0201] On the most challenging Occluded-Duke dataset, the Rank-1 accuracy of this invention reaches 76.7%, which is 12.5% ​​higher than the benchmark method TransReID and 2.9% higher than the state-of-the-art OAFR method. The Occluded-Duke dataset contains a large number of complex "person-to-person" (PTP) scenes, and this significant improvement directly demonstrates the effectiveness of the proposed many-to-one selection algorithm and progressive feature repair module in handling multi-person overlapping interference and fine-grained feature repair.

[0202] On the Partial-REID dataset, this invention also achieved leading performance, with a Rank-1 of 88.3%, which is 4.0% higher than OAFR. On the Occluded-REID dataset, this invention also achieved excellent performance, with a Rank-1 of 88.0%. This indicates that the progressive feature inpainting module of this invention can utilize information from unoccluded regions to reconstruct missing features extremely effectively, making it very suitable for handling the problem of occluded pedestrian re-identification.

[0203] In summary, this invention, by combining pose prior, hybrid supervised training, and progressive feature inpainting, demonstrates excellent performance in various occlusion scenarios, significantly improving the accuracy of occluded person re-identification.

[0204] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0205] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for re-identifying occluded pedestrians based on pose guidance and progressive feature repair, characterized in that: The following steps are included: Step S1: Obtain the original pedestrian re-identification dataset, perform occlusion simulation enhancement processing on the dataset, and construct the enhanced training sample dataset; Step S2: Use the pre-trained pose estimation model to extract human key points from the images in the training sample dataset, and execute a many-to-one selection strategy to automatically filter out the retrieval target in the image. Step S3: Construct a skeleton-contour image and generate visibility pseudo-labels; Step S4: Construct a feature extraction backbone network based on the Vision Transformer architecture; Step S5: Implement attention guidance based on skeleton information; The implementation of step S5 includes: Step S51: Perform max pooling on the generated binarized skeleton-contour image to generate an attention mask sequence that corresponds in number to the tile token sequence; Step S52: Two attention calculation paths are executed in parallel at the first layer of the Transformer encoder. One path performs standard self-attention calculation to capture global context information, and the other path uses the attention mask sequence to remove background interference and focus on target human features. Step S53: Introduce a learnable balancing parameter to weight and fuse the outputs of the two attention calculation paths to obtain the final first-layer output feature sequence; Step S6: Construct a visibility estimation branch network containing fully connected layers; Step S7: Construct a hybrid loss function that includes identity loss, triplet loss, and visibility loss, and perform hybrid supervised joint training; Step S8: Perform inference and feature repair based on a progressive strategy, and output the final pedestrian re-identification retrieval results; The implementation of step S8 includes: Step S81: Obtain the global and local feature representations and visibility scores of the query image and the library images, calculate the standard Euclidean distance based on the global features, and obtain the initial K-nearest neighbor sample set of the query image; Step S82: Perform progressive feature repair iteration. In each iteration, the K nearest neighbor features are aggregated based on the visibility score to obtain a reference feature vector. The reference feature vector is used to perform weighted fusion repair on the local features of the query image. Based on the repaired local features, the visibility weighted Euclidean distance is calculated to re-retrieve and update the K nearest neighbor sample set. The repair continues in the next round until the iteration is completed. Step S83: Calculate and sort the visibility-weighted Euclidean distance between the query image and the image in the library using the local features of the finally repaired query image, and output the final pedestrian re-identification retrieval results.

2. The occluded pedestrian re-identification method based on pose guidance and progressive feature repair according to claim 1, characterized in that: The implementation process of step S1 includes: Step S11: Obtain the original pedestrian re-identification dataset. For each pedestrian image in the dataset, select another pedestrian as the occlusion source image. Step S12: Set a probability threshold to achieve probability-based hybrid occlusion data enhancement; Step S13: Combine the enhanced image with the original unenhanced image to form the final training sample set output.

3. The occluded pedestrian re-identification method based on pose guidance and progressive feature repair according to claim 2, characterized in that: Step S2 includes: Pose key point extraction stage: The input image is processed using a pre-trained pose estimation model to extract the pose information of all detected pedestrians in the image and output the set of key point coordinates for each pedestrian. The many-to-one selection algorithm stage: For each detected pedestrian, calculate the geometric centroid of all its visible key points, and calculate the absolute distance between the horizontal coordinate of the centroid and the horizontal center of the image. Set a horizontal distance threshold, and use a hierarchical strategy to determine the search target based on the distribution of pedestrians detected in the image. Based on the filtering results of the many-to-one selection algorithm, only the pose key point data of the locked search target are retained, and the key point information that is judged to be interfering with pedestrians is filtered out.

4. The occluded pedestrian re-identification method based on pose guidance and progressive feature repair according to claim 3, characterized in that: Step S3, constructing the skeleton-contour image, includes: Using the data-enhanced image as a base, based on the extracted key point coordinates, adjacent key points are connected according to the human pose topology, and colored skeleton lines are drawn directly on the enhanced image to generate an enhanced image with superimposed skeleton information. The pixel-by-pixel difference is calculated by using the superimposed image and the original enhanced image, and for each pixel position in the image, the sum of the absolute differences in the three RGB channels is calculated to obtain the cumulative difference map of a single channel. Set a binarization threshold and perform binarization processing on the cumulative difference map to obtain a preliminary mask; Finally, morphological dilation is used to process the initial mask to generate the final binary skeleton-contour image.

5. The occluded pedestrian re-identification method based on pose guidance and progressive feature repair according to claim 4, characterized in that: Step S3, generating visibility pseudo-labels, includes: The final binarized skeleton-contour image is uniformly divided vertically into three parts. For each horizontal strip region, the sum of the individual pixels of each horizontal strip is calculated. A top-down scanning strategy is used to locate the occlusion boundary, and the first strip where the sum of the first pixel is 0 is used as the cutoff strip. After determining the cutoff band, the visibility pseudo-labels for each of the horizontal bands are generated based on the prior knowledge of the continuity of human body structure.

6. The occluded pedestrian re-identification method based on pose guidance and progressive feature repair according to claim 5, characterized in that: The implementation of step S4 includes: Step S41: Divide the input image into non-overlapping image blocks of fixed size. Through a learnable linear projection layer, map each two-dimensional image block into a one-dimensional embedding vector to form the original tile token sequence. Step S42: The learnable location is embedded and superimposed into the tile token sequence, and then the sequence is input into the Transformer encoder for interaction and extraction of global context information, and the output includes a category token for classification and a tile token sequence containing image content. Step S43: Based on the spatial distribution of the original image, the tile token sequence output by the encoder is spatially rearranged to restore it to a two-dimensional feature map with the original spatial structure. Step S44: Perform multi-granularity feature extraction based on the two-dimensional feature map: extract the global feature vector through global average pooling, and utilize horizontal uniformity. The equal-division pooling strategy extracts multiple local feature vectors.

7. The occluded pedestrian re-identification method based on pose guidance and progressive feature repair according to claim 6, characterized in that: The visibility estimation branch network in step S6 includes: The first fully connected layer receives local feature vectors as input and connects to the ReLU activation function at the output, which is used to introduce nonlinear transformation and enhance the discriminative representation ability of features; The second fully connected layer receives the feature vector output from the first fully connected layer, further reduces its dimensionality and maps it, and the output is also connected to the ReLU activation function. The third fully connected layer serves as the classification output layer. It receives the feature vector output from the second fully connected layer and maps it to a 2-dimensional classification prediction vector. The two dimensions of this vector correspond to the two state categories of "occlusion" and "visibility". The output vector of the third fully connected layer is normalized using the Softmax function to obtain the predicted probability distribution of the local feature belonging to the "occluded" and "visible" categories.

8. The occluded pedestrian re-identification method based on pose guidance and progressive feature repair according to claim 7, characterized in that: The mixed loss function in step S7 includes: Identity recognition loss: Using the real pedestrian identity labels provided by the dataset, calculate the cross-entropy loss for the global feature vector and each local feature vector of each input image output by the backbone network; Triple loss metric: Construct triple samples in the feature space based on identity labels, including anchor samples, positive samples of the same class and negative samples of different classes, and calculate triple loss; Visibility estimation loss: Using the generated fine-grained visibility pseudo-labels as supervision ground truth, the cross-entropy loss is calculated on the prediction score output by the visibility estimation branch.

Images