Pedestrian occlusion re-identification method based on attitude guidance
By using a pose-guided approach and leveraging the symmetry feature repair and grouping aggregation module GAM, the problem of missing features in pedestrian re-identification under occlusion scenarios was solved, improving the robustness and accuracy of the model and achieving better pedestrian re-identification performance under occlusion.
Patent Information
- Application Number
- CN202510797604.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-16
- Publication Date
- 2025-11-18
AI Technical Summary
In complex industrial environments, existing pedestrian re-identification methods struggle to effectively handle feature loss and background noise in occluded areas due to equipment and pipeline obstructions, resulting in a significant drop in model recognition accuracy in occluded scenarios.
A pose-guided approach is adopted, which uses the Symmetric Feature Repair Module (SFRM) and the HR-Net pose estimation model to accurately identify occluded regions. The occluded regions are then completed by mapping the features of the symmetrical keypoint regions. Combined with the Group Aggregation Module (GAM) and the Global Enhancement Module (GEM), the robustness and discriminative ability of the model in occluded scenarios are improved.
It significantly improves the accuracy and robustness of pedestrian re-identification in occluded scenarios. By repairing symmetrical features and compensating for local regional information, it enhances the semantic integrity of occluded regions and the global feature representation ability, thereby improving recognition accuracy and generalization ability.
Smart Images
Figure CN120976841A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of occluded pedestrian re-identification, and particularly relates to a pedestrian occlusion re-identification method based on posture guidance. BACKGROUND
[0002] Pedestrian re-identification, as one of the important research topics in the field of computer vision, aims to associate pedestrian images captured by different physical locations and different cameras, and realize pedestrian identification and retrieval across cameras and scenes, and is widely used in intelligent business, intelligent security and other fields.
[0003] In complex industrial environments such as chemical plants, due to physical occlusions such as equipment and pipelines, the pedestrian images captured by the camera are often occluded by objects or other pedestrians, and the robust pedestrian feature expression cannot be extracted, and can only be in a partially visible state. The existing pedestrian re-identification method relies on attention mechanism to weight the unoccluded area, or uses human key points to divide the area and align the features, but lacks effective reconstruction means for the occluded area features, and the accuracy is severely reduced in such occlusion scenes.
[0004] Under different occlusion modes, the occluded area causes feature missing, background noise enhancement, and incomplete posture information, bringing great challenges to model training and matching. The randomness and uncertainty of the occluded area make it difficult for the model to capture stable semantic features, thereby affecting the modeling accuracy of the global features. SUMMARY
[0005] To solve the above technical problems, the present application provides a pedestrian occlusion re-identification method based on posture guidance, which first uses a symmetric feature repair module SFRM to repair the occluded area based on the structural semantic information of the posture estimation key points, and combines the key point coordinates and confidence output by the HR-Net to accurately determine the occluded area, and through the mapping of the features of the symmetric key point area, effectively compensates for the feature breakage caused by the occlusion, significantly improves the robustness and discriminability of the model in the occlusion scene, and effectively solves the above problems.
[0006] The present application is realized by the following technical solutions:
[0007] A pedestrian occlusion re-identification method based on posture guidance, comprising the steps of:
[0008] Step 1: input the pedestrian image into the posture estimation network, use the posture estimation model to detect the human body key points in the image, obtain multiple human body key point position coordinates and confidence scores, and process the detected multiple human body key points, merge the multiple key points of the head, and finally determine a certain number of key point regions;
[0009] Step 2: Determine whether the key point is occluded according to the confidence threshold, and mark the key point area below the preset threshold as an occluded area;
[0010] Step 3: Design a symmetric feature repair module SFRM. For the occluded key point area, determine its corresponding unoccluded symmetric key point area according to the symmetry of the human body structure, calculate the coordinate mapping relationship between the occluded area and the symmetric area, map the features of the unoccluded symmetric area to the occluded area, and realize feature repair;
[0011] Step 4: Construct a feature restoration loss function to optimize the repaired features; the loss function is composed of intra-class feature similarity loss and inter-class feature separability loss, to minimize the distance between occluded and non-occluded features of the same class, and maximize the distance between different classes;
[0012] Step 5: Aggregate the repaired key point areas again and divide them into 7 semantic regions: head region, left upper body region, right upper body region, left thigh region, right thigh region, left lower leg region, and right lower leg region;
[0013] Step 6: Design a grouping aggregation module GAM. The 7 semantic regions divided in step 5 are used as graph nodes to construct an inter-regional directed graph relationship matrix. The inter-regional directed graph relationship matrix is divided into an upper body group and a lower body group. The nodes are connected by a preset human body structure, and the left upper body and right upper body nodes are connected, and a directed edge is established between the upper body region and the lower body region. The local region features are aggregated and compensated to generate enhanced feature representations of the occluded regions, and 7 enhanced local region features are output. Then, global average pooling is performed on each of them, and the enhanced local features are obtained after concatenation;
[0014] Step 7: Design a global enhancement module GEM. The backbone network feature map F and the key point fusion feature map F' are input into the global enhancement module. The maximum pooling and average pooling operations are performed in parallel to extract multi-scale context feature information of the key point fusion feature map. The obtained features are fused with the backbone network features through a residual manner, and an enhanced global feature vector is output.
[0015] Step 8: Concatenate and fuse the enhanced local features obtained in step 6 and the global feature vector obtained in step 7 to obtain the final pedestrian feature representation vector.
[0016] Further, the pose estimation model in step 1 adopts an HRNet pose estimation model pre-trained based on a COCO dataset to detect 17 key points of the human body, which are: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle. Since the head region key points are relatively dense, change little, and are easy to lose in the occlusion scene, the five key points of the nose, left eye, right eye, left ear, and right ear are merged as one head region key point. The merging method is to perform weighted average on the coordinates of the five key points, and finally determine 13 key point regions. The weighted average formula is:
[0017]
[0018] In the above formula, K head represents the merged head key point; K nose , K left_eye , K right_eye , K left_ear , and K right_ear represent the five key points of the nose, left eye, right eye, left ear, and right ear, respectively.
[0019] Further, the occlusion region in step 2 needs to be screened out. In order to screen out the occluded key points, the confidence S k of the key point is calculated. k When S k is lower than a set threshold θ, the key point is considered to be occluded, as shown in the formula:
[0020]
[0021] In the above formula, M k represents the kth key point heat map, S k represents the confidence of the key point k, (x k , y k ) represents the coordinates of the key point k, and θ is the threshold value.
[0022] Further, in the symmetry feature repair module SFRM in step 3, the symmetry mapping repair formula is:
[0023] F occ = F noc (x+Δ x , y+Δ y )
[0024] In the above formula, F occ represents the unoccluded position coordinates, (x, y) is the occlusion region coordinates, Δ x and Δ y represent the movement distances of the key points in the x-axis and y-axis directions, respectively, and are positive values.
[0025] Furthermore, step 4, which involves constructing the feature restoration loss function, specifically includes the following sub-steps:
[0026] Step 4.1: Obtain the repaired features of each occluded keypoint region and match them with the unoccluded features of the same pedestrian in other images to calculate the intra-class similarity distance. To measure the degree of consistency between the restored features and the original true features;
[0027]
[0028] In the above formula, The features of the i-th occluded region are represented. The features of the k-th unoccluded image of the same pedestrian in the same key point region are represented by K, where K is the number of samples of unoccluded features of the same pedestrian.
[0029] Step 4.2: Compare the repaired features with the unoccluded features of the key point regions corresponding to different pedestrians, and calculate the inter-class dissimilarity distance. To reflect its separability from other category features;
[0030]
[0031] In the above formula, F p This represents the unoccluded features of different pedestrians in the same keypoint area, where P is the number of samples of unoccluded features of other pedestrians;
[0032] Step 4.3: Based on the two distances mentioned above, design the feature restoration loss function L. SFRM To simultaneously minimize intra-class distance and maximize inter-class distance, thereby improving the discriminative power and cluster compactness of the repaired features; the loss function is defined as follows:
[0033]
[0034] Where M is the number of all occluded keypoint regions, and α is a hyperparameter used to ensure sufficient discriminative power between intra-class and inter-class features; and These are respectively represented as the distance between the feature of each occluded region and the feature of the unoccluded region of the same pedestrian, and the distance between the feature of the unoccluded region of other pedestrians;
[0035] The goal of this loss function is to make the repaired features as close as possible to the true features of their respective categories, while keeping them away from features of other categories, thereby improving the recognition accuracy under occlusion conditions.
[0036] Furthermore, the seven regions described in step 5 are respectively denoted as f1, f2, ..., f7.
[0037] Furthermore, the directed graph relationship matrix between regions mentioned in step 6 includes the upper body group A. up and lower body group A down The upper body group includes the head, left upper body, and right upper body; the lower body group includes the left thigh, right thigh, left calf, and right calf; the specific form is as follows:
[0038]
[0039] Among them, w ij This represents the association weight between i and j (where i,j∈[1,2,...,7]); when calculating regional features, if a region's feature information is incomplete due to occlusion, the features of its associated regions can be used for compensation.
[0040] Furthermore, the seven enhanced local region features output in step 6 are represented as f1′, f2′, ..., f7′, respectively. The formulas for aggregating the features of a certain region of the upper and lower body into the formulas for calculating the features of the upper and lower body are as follows:
[0041]
[0042]
[0043] Among them, f α ′ represents a new feature of the upper body region aggregated in region α, f α "" indicates that region α aggregates new features of the lower body region, where α represents a certain region of the body and β is the aggregation coefficient. and They represent f respectively α New features are obtained by aggregating features from the upper and lower body regions.
[0044] Furthermore, step 7, which involves extracting multi-scale contextual feature information from the keypoint fusion feature map, specifically involves simultaneously performing global average pooling and global max pooling on the keypoint fusion feature map F′, and then summing the resulting features to obtain the pooled feature F. p Then, the backbone network feature map F is subjected to global average pooling and residual concatenated with the pooled features to obtain the enhanced global feature vector F. g The formula for calculating pooling features is:
[0045] F p =GAP(F′)+GMP(F′)
[0046] In the above formula, F pF′ represents the pooling feature, which is the feature map obtained by element-wise multiplying the feature map F output by ResNet-50 through the HRNet model with the heat map output by the HRNet model. GAP represents global max pooling and GEM represents full average pooling.
[0047] The enhanced global feature vector formula is:
[0048] F g =[GAP(F),F p ]
[0049] Among them, F g denoted as the global feature vector obtained by the global augmentation GEM module, and F represents the feature map output by ResNet-50.
[0050] Furthermore, the specific formula for splicing and fusion in step 8 is as follows:
[0051] F FAM =[GAP(f1′),GAP(f2′),...,GAP(f7′)]
[0052] F final =Concat(F g ,F FAM )
[0053] Among them, F FAM F represents the enhanced feature obtained by concatenating the enhanced local features individually. final This represents the final pedestrian features; Concat means concatenating multiple feature tensors along the channel dimension to form the final complete pedestrian feature vector.
[0054] Beneficial effects
[0055] The present invention proposes a pose-guided pedestrian occlusion re-identification method, which has the following advantages compared with the prior art:
[0056] (1) This invention utilizes the natural symmetry of the human body structure to design a Symmetry Feature Repair Module (SFRM). Based on the structural semantic information of pose estimation keypoints, it performs explicit feature repair on occluded regions. Combined with the keypoint coordinates and confidence scores output by the HR-Net pose estimation model, it accurately identifies occluded regions and completes the repair by mapping the features of symmetrical keypoint regions. This module not only possesses good interpretability but also, by constructing a restoration loss function, ensures that the repaired features are compact within classes and discriminative between classes, effectively compensating for feature fragmentation caused by occlusion. This significantly improves the robustness and discriminative ability of the model in occluded scenarios.
[0057] (2) This invention first divides 13 key point regions into 7 semantic regions and constructs graph structure relationships between regions. A directed graph adjacency matrix is used to model the interaction relationships between regions of the upper and lower body. The Group Aggregation Module (GAM) is then used to perform fine-grained modeling of local regions in a structured manner, fully exploring the contextual relationships between key points of the human body to achieve semantic enhancement of occluded regions. During the aggregation process, the GAM module dynamically fuses information from associated regions using learnable weights, strengthening the semantic integrity and feature representation capabilities of occluded regions. Especially under various uncertain occlusion modes, GAM can adaptively utilize non-occluded regions to compensate for occluded region information, thereby effectively improving the stability and robustness of local features.
[0058] (3) The Global Enhancement Module (GEM) proposed in this invention integrates a dual-path pooling mechanism with pose heatmap guidance to construct a global feature representation that balances discriminativity and robustness. The GEM module captures contextual semantics and salient region information in the image by performing global average pooling and max pooling in parallel; simultaneously, it introduces pose estimation guidance features as external attention cues and performs residual connections with the backbone network feature map, effectively suppressing background interference and local noise. Compared to traditional single pooling strategies or manual pruning, GEM achieves the fusion of multi-scale information and guided enhancement, ensuring that the final global representation possesses both structural integrity and retains key discriminative details. The introduction of this module significantly improves the model's generalization ability and recognition accuracy under multi-pose and multi-background occlusion conditions. Attached Figure Description
[0059] Figure 1 This is a schematic diagram of the overall process of the present invention.
[0060] Figure 2 This is a schematic diagram of the overall architecture of the present invention.
[0061] Figure 3 This is a schematic diagram of the SFRM module structure in this invention.
[0062] Figure 4 This is a schematic diagram of the GAM module structure in this invention.
[0063] Figure 5 This is a schematic diagram of the GEM module structure in this invention.
[0064] Figure 6 This is a schematic diagram of the occlusion repair in this invention.
[0065] Figure 7 This is a comparison chart of the aggregation coefficient values in the GAM module of this invention. Detailed Implementation
[0066] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. The described embodiments are merely some embodiments of the present invention, and not all embodiments. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the design concept of the present invention should fall within the protection scope of the present invention.
[0067] Example 1:
[0068] like Figure 1 As shown, a pose-guided pedestrian occlusion re-identification method includes the following steps:
[0069] Step 1: Input the pedestrian image into the pose estimation network. The overall architecture of the pose estimation network is as follows: Figure 2 As shown, a pose estimation model is used to detect key points in the human body in the image, and the position coordinates and confidence scores of 17 human key points are obtained. The 17 detected human key points are then processed, and the 5 key points of the head are merged into 1, and finally 13 key point regions are determined.
[0070] The pose estimation model described uses an HRNet pose estimation model pre-trained on the COCO dataset to detect 17 key points of the human body, namely: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle.
[0071] Considering that keypoints in the head region are relatively dense, do not change much, and are easily lost in occluded scenes, five keypoints—the nose, left eye, right eye, left ear, and right ear—were merged into one head region keypoint. The merging method involved weighted averaging the coordinates of these five keypoints, ultimately determining 13 keypoint regions. The weighted average formula is:
[0072]
[0073] Among them, K head K nose K left_eye K right_eye K left_ear and K right_ear These represent five key points: the nose, left eye, right eye, left ear, and right ear.
[0074] Step 2: Determine whether the 13 keypoints are occluded based on the confidence threshold. The confidence threshold is 0.3. If the value is lower than this threshold, the keypoint is considered to be occluded. The confidence score is the keypoint confidence score output by the HRNet model, and the value range is 0 to 1.
[0075] Occluded areas need to be filtered out. To filter out occluded keypoints, the confidence S of the keypoints is calculated. k When S k When the value is below a set threshold θ, the key point is considered an occluded area, as shown in the formula:
[0076]
[0077] In the above formula, M k S represents the heatmap of the k-th key point. k This represents the confidence level of key point k, (x k ,y k ) represents the coordinates of key point k, and θ is the threshold.
[0078] Step 3: Design the Symmetry Feature Repair Module (SFRM). The structure of the SFRM module is as follows: Figure 3 As shown, for occluded keypoint regions, based on the symmetry of the human body structure, the corresponding unoccluded symmetrical keypoint regions are determined. The coordinate mapping relationship between the occluded region and its symmetrical region is calculated, and the features of the unoccluded symmetrical region are mapped to the occluded region to achieve feature repair. In the symmetric feature repair module SFRM, the symmetric mapping repair formula is:
[0079] F occ =F noc (x+Δ x ,y+Δ y )
[0080] In the above formula, F occ (x, y) represents the coordinates of the unobstructed position, and (y, y) represents the coordinates of the obstructed area. Δ x and Δ y These represent the distances the keypoints moved along the x-axis and y-axis, respectively, both of which are positive values.
[0081] To intuitively explain the repair principle of this method, such as Figure 6 Let's take the left knee area being obscured as an example for further explanation. The coordinates of the key points on the left knee are (x... l_knee ,y l_knee The boundary is set as follows: Left boundary B left Defined as the origin, it is set to 0. The vertical direction is used as the x-axis, and the horizontal direction is used as the y-axis. Upper boundary B up Defined as the average of the coordinates of key points on the left hip and left knee; the lower and right boundaries are calculated similarly. The formula for the upper boundary is as follows:
[0082] B up =(y l_hip +y l_knee ) / 2
[0083] Among them, yl_hip y l_knee These represent the values on the y-axis of the key points of the left hip and left knee, respectively.
[0084] Step 4: Construct a feature restoration loss function to optimize the repaired features; the loss function consists of intra-class feature similarity loss and inter-class feature separation loss, to minimize the distance between occluded and unoccluded features of the same class and maximize the distance with different classes; the construction of the feature restoration loss function specifically includes the following sub-steps:
[0085] Step 4.1: Obtain the repaired features of each occluded keypoint region and match them with the unoccluded features of the same pedestrian in other images to calculate the intra-class similarity distance. To measure the degree of consistency between the restored features and the original true features;
[0086]
[0087] In the above formula, The features of the i-th occluded region are represented. The feature of the k-th unoccluded image of the same pedestrian in the same key point region is represented by K, where K is the number of samples of the unoccluded features of the same pedestrian.
[0088] Step 4.2: Compare the repaired features with the unoccluded features of the key point regions corresponding to different pedestrians, and calculate the inter-class dissimilarity distance. To reflect its separability from other category features;
[0089]
[0090] In the above formula, F p P represents the unoccluded features of different pedestrians in the same key point area, where P is the number of samples of unoccluded features of other pedestrians.
[0091] Step 4.3: Based on the two distances mentioned above, design the feature restoration loss function L. SFRM To simultaneously minimize intra-class distance and maximize inter-class distance, thereby improving the discriminative power and cluster compactness of the repaired features; the loss function is defined as follows:
[0092]
[0093] Where M is the number of all occluded keypoint regions, and α is a hyperparameter used to ensure sufficient discriminative power between intra-class and inter-class features; and These represent the distances between the features of each occluded region and the features of the unoccluded regions of the same pedestrian, and the distances between the features of the unoccluded regions of other pedestrians, respectively.
[0094] The goal of this loss function is to make the repaired features as close as possible to the true features of their respective categories, while keeping them away from features of other categories, thereby improving the recognition accuracy under occlusion conditions.
[0095] Step 5: Perform secondary aggregation on the 13 repaired key point regions and divide them into 7 semantic regions, namely the head region, the upper left body region, the upper right body region, the left thigh region, the right thigh region, the left calf region, and the right calf region; the seven regions are represented as f1, f2, ..., f7 respectively.
[0096] Step 6: Design the Group Aggregation Module (GAM). The structure of the Group Aggregation Module (GAM) is as follows: Figure 4 As shown, the seven semantic regions divided in step 5 above are used as graph nodes to construct a directed graph relation matrix between regions. The directed graph relation matrix between regions is divided into an upper body group and a lower body group. The nodes are connected through a preset human body structure. The upper left and upper right half of the nodes are connected, and a directed edge is established between the upper body region and the lower body region. Information aggregation compensation is performed on the local region features to generate enhanced feature representations of the occluded region. Seven enhanced local region features are output, and then they are all subjected to global average pooling and concatenated to obtain the enhanced local features.
[0097] The directed graph relationship matrix between regions includes upper body group A. up and lower body group A down The upper body group includes the head, left upper body, and right upper body; the lower body group includes the left thigh, right thigh, left calf, and right calf; the specific form is as follows:
[0098]
[0099] Among them, w ij This represents the association weight between i and j (where i,j∈[1,2,...,7]); when calculating regional features, if a region's feature information is incomplete due to occlusion, the features of its associated regions can be used for compensation.
[0100] The seven enhanced local region features output are represented as f1′, f2′, ..., f7′, respectively. The formulas for aggregating the features of a certain region of the upper and lower body into the formulas for the upper and lower body features are as follows:
[0101]
[0102] Among them, f α ′ represents a new feature of the upper body region aggregated in region α, f α "" indicates that region α aggregates new features of the lower body region, where α represents a certain region of the body and β is the aggregation coefficient. and They represent f respectively α New features are obtained by aggregating features from the upper and lower body regions.
[0103] Step 7: Design the Global Enhancement Module (GEM). The structure of the Global Enhancement Module (GEM) is as follows: Figure 5 As shown, the feature map F output by the backbone network ResNet-50 and the keypoint fusion feature map F′ are input into the global enhancement module. Multi-scale contextual feature information of the keypoint fusion feature map is extracted through parallel max pooling and average pooling operations. The obtained features are fused with the backbone network features through residual method to output the enhanced global feature vector.
[0104] The extraction of multi-scale contextual feature information from the keypoint fusion feature map specifically involves simultaneously performing global average pooling and global max pooling on the keypoint fusion feature map F′, and then summing the resulting features to obtain the pooled feature F. p Then, the backbone network feature map F is subjected to global average pooling and residual concatenated with the pooled features to obtain the enhanced global feature vector F. g The formula for calculating pooling features is:
[0105] F p =GAP(F′)+GMP(F′)
[0106] In the above formula, F p F′ represents the pooling feature, which is the feature map obtained by element-wise multiplying the feature map F output by ResNet-50 through the HRNet model with the heat map output by the HRNet model. GAP represents global max pooling and GEM represents full average pooling.
[0107] The enhanced global feature vector formula is:
[0108] F g =[GAP(F),F p ]
[0109] Among them, F g denoted as the global feature vector obtained by the global augmentation GEM module, and F represents the feature map output by ResNet-50.
[0110] Step 8: Concatenate and fuse the enhanced local features obtained in Step 6 with the global feature vector obtained in Step 7 to obtain the final pedestrian feature representation vector.
[0111] The specific formula for splicing and fusion is as follows:
[0112] F FAM =[GAP(f1′),GAP(f2′),...,GAP(f7′)]
[0113] F final =Concat(F g ,F FAM )
[0114] Among them, F FAM F represents the enhanced feature obtained by concatenating the enhanced local features individually. final This represents the final pedestrian features; Concat means concatenating multiple feature tensors along the channel dimension to form the final complete pedestrian feature vector.
[0115] Experimental verification:
[0116] In this invention, in order to verify the effectiveness and module contribution of the proposed pose-guided occluded pedestrian re-identification method, multiple ablation experiments were designed to compare the impact of different module combinations on recognition performance.
[0117] All experiments were conducted on the PyTorch platform, using an NVIDIA RTX 4090D GPU for training. The batch size was 64, the optimizer was Adam, the initial learning rate was set to 0.0003, and the weight decay was set to 1e. -4 The training run consisted of 120 rounds. Evaluation metrics used in pedestrian re-identification tasks included Rank-1 and mAP. Rank-1 indicates whether the model can find the correct match in the first position among all pedestrians in the database, while mAP measures the model's average accuracy across different queries. Together, these metrics reflect the model's performance in real-world retrieval tasks.
[0118] To verify the effectiveness of each module, the following experimental combinations were constructed: Baseline: The backbone is ResNet-50, with only the pooling layers removed and the stride of the fourth residual block adjusted. SFRM: A symmetric feature repair module was added. GAM: A semantic grouping and aggregation module was added. GEM: A global enhancement module was added. The experimental results are shown in Table 1:
[0119]
[0120]
[0121] As shown in Table 1, the introduction of the SFRM module improved Rank-1 by 16.2% and mAP by 16.0%, verifying the importance of the symmetric feature repair mechanism for completing information in occluded regions. Further addition of the GAM module increased Rank-1 to 66.9%, indicating that local semantic aggregation enhanced region representation. The GEM module alone also brought significant improvements, demonstrating that multi-scale global contextual information significantly aids in global understanding under occluded conditions. When all modules were used together, the model achieved a Rank-1 of 68.1% and an mAP of 58.4% on the Market-1501 dataset, representing improvements of 20.5% and 19.2% respectively compared to the baseline model, validating the effectiveness of each module and the superiority of the overall architecture design. Therefore, the method proposed in this invention exhibits higher matching accuracy and stronger robustness in occluded scenarios, effectively improving the practical performance of pedestrian re-identification systems.
[0122] In the GAM module, due to the aggregation coefficient β, different values represent different aggregation weights. When β is 1, only the half-body region features corresponding to that region are aggregated. To aggregate upper and lower body region features simultaneously, β needs to take a value within (0,1). Therefore, to find the most suitable parameter, experiments were conducted on the Occluded-Duke dataset to determine the value of this coefficient. The results are as follows: Figure 7 As shown.
[0123] Experimental results show that the aggregation coefficient β has a significant impact on model performance. When β is small, the feature aggregation range is too dispersed; the upper body region may incorporate too much lower body information, while the lower body region may incorporate too much upper body information, leading to reduced discriminative power of feature representation and affecting the final recognition result. Experiments show that when β = 0.7, both Rank-1 and mAP metrics reach their optimal values, indicating that this value strikes a balance between local feature integrity and global information fusion, improving the model's recognition ability. Therefore, β plays a role in controlling the feature aggregation ratio of different regions in the GAM module, and setting its value appropriately can effectively improve model performance.
[0124] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered by the present invention.
Claims
1. A pedestrian occlusion re-identification method based on pose guidance, characterized in that: Including the following steps: Step 1: Input the pedestrian image into the pose estimation network, use the pose estimation model to detect key points of the human body in the image, obtain the location coordinates and confidence scores of multiple human key points; process the detected multiple human key points, merge multiple key points of the head, and finally determine a certain number of key point regions. Step 2: Determine whether key points are occluded based on the confidence threshold, and mark key point areas below the preset threshold as occluded areas; Step 3: Design the Symmetry Feature Repair Module (SFRM). For occluded key point regions, based on the symmetry of the human body structure, determine the corresponding unoccluded symmetrical key point regions, calculate the coordinate mapping relationship between the occluded region and its symmetrical region, and map the features of the unoccluded symmetrical region to the occluded region to achieve feature repair. Step 4: Construct a feature restoration loss function to optimize the repaired features; the loss function consists of intra-class feature similarity loss and inter-class feature separation loss, in order to minimize the distance between occluded and unoccluded features of the same class and maximize the distance between different classes; Step 5: The repaired key point regions are aggregated again and divided into 7 semantic regions, namely the head region, the upper left body region, the upper right body region, the left thigh region, the right thigh region, the left calf region, and the right calf region. Step 6: Design the Grouping and Aggregation Module (GAM). The seven semantic regions defined in Step 5 are used as graph nodes to construct a directed graph relation matrix between regions. This matrix is divided into an upper body group and a lower body group. Nodes are connected through a predefined human body structure, connecting the upper left and upper right body nodes and establishing directed edges between the upper body and lower body regions. Information aggregation and compensation are performed on local region features to generate enhanced feature representations of occluded areas. Seven enhanced local region features are output, and then globally average pooled and concatenated to obtain the enhanced local features. Step 7: Design the global enhancement module GEM. Input the backbone network feature map F and the key point fusion feature map F′ into the global enhancement module. Extract multi-scale contextual feature information of the key point fusion feature map through parallel max pooling and average pooling operations. The obtained features are fused with the backbone network features through residual method to output the enhanced global feature vector. Step 8: Concatenate and fuse the enhanced local features obtained in Step 6 with the global feature vector obtained in Step 7 to obtain the final pedestrian feature representation vector.
2. The pedestrian occlusion re-identification method based on posture guidance according to claim 1, characterized in that: The pose estimation model described in step 1 uses an HRNet pose estimation model pre-trained on the COCO dataset, detecting 17 key points of the human body: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle. Since the key points in the head region are relatively dense, do not change much, and are easily lost in occluded scenes, the five key points (nose, left eye, right eye, left ear, and right ear) are merged into one head region key point. The merging method is to perform a weighted average of the coordinates of these five key points, ultimately determining 13 key point regions. The weighted average formula is: In the above formula, K head Indicates the key points of the merged head; K nose K left_eye K right_eye K left_ear and K right_ear These represent five key points: the nose, left eye, right eye, left ear, and right ear.
3. The pedestrian occlusion re-identification method based on posture guidance according to claim 1, characterized in that: The occluded areas mentioned in step 2 need to be screened out. In order to screen out the occluded key points, the confidence S of the key points is calculated. k When S k When the value is below a set threshold θ, the key point is considered occluded, as shown in the formula: In the above formula, M k S represents the heatmap of the k-th key point. k This represents the confidence level of key point k, (x k ,y k ) represents the coordinates of key point k, and θ is the threshold.
4. The pedestrian occlusion re-identification method based on posture guidance according to claim 1, characterized in that: In the symmetry feature repair module SFRM described in step 3, the symmetry mapping repair formula is: F occ =F noc (x+Δ x ,y+Δ y ) In the above formula, F occ (x, y) represents the coordinates of the unobstructed position, and (y, y) represents the coordinates of the obstructed area. Δ x and Δ y These represent the distances the keypoints moved along the x-axis and y-axis, respectively, both of which are positive values.
5. The pedestrian occlusion re-identification method based on posture guidance according to claim 1, characterized in that: Step 4, which involves constructing the feature restoration loss function, specifically includes the following sub-steps: Step 4.1: Obtain the repaired features of each occluded keypoint region and match them with the unoccluded features of the same pedestrian in other images to calculate the intra-class similarity distance. To measure the degree of consistency between the restored features and the original true features; In the above formula, The features of the i-th occluded region are represented. The features of the k-th unoccluded image of the same pedestrian in the same key point region are represented by K, where K is the number of samples of unoccluded features of the same pedestrian. Step 4.2: Compare the repaired features with the unoccluded features of the key point regions corresponding to different pedestrians, and calculate the inter-class dissimilarity distance. To reflect its separability from other category features; In the above formula, F p This represents the unoccluded features of different pedestrians in the same keypoint area, where P is the number of samples of unoccluded features of other pedestrians; Step 4.3: Based on the two distances mentioned above, design the feature restoration loss function L. SFRM To simultaneously minimize intra-class distance and maximize inter-class distance, thereby improving the discriminative power and cluster compactness of the repaired features; the loss function is defined as follows: Where M is the number of all occluded keypoint regions, and α is a hyperparameter used to ensure sufficient discriminative power between intra-class and inter-class features; and These are respectively represented as the distance between the feature of each occluded region and the feature of the unoccluded region of the same pedestrian, and the distance between the feature of the unoccluded region of other pedestrians; The goal of this loss function is to make the repaired features as close as possible to the true features of their respective categories, while keeping them away from features of other categories, thereby improving the recognition accuracy under occlusion conditions.
6. The pedestrian occlusion re-identification method based on posture guidance according to claim 1, characterized in that: The seven regions defined in step 5 are represented as f1, f2, ..., f7.
7. The pedestrian occlusion re-identification method based on posture guidance according to claim 1, characterized in that: The directed graph relationship matrix between regions mentioned in step 6 includes the upper body group A. up and lower body group A down The upper body group includes the head, left upper body, and right upper body; the lower body group includes the left thigh, right thigh, left calf, and right calf; the specific form is as follows: Among them, w ij This represents the association weight between i and j (where i,j∈[1,2,...,7]); when calculating regional features, if a region's feature information is incomplete due to occlusion, the features of its associated regions can be used for compensation.
8. The pedestrian occlusion re-identification method based on posture guidance according to claim 1, characterized in that: The seven enhanced local region features output in step 6 are represented as f1′, f2′, ..., f7′, respectively. The formulas for aggregating the features of a certain region of the upper and lower body into the formulas for calculating the features of the upper and lower body are as follows: Among them, f α ′ represents a new feature of the upper body region aggregated in region α, f α "" indicates that region α aggregates new features of the lower body region, where α represents a certain region of the body and β is the aggregation coefficient. and They represent f respectively α New features are obtained by aggregating features from the upper and lower body regions.
9. A pedestrian occlusion re-identification method based on pose guidance according to claim 1, characterized in that: Step 7, extracting multi-scale contextual feature information from the keypoint fusion feature map, specifically involves simultaneously performing global average pooling and global max pooling on the keypoint fusion feature map F′, and then summing the resulting features to obtain the pooled feature F. p Then, the backbone network feature map F is subjected to global average pooling and residual concatenated with the pooled features to obtain the enhanced global feature vector F. g The formula for calculating pooling features is: F p =GAP(F′)+GMP(F′) In the above formula, F p F′ represents the pooling feature, which is the feature map obtained by element-wise multiplying the feature map F output by ResNet-50 through the HRNet model with the heat map output by the HRNet model. GAP represents global max pooling and GEM represents full average pooling. The enhanced global feature vector formula is: F g =[GAP(F),F p ] Among them, F g denoted as the global feature vector obtained by the global augmentation GEM module, and F represents the feature map output by ResNet-50.
10. A pedestrian occlusion re-identification method based on pose guidance according to claim 1, characterized in that: The specific formula for splicing and fusion in step 8 is as follows: F FAM =[GAP(f1′),GAP(f2′),...,GAP(f7′)] F final =Concat(F g ,F FAM ) Among them, F FAM F represents the enhanced feature obtained by concatenating the enhanced local features individually. final This represents the final pedestrian features; Concat means concatenating multiple feature tensors along the channel dimension to form the final complete pedestrian feature vector.