Human body posture estimation method based on improved YOLO

Abstract

The invention discloses a human body posture estimation method based on improved YOLO11-Pose, and belongs to the technical field of 2D human body posture estimation.The method comprises the steps that a target image data set and a corresponding real label are obtained and preprocessed, and a subset only containing a human body target and a corresponding label are obtained; the method comprises the following steps: carrying out improvement on the basis of a YOLO11-Pose model, and establishing a YOLO11-CBP-Pose model; establishing a software and hardware environment supporting model training; and inputting the preprocessed data set into a YOLO11-CBP-Pose model for training, and after the training is completed, obtaining a human body target detection result and a corresponding key point detection result through model reasoning. The method has the advantages that the problem of low model precision caused by missing detection and mutual shielding of human body key points under the multi-target crowd condition is effectively solved, and the model precision and the calculation complexity are well balanced.

Description

Technical Field

[0001] This invention belongs to the field of 2D human pose estimation, and specifically relates to a human pose estimation method based on an improved YOLO11-Pose. Background Technology

[0002] With the rapid development of deep learning technology, human pose estimation has become one of the core research directions in the field of computer vision. Its core objective is to accurately locate key points of the human body (such as the head, shoulders, wrists, and ankles) from images or videos, and to construct a human skeleton based on these key points to infer pose, providing support for downstream tasks. Currently, this technology has been widely used in fields such as intelligent monitoring, motion analysis, and human-computer interaction, and has significant practical value.

[0003] The YOLO series of algorithms, with their advantages of fast detection speed and balanced accuracy, has become the mainstream solution for human pose estimation tasks. Among them, YOLO11-Pose, as the latest version, has improved feature extraction efficiency and pose estimation performance to some extent. However, in real-world complex scenarios, YOLO11-Pose still has significant shortcomings: First, it is easily affected by irrelevant information in complex backgrounds, leading to insufficient key point feature extraction; second, when facing small human targets or multi-target crowds, it is prone to problems such as missed key point detection and occlusion interference, resulting in a significant decrease in pose estimation accuracy; third, as the network depth increases, it is difficult to balance detection accuracy and computational complexity. The large number of model parameters and high computational cost not only lead to deployment difficulties but also reduce inference real-time performance, limiting its application in resource-constrained scenarios such as embedded devices.

[0004] Therefore, how to optimize YOLO11-Pose to address the aforementioned shortcomings, while improving the accuracy of pose estimation in complex scenarios and taking into account the model's lightweight nature and real-time performance, has become a key issue that current human pose estimation technology urgently needs to address. Summary of the Invention

[0005] The core technical problem that this invention aims to solve is the shortcomings of the existing YOLO11-Pose model in terms of adaptability to complex scenarios, accuracy and balance of computational complexity.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: acquiring the target image dataset and its corresponding real labels and preprocessing it to obtain a subset containing only human targets and their corresponding labels, the subset consisting of a training set and a validation set; Based on the YOLO11-Pose model, an improved YOLO11-CBP-Pose model was established. The preprocessed dataset is input into the YOLO11-CBP-Pose model for training. During each epoch of training, the model performance is validated in real time using a validation set. After training, the human target detection results and corresponding key point detection results are obtained through model inference.

[0007] The YOLO11-CBP-Pose model is based on the YOLO11n-Pose model architecture. First, it introduces a Convolutional Additive Self-attention (CAS) hybrid structure to replace the Bottleneck part of the C3k2 module in the backbone and neck network of the original YOLO11-Pose model. Secondly, the path aggregation network (PANET) used in the neck network of the original YOLO11-Pose model is replaced by a bidirectional multi-scale information fusion network (BMIFNet) proposed in this invention, which optimizes the multi-scale feature fusion method. Finally, the original YOLO11-Pose model's Pose detection head was replaced with a parameter-shared convolutional detection head (PSCD) proposed in this invention.

[0008] In some embodiments, the dataset is the COCO2017 public dataset released by Microsoft, which is currently the most commonly used dataset for detecting human keypoints in multiple people. The training set contains 118,282 images, and the validation set contains 5,000 images. After preprocessing, a subset COCO-POSE containing only human targets is obtained, in which the training set contains 56,599 images, the validation set contains 2,346 images, and corresponding ground truth labels are provided.

[0009] In some embodiments, the convolutional additive self-attention hybrid structure consists of three parts: an integration module, a CATM module, and an MLP module, and uses skip connections; The Integration module includes multi-layer convolution operations, as well as batch normalization and activation operations, and uses skip connections. The CATM module is a token hybrid mechanism that combines convolutional operations and additive interactions. It is a self-attention structure based on latent channel convolution and spatial convolution. Unlike the multiplicative interactions of traditional self-attention mechanisms, this module extracts local features through convolution and models long-distance dependencies through additive operations, and eliminates complex operations such as matrix multiplication and Softmax. The MLP module, or Multilayer Perceptron, also includes multilayer convolution operations, batch normalization operations, and activation operations. It further processes the features output by the CATM module, extracts and integrates features through nonlinear transformations, and converts them into feature representations more suitable for the final task. At the same time, the multilayer structure of the MLP can learn more complex and abstract patterns in the data, enhancing the expressive power of the model.

[0010] In some embodiments, the bidirectional multi-scale information fusion network, compared with the path aggregation network in the original model, achieves richer feature interaction and information fusion, enhances the network's multi-scale representation capability, integrates high-level semantic cues, and preserves low-level spatial details. It adopts a weighted feature fusion method to learn the importance of different input features during feature fusion and performs differentiated fusion of different features to improve the network's fusion performance.

[0011] In some embodiments, the parameter-shared convolutional detection head employs a shared depth-separable convolutional module on detection branches at different scales, and each detection branch undergoes independent batch normalization and activation operations. The depthwise separable convolution includes depthwise convolution and pointwise convolution. Depthwise convolution is used to extract spatial features, while pointwise convolution is used to extract channel features.

[0012] Compared with existing technologies, this invention has the following advantages: Based on the YOLO11m-Pose architecture, this invention improves the C3k2 module by introducing a convolutional additive self-attention hybrid structure. It avoids the high complexity of matrix multiplication by extracting local features through convolution and modeling long-distance dependencies through additive operations. It proposes a bidirectional multi-scale information fusion network to replace the path aggregation network, realizing richer feature interaction and information fusion. It proposes a parameter-sharing convolutional detection head to replace the Pose detection head, which maintains the detection performance of human key points while reducing computation and memory consumption, and improves the model's ability to capture details. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of a 2D human pose estimation method based on YOLO11-Pose provided in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the overall structure of the YOLO11-Pose model; Figure 3 This is a schematic diagram of the overall structure of the YOLO11-CBP-Pose model provided in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of the convolutional additive self-attention hybrid structure provided in Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of a bidirectional multi-scale information fusion network provided in Embodiment 1 of the present invention; Figure 6 This is a schematic diagram of the parameter-sharing convolutional detection head structure provided in Embodiment 1 of the present invention; Figure 7 This is a schematic diagram comparing the inference results before and after model improvement in a multi-target crowd scenario provided in Embodiment 1 of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example Example 1

[0015] This invention provides a human pose estimation method based on an improved YOLO11-Pose, comprising the following steps: S1: Obtain the target image dataset and its corresponding real labels and preprocess them to obtain a subset containing only human targets and their corresponding labels. The dataset subset includes a training set and a validation set. S2: Improve the YOLO11-Pose model and establish the YOLO11-CBP-Pose model. The improvements include: S21: Introduce a convolutional additive self-attention hybrid structure (CAS) to replace the Bottleneck portion of the C3k2 module in the backbone and neck network of the YOLO11-Pose model; S22: The original path aggregation network (PANet) of the neck network of the YOLO11-Pose model is replaced by a bidirectional multi-scale information fusion network (BMIFNet). S23: Replace the original Pose detector head of the YOLO11-Pose model with a parameter-shared convolutional detector head (PSCD); S3: Input the subset of the dataset preprocessed in step S1 into the YOLO11-CBP-Pose model for training. In each epoch of the training process, the model performance is verified in real time using the validation set. After training is completed, the human target detection results and the corresponding human key point detection results are obtained through model inference.

[0016] Furthermore, the convolutional additive self-attention hybrid structure (CAS) in step S21 is composed of an Integration module, a CATM module, and an MLP module connected in series, and the CAS structure has cross-module skip connections. The Integration module includes multi-layer convolution operations, batch normalization operations, and ReLU activation operations, and the module has skip connections inside. The CATM module is a token hybrid mechanism that combines convolutional operations and additive interactions. It extracts local features through 3×3 convolution and models long-distance dependencies through additive operations, but does not include matrix multiplication and Softmax operations. The MLP module is a multilayer perceptron, which includes multilayer convolution operations, batch normalization operations, and SiLU activation operations. It is used to perform nonlinear transformations on the features output by the CATM module to extract and integrate features.

[0017] Furthermore, in the CATM module, the spatial attention map is obtained by sequentially processing 3×3 convolution, batch normalization, ReLU activation, 1×1 convolution, and Sigmoid activation, and its calculation process satisfies the formula: (1); in, This is the final spatial attention map, where x represents the input feature map of the CATM module. This indicates a 3×3 convolution operation used to extract local spatial features; Represents a 1×1 convolution operation, used to reduce the channel dimension; ReLU represents the ReLU activation function; The channel attention map is obtained by sequentially processing global average pooling, 1×1 convolution, and sigmoid activation, and its calculation process satisfies the following formula: (2); in, Here, H, W, and C represent the height, width, and number of channels of the input feature map, respectively. P is the adaptive pooling operation, and D is the grouped convolutional layer. By default, the number of groups is the same as the number of channels. This is a spatial attention map. This is a channel attention map.

[0018] Furthermore, the similarity function of the CATM module is defined as the sum of the context scores of the query (Q) and the key (K), satisfying the formula: (3); Q, K, and the value (V) are obtained through independent 1×1 convolutional linear transformations. This is a context mapping function; Q and K are integrated through additive operations, and the output of the CATM module satisfies the formula: (4); in, This represents a linear transformation used to integrate contextual information. This represents the output feature map. Let N represent the set of real numbers, and let N×C represent the dimension.

[0019] Furthermore, in the CATM module, the interaction between the spatial and channel domains is achieved through a combination of depthwise convolution and the sigmoid activation function, satisfying the formula: (5); in: This indicates the computational complexity of a specific operation. For context mapping functions, S and C are the spatial domain and channel domain, respectively. The computational complexity of the CATM module is linearly related to the size of the input feature map, satisfying the formula: (6); in, This represents the computational complexity of query (Q), key (K), and value (V) operations in the CATM module. The computational complexity of the linear transformation used to integrate contextual information is represented by H, W, and C, which are the height, width, and number of channels of the input feature map, respectively.

[0020] H, W, and C represent the height, width, and number of channels of the input feature map, respectively.

[0021] Furthermore, the bidirectional multi-scale information fusion network (BMIFNet) in step S22 includes multiple Conv modules, Concat modules, Upsample modules, and C3k2-CAS modules, and BMIFNet adopts a weighted feature fusion method; The weighted feature fusion method introduces weight parameters for each input feature, satisfying the formula: (7); in, Here, ε represents the weight parameters for the input features of the i-th layer, and ε is a preset small value (ranging from 1e-6 to 1e-4) used to mitigate numerical instability. For the i-th layer input features, These are the features after weighted fusion.

[0022] The weighted feature fusion method is used to learn the importance of different input features during the feature fusion process and to perform differentiated fusion on different features; The BMIFNet can integrate high-level semantic cues while preserving low-level spatial details, achieving richer feature interactions and information fusion than path aggregation networks.

[0023] Furthermore, the fusion process of the P4 feature layer by BMIFNet satisfies formulas (8) and (9): (8); (9); in, Represents the original input feature map. This represents the intermediate features of the 4th feature layer. This represents the output features of the 4th feature layer. This indicates the upsampling or downsampling operation used for resolution matching. This indicates a convolution operation used for feature processing.

[0024] Furthermore, the parameter-shared convolutional detector head (PSCD) in step S23 includes three independent detection branches, corresponding to feature layers P3, P4, and P5 respectively. Each detection branch has a shared depthwise separable convolutional module, and each detection branch independently has group normalization and SiLU activation operations. Each detection branch includes a bounding box regression branch, a classification branch, and a keypoint regression branch, wherein: The bounding box regression branch is used to output the center coordinates, width, height, and confidence score of the human target. The classification branch is used to output the probability of the human body category, and FocalLoss is used to handle the class imbalance problem. The keypoint regression branch is used to output the coordinates of human keypoints and the visibility score (the visibility score takes values ​​of 0, 1, and 2, corresponding to invisible, occluded, and visible, respectively).

[0025] Furthermore, the specific implementation process of the PSCD is as follows: (1) After each detection branch receives the feature map from the neck network, it first performs channel conversion through a 3×3 Conv module; (2) Feature extraction is enhanced by two shared depth separable convolutional modules. Each shared depth separable convolutional module is followed by an independent group normalization operation and SiLU activation operation. (3) Output the corresponding results through the bounding box regression branch, classification branch and key point regression branch; the bounding box regression branch is provided with a scale layer, which provides a learnable scaling factor to dynamically adjust the scale range of the bounding box prediction.

[0026] Furthermore, the YOLO11-CBP-Pose model is based on the YOLO11m-Pose model architecture, which includes a backbone network, a neck network, and a head network. The backbone network is connected in series with Conv modules, Conv modules, C3k2-CAS modules ×4, SPPF modules, and C2PSA modules. Multi-scale feature maps P2, P3, P4, and P5 are obtained through 3×3 convolution downsampling operations of multiple Conv modules. The SPPF module employs max pooling operations at three scales: 5×5, 9×9, and 13×13, concatenating the pooled feature map with the original feature map. The C2PSA module divides the input feature map into two parts: one part is directly passed through short connections, and the other part is processed by multiple convolutional layers and then fused with the short connection feature map. The neck network consists of 3 Conv modules, 4 Concat modules, 2 Upsample modules, and 4 C3k2-CAS modules connected according to feature fusion logic. The head network is a parameter-shared convolutional detector head (PSCD). Example 2

[0027] Based on the core technical solution disclosed in Example 1 and the COCO2017 public dataset, this example further refines the implementation process of human pose estimation: Please see Figures 1-6 A 2D human pose estimation method based on improved YOLO11-Pose is proposed. First, the target image dataset and its corresponding ground truth labels are obtained and preprocessed. Then, the hardware and software environment required to support the training of the YOLO11-CMP-Pose model is set up. Next, the partitioned dataset is input into the network for training. Finally, the performance of the improved model after training is evaluated.

[0028] like Figure 3As shown, the YOLO11-CBP-Pose model is based on the YOLO11m-Pose model architecture, which consists of three parts: the backbone, the neck, and the head. The backbone primarily extracts features from the input image, converting the original image into a feature map containing rich semantic information, providing a foundation for subsequent pose estimation tasks. The neck is mainly responsible for further processing and fusing the features extracted by the backbone to enhance their expressive power, thereby improving the performance of human keypoint detection. The head consists of three independent detection branches, each of which is further composed of three independent branches: a bounding box regression branch, a classification branch, and a keypoint regression branch. This decoupled multi-task design allows it to output bounding boxes, class confidence scores, and keypoint coordinates in parallel within the same network.

[0029] The backbone network includes a series of Conv modules, Conv modules, C3k2-CAS modules, Conv modules, C3k2-CAS modules, Conv modules, C3k2-CAS modules, Conv modules, C3k2-CAS modules, SPPF modules, and C2PSA modules. After 3×3 convolution downsampling operations in multiple Conv modules, feature maps P2, P3, P4, and P5 at multiple levels are obtained (assuming the initial feature map is P0) for further fusion of multi-scale feature information in the subsequent neck network.

[0030] The neck network described in this invention replaces the path aggregation network (PANet) used in the original YOLO11-pose model's neck network with a bidirectional multi-scale information fusion network (Bidirectional Multi-scale Information Fusion Network). This neck network consists of multiple Conv modules, Concat modules, Upsample modules, and C3k2-CAS modules connected in a specific order. The Concat module performs concatenation, stitching together feature information from different scales to achieve multi-scale information fusion. The Upsample module performs upsampling, magnifying large target information with low resolution and high semantic features to align with small target details with high resolution and low semantic features.

[0031] The head network adopts a parameter-shared convolutional detection head (PSCD) proposed in this invention. This network uses shared depthwise separable convolutional modules on detection branches at different scales, and each detection branch performs independent batch normalization and activation operations. The aim is to reduce computation and memory consumption by sharing the parameters of the convolutional layers, while preventing errors caused by the moving average, thereby improving model efficiency while maintaining detection performance.

[0032] In this embodiment, the Conv module first uses a standard two-dimensional convolution with a kernel size of 3×3 and a stride of 2, then performs batch normalization, and finally uses the SiLU activation function to obtain the downsampled feature map, thereby optimizing the downsampling efficiency of the feature map and reducing information loss.

[0033] The C3k2-CAS module replaces the C3k part of the original C3k2 module by introducing a Convolutional Additive Self-attention (CAS) hybrid structure, which effectively enhances the model's ability to capture local features in a refined manner, solves the problem of insufficient modeling of long-distance feature dependencies, and, through model inference verification, found that in complex scenarios, this module can effectively associate global semantic information and enhance the robustness of feature representation.

[0034] like Figure 4 As shown, the convolutional additive self-attention hybrid structure mainly consists of three parts from top to bottom: Integration, CATM, and MLP. All of them use skip connections to preserve the original features. By directly passing the detailed information of the lower layer to the higher layer and fusing it with the richer semantic information of the higher layer, the loss of information in the process of passing through the layers is avoided, thereby improving the model's ability to express complex image data. The Integration module consists of multiple consecutive components, including multi-layer convolutional (Conv) operations for extracting local image features, normalization (Norm) operations to accelerate model convergence and improve training stability, and ReLU activation to introduce non-linearity, enabling the model to learn and fit more complex functional relationships and enhancing its expressive power. Through these operations, the image's feature information is initially extracted and processed, the receptive field is increased, and it acts as a location encoding layer, providing more valuable input for subsequent modules. The CATM module is the core module of this hybrid structure. Its core idea is to enhance the query (Q) and key (K) through spatial and channel operations respectively, ultimately generating attention-weighted feature outputs. By employing an additive similarity function activated by Sigmoid, complex matrix operations are eliminated, achieving self-attention convolutional substitution. This allows for exploring more flexible network designs and improving the model's computational efficiency. Furthermore, with the support of spatial and channel attention mechanisms, the model focuses more on regions and channel information important to the current task, helping to better capture key features in the image. Considering the interactions between spatial domains, given an input feature map (Where H is the height, W is the width, and C is the number of channels). First, a 3×3 convolutional layer is applied to x, along with normalization and ReLU activation to integrate local information and capture local spatial correlations. Then, a 1×1 convolutional layer is used to reduce the channel dimension to 1. Finally, the Sigmoid activation function is applied to the single-channel feature map to obtain the final spatial attention map. (The value is between 0 and 1, used to represent the importance weight of each spatial location), the calculation process is shown in formula (1): (1); For the interactions between channel domains, global average pooling, 1×1 convolution, and sigmoid activation were used sequentially to integrate the information between channels, resulting in the final channel attention map. The calculation process is shown in formula (2): (2); in Indicates adaptive pooling. This represents a grouped convolutional layer. By default, the number of groups is the same as the number of channels. Stacking the above two operations yields a feature map after the interaction of the spatial and channel domains, denoted as . .

[0035] This module innovatively defines the similarity function as follows: The sum of context scores is shown in formula (3): (3); In this context, the query (Q), key (K), and value (V) are obtained through independent linear transformations, for example... , , , The context mapping function contains the interaction relationships between key information. The advantage of this generalization is that it is not limited to manually designed contexts, giving the possibility of implementing convolution operations. Multiple interactions performed on Q and K enable the network to learn and pay attention to more valuable tokens. The two are integrated through addition, which is a linear method that avoids the high complexity of matrix multiplication while retaining key information. Therefore, the output of CATM is shown in formula (4): (4); in This represents a linear transformation used to integrate contextual information. O This represents the output feature map. Let N represent the set of real numbers, and let N×C represent the dimension.

[0036] Complexity Analysis: In the specific implementation, the interaction between the spatial and channel domains is designed as a combination of depthwise convolution and the sigmoid activation function, where the kernel size is set to 3×3, i.e.: (5); CATM consists of four parts: QKV separable convolution, , and the final mapping layer Meanwhile, CATM maintains a linear complexity related to the input size: (6); The SPPF module uses multiple max pooling operations of different scales, such as pooling kernel sizes of 5×5, 9×9, and 13×13, to pool the feature maps. Then, these pooled feature maps are concatenated with the original feature maps. This preserves the details of the original features and includes the contextual information after multiple pooling, enabling the model to maintain high detection accuracy in human pose estimation tasks and to identify and locate joints at different positions well.

[0037] The C2PSA module divides the input feature map into two parts. One part of the feature map is directly passed to the subsequent layers through short connections, while the other part of the feature map enters a processing branch containing multiple convolutional layers. This branch extracts and transforms the image features step by step, extracting richer features through convolution operations. Then, the feature map processed by the branch is fused with the directly passed feature map. This approach can reduce the amount of computation while ensuring the diversity and integrity of features, thus reducing computational complexity.

[0038] The Concat module is used to perform splicing operations, combining feature information at different scales to achieve multi-scale information fusion.

[0039] The Upsample module is used for upsampling operations, which amplifies the information of large targets with low resolution and high semantic features to align with the details of small targets with high resolution and low semantic features.

[0040] The original YOLO11-Pose model's neck network uses the Path Aggregation Network (PANet), which focuses on cross-layer feature fusion and employs a bidirectional feature fusion approach, both top-down and bottom-up. However, its fusion effect is still significantly limited due to insufficient feature fusion and neglect of the differences between different features. Specifically, this fusion mechanism does not consider feature heterogeneity. High-level features (such as P5) contain strong semantic information but have low spatial resolution and less detailed information; low-level features (such as P2) have high spatial resolution and rich detailed information but weak semantic information. PANet uses a fusion method of "ordinary convolution + direct concatenation," which does not adapt to this heterogeneity.

[0041] like Figure 5 As shown, the bidirectional multi-scale information fusion network proposed in this invention achieves a more thorough feature fusion process compared to PANet. Relying on the feature fusion logic of "single-level enhancement + cross-level complementarity", it not only retains the key features refined at its own level, but also incorporates complementary information between features of different scales at other levels. Furthermore, it adopts a weighted fusion method to assign dynamic weights to features at different levels, and performs differentiated fusion of different features to improve the fusion performance of the network. At the same time, it uses depthwise separable convolution to replace ordinary convolution to reduce channel redundancy, making the fusion precision far exceed that of PANet.

[0042] Given a list of multi-scale features ,in Indicates the first The input features of the layer are transformed by a multi-scale information fusion network, and features at different scales are effectively aggregated to obtain a new feature list. As output.

[0043] The weighted fusion method introduces additional weights to each input feature, and the network adjusts the fusion weights for different inputs, as shown in the following formula: (7); To ensure each The values ​​are all non-negative. ), in each Then, the ReLU activation function is used, and a smaller value is introduced. To mitigate potential numerical instability.

[0044] Taking layer P4 as an example, the specific feature fusion process is shown in formulas (8) and (9): (8); (9); in, Represents the original input feature map. This represents the intermediate features of the 4th feature layer. This represents the output features of the 4th feature layer. This indicates the upsampling or downsampling operation used for resolution matching. This represents the convolution operation used for feature processing. w This indicates the corresponding weight.

[0045] like Figure 6 As shown, the parameter-shared convolutional detection head consists of three independent detection branches, corresponding to the highest resolution P3 detection layer, the medium resolution P4 detection layer, and the low resolution P5 detection layer, respectively. Each detection branch consists of three independent branches: a bounding box regression branch, a classification branch, and a keypoint regression branch. This decoupled multi-task design enables it to output bounding boxes, class confidence scores, and keypoint coordinates in parallel within the same network.

[0046] The bounding box regression branch is mainly responsible for outputting the center coordinates of the target. Width and height In addition to confidence level, a dynamic anchor box adjustment strategy is introduced to automatically adapt the anchor box size according to the dataset.

[0047] The classification branch is mainly responsible for predicting the probability of the target class, and Focal loss is used to handle the class imbalance problem.

[0048] The keypoint regression branch is mainly responsible for direct offset prediction. The coordinates of each keypoint are based on the anchor box center, and fine-grained localization is achieved through offset prediction. Compared with the traditional heatmap method, this effectively reduces quantization error and post-processing complexity. At the same time, this branch performs visibility prediction on the input feature map and outputs the visibility score of each keypoint (0=invisible, 1=occluded, 2=visible).

[0049] The original YOLO11-Pose model's Pose detection head processes input feature maps of different scales and generates detection results through separate convolution operations for each detection branch. This independent operation significantly reduces the model's computational efficiency, thereby increasing the number of parameters and computational complexity, and the branches lack information interaction. At the same time, the original detection head uses a large number of stacked standard convolutions to further extract features from the feature maps. This method generates a lot of redundant computation during model training, which is not conducive to balancing the model's detection accuracy and computational complexity. In addition, the convolutional layers in the original detection head rely excessively on batch normalization as a normalization strategy. When the batch size is small, the number of samples in the batch is insufficient, and the calculated mean and variance cannot represent the global data distribution, which may lead to distortion of the feature distribution after normalization.

[0050] To address the aforementioned issues, the parameter-sharing convolutional detection head improves the model's computational efficiency by using shared convolutional layers. Simultaneously, this detection head replaces standard convolutions with depthwise separable convolutions, further reducing the model's parameter count and computational burden. Furthermore, for different detection branches at different scales, although they share a set of convolutional kernels for weight sharing, this shared convolution can cause the feature distributions of different detection branches to shift due to input differences. Therefore, by using independent group normalization strategies for each detection branch to maintain the relative relationships between channels, each detection branch can independently calculate its mean and variance based on its own feature distribution. This avoids the problem of forced normalization of different distribution features when sharing group normalization parameters, as well as interference from cross-scale samples, thus achieving more accurate and stable feature distributions.

[0051] The specific implementation process of the parameter-sharing convolutional detection head is as follows: After receiving feature maps P3, P4, and P5 from the neck network after fusion, each detection branch first extracts features using a 3×3 Conv module and performs channel conversion to ensure consistent processing in subsequent shared convolutions. Next, feature extraction is enhanced by two shared depthwise separable convolution modules, consisting of a 3×3 depthwise convolution and a pointwise convolution. Independent group normalization and SiLU activation operations are applied after each shared depthwise separable convolution module to better adapt the model to the differences between detection branches. Finally, the feature information obtained through the above process is processed by bounding box regression, classification, and keypoint regression branches, outputting bounding box parameters, class probabilities, and human keypoint information, respectively. To enable the model to adapt to different scale regression requirements, a Scale layer is added to the regression branch of each detection branch, providing a learnable scaling factor to dynamically adjust the scale range of bounding box prediction.

[0052] The dataset used in this embodiment is the COCO2017 public dataset released by Microsoft. This dataset is currently the most commonly used dataset for detecting human keypoints in multiple people. The training set contains 118,282 images, and the validation set contains 5,000 images. After preprocessing, a subset COCO-POSE containing only human targets is obtained, with the training set containing 56,599 images and the validation set containing 2,346 images.

[0053] In this embodiment, to verify the impact of each improved module on the performance of the baseline model YOLO11m-Pose, ablation experiments were conducted on each improved module. The experimental results are shown in Table 1: Table 1 ; Experimental results show that after improving the original model's C3k2 module using a CAS hybrid structure, the mAP50 and mAP50:95 scores increased by 1.1 and 0.9 percentage points, respectively, indicating that it effectively enhances the model's feature representation ability. Replacing the original model's path aggregation network PANet with the bidirectional multi-scale information fusion network BMIFNet increases computational overhead, but significantly improves model accuracy. Replacing the original model's Pose detection head with a parameter-shared convolutional detection head significantly reduces the number of parameters and computational cost, while still slightly improving accuracy. This demonstrates that the detection head can provide auxiliary gains in detection performance while reducing model resource consumption. Each module contributes effectively to the overall improvement; CAS and BMIFNet focus on improving model accuracy, while PSCD emphasizes high efficiency and lightweight design. Meanwhile, the inter-module synergy is significant, especially when all modules are combined. Compared with the baseline model, the mAP50 and mAP50:95 indices increased by 2.8 and 2.6 percentage points, respectively. However, the number of parameters and the amount of computation only increased by 1.91% and 1.96%, respectively, and the model accuracy and computational complexity were effectively balanced.

[0054] In this example, to verify the effectiveness of the YOLO11-CBP-Pose model proposed in this invention, a comparative experiment was conducted with some mainstream human pose estimation models. The experimental results are shown in Table 2: Table 2 ; Experimental results show that, compared with the baseline model YOLO11m-Pose, the proposed YOLO11-CBP-Pose model achieves significant improvements in all accuracy metrics. In particular, this method improves accuracy in AP... 50The accuracy reached 92.0%, the highest among all methods in the table, indicating excellent performance under a relaxed IoU threshold. Compared to the YOLO-Pose and YOLOv8m-Pose models in the same YOLO series, the model proposed in this invention significantly outperforms in all accuracy metrics. Compared to the high-performance top-down method HRNet-W32, although this method lags behind in most accuracy metrics, it's important to note the differences in algorithmic paradigms and input sizes. As a single-stage detection model, this method does not rely on an additional human detector, thus achieving end-to-end inference and offering advantages in efficiency and practicality. Overall, this method achieves an optimal trade-off between model accuracy and parameter count, providing a powerful candidate for real-time, high-precision human pose estimation applications.

[0055] like Figure 7 As shown, this embodiment verifies the actual effect of the improvement measures by comparing the inference results of the original model and the improved model in a multi-target crowd scenario. First, the original model exhibits fluctuations in the confidence level of human target detection, with some detection results showing low confidence, reflecting insufficient reliability in human target detection. In contrast, the improved model generally demonstrates higher confidence in human target detection, indicating more stable and reliable accuracy in human target recognition. Second, the original model's bounding boxes do not fit the human target well, with some selection ranges not matching the actual outline of the person. In comparison, the improved model's bounding boxes more accurately surround the person, significantly improving the matching degree and resulting in more accurate person localization. Finally, the original model has deviations in the connection and annotation of human key points, with some key point connections not conforming to the logical structure of the human body. The improved model's connections of human key points better conform to the human motion structure, with better accuracy and consistency in annotation, more accurately reflecting human posture characteristics.

[0056] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A human pose estimation method based on an improved YOLO11-Pose, characterized in that, Includes the following steps: S1: Obtain the target image dataset and its corresponding real labels and preprocess them to obtain a subset containing only human targets and their corresponding labels. The dataset subset includes a training set and a validation set. S2: Improve the YOLO11-Pose model and establish the YOLO11-CBP-Pose model. The improvements include: S21: Introduce a convolutional additive self-attention hybrid structure to replace the Bottleneck portion of the C3k2 module in the backbone and neck network of the YOLO11-Pose model; S22: The original path aggregation network of the neck network of the YOLO11-Pose model is replaced with a bidirectional multi-scale information fusion network; S23: Replace the original Pose detection head of the YOLO11-Pose model with a parameter-shared convolutional detection head; S3: Input the subset of the dataset preprocessed in step S1 into the YOLO11-CBP-Pose model for training. In each epoch of the training process, the model performance is verified in real time using the validation set. After training is completed, the human target detection results and the corresponding human key point detection results are obtained through model inference.

2. The human pose estimation method based on the improved YOLO11-Pose according to claim 1, characterized in that, The convolutional additive self-attention hybrid structure in step S21 is composed of an Integration module, a CATM module, and an MLP module connected in series, and the CAS structure has a jump connection across modules. The Integration module includes multi-layer convolution operations, batch normalization operations, and ReLU activation operations, and the module has skip connections inside. The CATM module is a token hybrid mechanism that combines convolutional operations and additive interactions. It extracts local features through convolution and models long-distance dependencies through additive operations, but does not include matrix multiplication and Softmax operations. The MLP module is a multilayer perceptron, which includes multilayer convolution operations, batch normalization operations, and SiLU activation operations. It is used to perform nonlinear transformations on the features output by the CATM module to extract and integrate features.

3. The human pose estimation method based on the improved YOLO11-Pose according to claim 2, characterized in that, In the CATM module, the spatial attention map is obtained by sequentially processing 3×3 convolution, batch normalization, ReLU activation, 1×1 convolution, and Sigmoid activation, and its calculation process satisfies the following formula: （1）； in, This is the final spatial attention map, where x represents the input feature map of the CATM module. This indicates a 3×3 convolution operation used to extract local spatial features; Represents a 1×1 convolution operation, used to reduce the channel dimension; ReLU represents the ReLU activation function; The channel attention map is obtained by sequentially processing global average pooling, 1×1 convolution, and sigmoid activation, and its calculation process satisfies the following formula: （2）； in, Let H be the input feature map, W be the height, W be the width, and C be the number of channels of the input feature map, respectively. P is the adaptive pooling operation, and D is the grouped convolutional layer. This is a spatial attention map. This is a channel attention map.

4. The human pose estimation method based on the improved YOLO11-Pose according to claim 3, characterized in that, The similarity function of the CATM module is defined as the sum of the context scores of the query (Q) and the key (K), satisfying the formula: （3）； Among them, Q, K, and V (values) are obtained through independent 1×1 convolutional linear transformations. For context mapping functions; Q and K are integrated through additive operations, and the output of the CATM module satisfies the formula: （4）； in, This represents a linear transformation used to integrate contextual information. This represents the output feature map. Let N represent the set of real numbers, and let N×C represent the dimension.

5. The human pose estimation method based on the improved YOLO11-Pose according to claim 3, characterized in that, In the CATM module, the interaction between the spatial and channel domains is achieved through a combination of depthwise convolution and the Sigmoid activation function, satisfying the formula: （5）； in, This indicates the computational complexity of a specific operation. S , C These are respectively the spatial domain and the channel domain; The computational complexity of the CATM module is linearly related to the size of the input feature map, satisfying the formula: （6）； in, This represents the computational complexity of query (Q), key (K), and value (V) operations in the CATM module. The computational complexity of the linear transformation used to integrate contextual information is represented by H, W, and C, which are the height, width, and number of channels of the input feature map, respectively.

6. The human pose estimation method based on the improved YOLO11-Pose according to claim 1, characterized in that, The bidirectional multi-scale information fusion network in step S22 includes multiple Conv modules, Concat modules, Upsample modules, and C3k2-CAS modules, and BMIFNet adopts a weighted feature fusion method. The weighted feature fusion method introduces weight parameters for each input feature, satisfying the formula: （7）； in, Here, ε represents the weight parameters for the input features of the i-th layer, and ε is a preset small value, ranging from 1e-6 to 1e-4, used to mitigate numerical instability. For the i-th layer input features, These are the features after weighted fusion.

7. The human pose estimation method based on improved YOLO11-Pose according to claim 6, characterized in that, The fusion process of the P4 feature layer in BMIFNet satisfies formulas (8) and (9): （8） （9）； in, , , Represents the original input feature map. This represents the intermediate features of the 4th feature layer. This represents the output features of the 4th feature layer. This indicates the upsampling or downsampling operation used for resolution matching. This represents the convolution operation used for feature processing. w This indicates the corresponding weight.

8. The human pose estimation method based on the improved YOLO11-Pose according to claim 1, characterized in that, The parameter-shared convolutional detection head in step S23 includes three independent detection branches, corresponding to feature layers P3, P4, and P5 respectively. Each detection branch has a shared depthwise separable convolutional module, and each detection branch independently has group normalization and SiLU activation operations. Each detection branch includes a bounding box regression branch, a classification branch, and a keypoint regression branch, wherein: The bounding box regression branch is used to output the center coordinates, width, height, and confidence score of the human target. The classification branch is used to output the probability of the human body category, and FocalLoss is used to handle the class imbalance problem. The keypoint regression branch is used to output the coordinates of human keypoints and visibility scores.

9. The human pose estimation method based on the improved YOLO11-Pose according to claim 8, characterized in that, The specific implementation process of the PSCD is as follows: (1) After each detection branch receives the feature map from the neck network, it first performs channel conversion through a 3×3 Conv module; (2) Feature extraction is enhanced by two shared depth separable convolutional modules. Each shared depth separable convolutional module is followed by an independent group normalization operation and SiLU activation operation. (3) Output the corresponding results through the bounding box regression branch, classification branch and key point regression branch; the bounding box regression branch is provided with a scale layer, which provides a learnable scaling factor to dynamically adjust the scale range of the bounding box prediction.

10. The human pose estimation method based on the improved YOLO11-Pose according to claim 1, characterized in that, The YOLO11-CBP-Pose model is based on the YOLO11m-Pose model architecture, which includes a backbone network, a neck network, and a head network. The backbone network is connected in series with Conv module, Conv module, C3k2-CAS module, SPPF module, and C2PSA module. Multi-scale feature maps P2, P3, P4, and P5 are obtained through 3×3 convolution downsampling operations of multiple Conv modules. The SPPF module employs max pooling operations at three scales: 5×5, 9×9, and 13×13, concatenating the pooled feature map with the original feature map. The C2PSA module divides the input feature map into two parts: one part is directly passed through short connections, and the other part is processed by multiple convolutional layers and then fused with the short connection feature map. The neck network consists of 3 Conv modules, 4 Concat modules, 2 Upsample modules, and 4 C3k2-CAS modules connected according to feature fusion logic. The head network is a parameter-shared convolutional detection head.

Images