A lightweight human pose estimation method based on PLES-YOLO

Abstract

The application discloses a kind of light human posture estimation methods based on PLES-YOLO, belong to computer vision field, this method includes the following steps: using improved down-sampling module to carry out feature extraction to input image, obtain initial feature map;Based on the initial feature map, using multi-scale feature fusion module carries out progressive feature enhancement, obtains multi-scale fusion feature;Based on the multi-scale fusion feature, using the spatial pyramid pooling module of combination attention mechanism carries out feature optimization, obtains weighted multi-scale feature;Based on the weighted multi-scale feature, using light shared convolution detection head predicts human key point coordinates, outputs final posture estimation result.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and in particular relates to a lightweight human pose estimation method based on PLES-YOLO. Background Technology

[0002] Traditional human pose estimation algorithms are based on graph structure models, relying on manually designed joint topology relationships and image feature descriptions. They lack adaptive learning mechanisms and have significant shortcomings in algorithm accuracy and cross-scene generalization ability. With the continuous development of deep learning technology and neural network models, human pose estimation has gradually transitioned from image processing to iterative inference using neural networks, achieving significant progress. Currently, deep learning-based human pose estimation algorithms are mainly divided into bottom-up and top-down approaches. Bottom-up methods prioritize identifying all keypoints in the image and then construct pose models for each individual. Represented by the OpenPose algorithm developed by Cao et al., it uses PAF (Part Affinity Fields) technology to achieve cross-individual keypoint association, meeting the needs of real-time pose estimation for multiple users. The HigherHRNet algorithm proposed by Cheng et al. uses a multi-branch structure to process multi-resolution feature maps in parallel, improving the accuracy of keypoint detection based on an iterative optimization mechanism, and exhibiting excellent performance in multi-user scenarios. However, this method requires extremely high accuracy in the keypoint matching algorithm and is prone to association errors. Top-down methods, on the other hand, first locate the human target in the image and then perform pose estimation independently for each target. Represented by SimpleBaseline proposed by Xiao et al., its network structure consists of a backbone network and simple deconvolution modules. It lacks complex skip connections or multi-branch structures, making it easy to understand and implement with relatively low computational cost, achieving a good balance between speed and accuracy. The HRNet algorithm proposed by Sun et al. effectively preserves image details through a parallel multi-resolution subnet architecture, avoiding the feature loss problem caused by traditional downsampling operations. However, in complex background environments, these methods require scaling up the model to ensure detection accuracy, thus affecting real-time performance. Furthermore, in multi-user interactive scenarios, they are easily affected by target occlusion and bounding box localization errors.

[0003] To address the above issues, in 2022, Maji et al. proposed the YOLOv5-Pose single-stage human pose estimation method based on the YOLO framework. Its core design abandons the heatmap generation and parsing process, optimizing the balance between real-time performance and accuracy through end-to-end coordinate prediction. In 2023, Zhao et al., based on YOLOv5-Pose, improved the accuracy of multi-person pose estimation in crowded scenes by combining a jump attention module and a Transformer. In 2024, Lu et al. proposed the KSL-POSE model based on YOLOv8-Pose, enhancing the model's feature extraction capabilities by introducing a non-linear activation function and combining it with KAN convolution. In 2025, Tu et al. proposed the HP-YOLO model based on the YOLOv8-Pose framework, improving the accuracy of keypoint and small target detection and accelerating inference speed by integrating an ELSKA mechanism into the backbone network and introducing a RepNCSPELAN4 module into the probe.

[0004] In summary, YOLO-based algorithms have made significant progress in human pose estimation tasks. However, the balance between model complexity and detection accuracy remains a challenge, which greatly limits the practical application of YOLO algorithms. Therefore, this invention proposes an optimized YOLOv11n-Pose human pose estimation scheme. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a lightweight human pose estimation method based on PLES-YOLO, thereby resolving the issues present in the prior art.

[0006] To achieve the above objectives, this invention provides a lightweight human pose estimation method based on PLES-YOLO, comprising:

[0007] An improved downsampling module is used to extract features from the input image to obtain an initial feature map;

[0008] Based on the initial feature map, a multi-scale feature fusion module is used to perform progressive feature enhancement to obtain multi-scale fused features;

[0009] Based on the multi-scale fusion features, a spatial pyramid pooling module combined with an attention mechanism is used for feature optimization to obtain weighted multi-scale features.

[0010] Based on the weighted multi-scale features, a lightweight shared convolutional detector head is used to predict the coordinates of human key points and output the final pose estimation result.

[0011] Optionally, the improved downsampling module adopts a windmill-shaped asymmetric convolution structure;

[0012] The process of extracting features from the input image using an improved downsampling module to obtain an initial feature map includes:

[0013] The windmill-shaped asymmetric convolutional structure extracts multi-directional branch tensors through separate convolutional kernels in the horizontal and vertical directions;

[0014] The multi-directional branch tensors are concatenated to obtain the concatenated tensor;

[0015] The concatenated tensor is convolved with a 2x2 kernel to obtain the concatenated features.

[0016] The spliced ​​features are normalized to obtain the output tensor;

[0017] An initial feature map is obtained based on the output tensor.

[0018] Optionally, the output tensor is:

[0019]

[0020] In the formula, h', w', and c' represent the height, width, and number of channels of the output tensor of each branch, respectively; X' represents the concatenated tensor; and Y represents the output tensor. This indicates a 2×2 convolution kernel with output channel c2. SiLU is the convolution operator, where h2, w2, and c2 represent the height, width, and channel size of the input tensor X(h2, w2, c2), respectively, and SiLU represents the activation function.

[0021] Optionally, the multi-scale feature fusion module is: Progressive Multi-Scale Feature Extraction (EPMF) module;

[0022] The process of progressive feature enhancement using a multi-scale feature fusion module to obtain multi-scale fused features includes:

[0023] The initial feature map is pre-processed using a 3×3 depthwise separable convolution to obtain the first-level features;

[0024] The first-level feature is evenly divided into two parts according to the channel dimension, and the first part of the feature is retained;

[0025] The first part of the features is processed by a 5×5 depthwise separable convolution to obtain the second-level features;

[0026] The second-level feature is further divided evenly along the channel dimension, retaining the second part of the feature;

[0027] The second part of the features is processed by a 7×7 depthwise separable convolution to obtain the third level of features;

[0028] The first-level features, second-level features, and third-level features are concatenated and integrated;

[0029] By adding the input features of the initial feature map to the concatenated features through residual connections, multi-scale fused features are obtained.

[0030] Optionally, the spatial pyramid pooling module that incorporates the attention mechanism is an SPPF_LSKA module;

[0031] The process of using a spatial pyramid pooling module incorporating an attention mechanism for feature optimization to obtain weighted multi-scale features includes:

[0032] Multi-level pooling features of different scales are generated through spatial pyramid pooling;

[0033] A large separable kernel attention LSKA mechanism is used to process the pooling features at each level to obtain attention-weighted multi-level pooling features. The LSKA mechanism decomposes a two-dimensional large convolution kernel by cascading horizontal and vertical one-dimensional kernels.

[0034] Attention-weighted multi-level pooling features are fused to obtain weighted multi-scale features.

[0035] Optionally, the lightweight shared convolutional detection head is an LSCP module;

[0036] The process of predicting human keypoint coordinates using a lightweight shared convolutional detector head includes:

[0037] Common features are extracted through a core feature extraction layer composed of shared 3×3 convolutions and 1×1 convolutions;

[0038] The bounding box prediction branch and key point prediction branch are applied to the general features respectively to obtain the prediction results of the human body key point coordinates.

[0039] The present invention also provides a computer terminal device, comprising:

[0040] One or more processors;

[0041] A memory, coupled to the processor, for storing one or more programs;

[0042] When the one or more programs are executed by the one or more processors, the one or more processors implement the method.

[0043] The present invention also provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method described thereon.

[0044] Compared with the prior art, the present invention has the following advantages and technical effects:

[0045] This invention significantly enhances feature extraction capabilities by improving the downsampling module, expanding the receptive field while controlling the increase in parameter size. A progressive multi-scale feature fusion module effectively preserves shallow details and fuses deep semantic information, improving the detection capability for targets at different scales. A spatial pyramid pooling module combined with an attention mechanism dynamically adjusts the receptive field and strengthens key features, effectively suppressing interference from complex backgrounds. A lightweight shared convolutional detection head significantly reduces computational complexity through parameter sharing and structural optimization. This scheme improves detection accuracy while reducing the number of model parameters, achieving a synergistic optimization of accuracy and efficiency. Attached Figure Description

[0046] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0047] Figure 1 This is a diagram of the YOLOv11-Pose network structure according to an embodiment of the present invention;

[0048] Figure 2 This is a diagram of the windmill-shaped convolutional network structure according to an embodiment of the present invention;

[0049] Figure 3 This is a network structure diagram of the SPPF_LSKA module in an embodiment of the present invention;

[0050] Figure 4 This is a diagram of the LSKA network structure according to an embodiment of the present invention;

[0051] Figure 5 This is a diagram of the progressive multi-scale feature extraction network structure according to an embodiment of the present invention;

[0052] Figure 6 This is a diagram of the lightweight shared convolutional pose detection head network structure according to an embodiment of the present invention;

[0053] Figure 7 This is a diagram showing the overall structure of the PLES-YOLO network, an improved human pose estimation network according to an embodiment of the present invention.

[0054] Figure 8 This is a comparison chart of the first actual detection results according to an embodiment of the present invention;

[0055] Figure 9 This is a comparison chart of the second actual detection results according to an embodiment of the present invention. Detailed Implementation

[0056] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0057] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0058] Example 1

[0059] like Figure 1 As shown, this embodiment provides a lightweight human pose estimation method based on PLES-YOLO. By constructing a novel feature fusion structure and a lightweight convolutional module, the method enhances the model's ability to estimate human pose while reducing network parameters and computational cost. The method includes the following steps:

[0060] YOLOv11-Pose is the state-of-the-art model in pose estimation developed by the Ultralytics team. It continues the excellent achievements of previous YOLO-Pose versions and features significant innovations in its architectural design. Its network structure diagram is shown below. Figure 1 As shown, YOLOv11-Pose significantly enhances its ability to extract features from images and improves object detection performance by reconstructing the backbone and neck networks and introducing innovative modules such as C3k2 and C2PSA. The C3k2 module is an improved design based on the traditional C2f and C3 modules. It constructs its core component, C3k, by fusing variable convolutional kernels (such as 3x3, 5x5, etc.) and channel separation strategies. Furthermore, the C3k2 module can control whether to use the C3k structure via the boolean variable c3k; if False, the conventional Bottleneck structure is used. This flexible design gives C3k2 more efficient feature extraction capabilities, making it particularly suitable for complex scenes and deep feature extraction tasks. The C2PSA module is a high-level module for enhanced feature extraction, combining the CSP (Cross Stage Partial) structure with the PSA (Pyramid Squeeze Attention) attention mechanism to further improve multi-scale feature extraction performance. This module is mainly used for object detection in complex scenes, and its performance is particularly outstanding when handling multi-scale objects.

[0061] The lightweight network structure of the present invention includes:

[0062] (1) PSConv: Traditional convolution involves a large amount of convolutional computation during downsampling to extract information from the input layer. It extracts spatial and channel features using fixed convolutional kernels, resulting in computational redundancy, insufficient spatial adaptability, and weak semantic focusing ability. Unlike traditional convolution, pinwheel-shaped convolutional (PSConv) uses asymmetric padding to create horizontal and vertical convolutional kernels for different regions of the image. Its network structure diagram is shown below. Figure 2 As shown in the diagram. Furthermore, batch normalization (BN) and the SiLU activation function are applied after each convolution to enhance training stability and speed. The formulas are shown below.

[0063]

[0064] Where h1, w1, and c1 represent the height, width, and channel size of the input tensor X(h1, w1, c1), respectively. This is the convolution operator. W1(1, 3, c') is a 1×3 convolution kernel with output channels c'. The padding parameter P(1, 0, 0, 3) represents the number of padding pixels in the left, right, top, and bottom directions, respectively. h', w', and c' represent the height, width, and number of channels of the output tensor of each branch, respectively. Their relationship with the input tensor is shown in the following formula:

[0065]

[0066] Where c2 is the number of channels in the final output feature map from the PSConv module, and S is the convolution stride. The four branches are concatenated together, fusing the features extracted from convolutions in different directions to enrich the feature information and form a new tensor X', the formula of which is shown below:

[0067]

[0068] The concatenated tensor is then subjected to a 2x2 convolution operation without padding. This convolution normalizes the concatenated features, adjusting the number of channels in the output feature map to meet the input requirements of subsequent networks, ultimately yielding the output feature map and the final tensor Y, as shown in the formula below:

[0069]

[0070] In the formula, h', w', and c' represent the height, width, and number of channels of the output tensor of each branch, respectively; X' represents the concatenated tensor; and Y represents the output tensor. This indicates a 2×2 convolution kernel with output channel c2. SiLU is the convolution operator, where h2, w2, and c2 represent the height, width, and channel size of the input tensor X(h2, w2, c2), respectively, and SiLU represents the activation function.

[0071] As shown in formula (4), h2 and w2 represent the height and width of the final output tensor Y, respectively, and their formulas are as follows:

[0072]

[0073] This design, by utilizing PSConv's unique windmill-shaped structure, can achieve a larger receptive field without significantly increasing the number of parameters, effectively capturing the target's contextual information. Especially when the target size is small and the background is complex, it can effectively reduce false negatives and improve detection accuracy and robustness.

[0074] (2) LSKA: In YOLOv11-Pose, Spatial Pyramid Pooling-Fast (SPPF), as a multi-scale feature fusion module, inherits the advantages of efficient computation. However, due to its design intent and structural characteristics, it is insufficient in adapting to complex pose scales, preserving fine-grained features, and complementing multi-scale information, especially in supporting high-precision joint localization. To address these issues, the SPPF_LSKA module is proposed, and its network structure diagram is shown below. Figure 3 As shown.

[0075] Visual Attention Networks (VANs) have demonstrated superior performance on various visual tasks, even surpassing Vision Transformers (ViTs), thanks to their Large Kernel Attention (LKA) modules. However, the computational cost and memory usage of deep convolutional layers in the LKA module increase quadratically with the size of the convolutional kernel, limiting the application of ultra-large convolutional kernels. To overcome this computational bottleneck, existing technologies have proposed Large Separable Kernel Attention (LSKA) modules. The core innovation of LSKA lies in decomposing the two-dimensional large convolutional kernels of deep convolutional layers into cascaded horizontal and vertical one-dimensional kernels, with a network structure as follows: Figure 4 As shown. Compared to standard LKA, this separable decomposition strategy allows VAN to directly and efficiently use maximal convolutional kernels in its attention module without introducing additional structures, significantly reducing computational complexity and memory footprint while maintaining performance comparable to standard LKA.

[0076] The SPPF_LSKA module introduces the LSKA attention mechanism after the concatenation operation. On the one hand, by leveraging the multi-size kernels of separable convolutions and the dynamic selection capability of attention, it overcomes the problem of rigid receptive field caused by the fixed pooling structure of traditional SPPF, and can adaptively match local details or global context according to the target scale. On the other hand, the channel and spatial attention weighting mechanism of LSKA replaces the simple concatenation of multi-scale features in traditional SPPF, effectively filtering redundant information and strengthening key features. At the same time, the design of separable convolutions reduces computational complexity while avoiding the loss of details caused by pooling, and improves the effectiveness and localization accuracy of multi-scale feature fusion.

[0077] (3) Progressive Multi-scale Feature Extraction: C3k2 is an innovative module in YOLOv11-Pose, but as the network depth increases, the feature map resolution decreases, details of small targets are easily lost, and its adaptability to targets of different scales is limited, making it difficult to process large and small targets simultaneously. To address these issues, the Progressive Multi-scale Feature Extraction (EPMF) module is proposed. The network structure diagram is shown below. Figure 5 As shown, the EPMF module does not directly perform multi-scale operations on the input features all at once. Instead, it processes the features step by step through feature decomposition and progressive propagation, capturing multi-scale information by combining convolutional kernels of different sizes (such as 3×3, 5×5, and 7×7). Secondly, it introduces depthwise separable convolution (DWConv) to reduce the parameter impact caused by large-size convolutional kernels, improving inference speed while maintaining performance.

[0078] The EPMF module achieves efficient feature enhancement and avoidance of redundant computation through a progressive feature processing chain and a multi-scale information fusion strategy. Its core logic lies in the hierarchical capture and precise integration of receptive fields at different scales. First, a 3×3 DWConv is used to preliminarily process the input raw features, significantly reducing computation while maintaining feature extraction capabilities. Then, the module uses a chunk function to evenly split the processed feature map into two parts along the channel dimension. One part of the features is then processed in the next stage, where a 5×5 DWConv is used to further expand the receptive field, capturing a wider range of spatial relationships than the 3×3 convolution. After processing, it is again evenly split into two parts along the channel dimension. Next, another part of the features is processed by a 7×7 DWConv, whose ultra-large receptive field effectively captures deep semantic information. Each of these processing steps is based on the splitting of features from the previous stage, forming a progressive chain and avoiding redundant computation caused by processing all-scale information at once. Finally, the outputs of each stage are spliced ​​and integrated, which not only preserves shallow details but also incorporates deep semantics. In addition, the original input features are added to the final integrated features through residual connections, which effectively alleviates the gradient vanishing problem in deep networks and ensures that shallow details can be smoothly transmitted to subsequent network layers, further enhancing the integrity of the information flow.

[0079] (4) Lightweight Shared Convolutional Pose Detection Head: YOLOv11-Pose, as the latest pose estimation version in the YOLO series, continues the previous "detection + pose" joint optimization approach in its detection head, emphasizing efficiency and versatility. To balance speed, coarse-grained feature fusion is typically used, and keypoint prediction and bounding box detection share most of their features, which may lead to a decrease in the localization accuracy of small-scale keypoints or occluded joints. To address these issues, a Lightweight Shared Convolutional Pose Detection Head (LSCP) is proposed, with the network structure as follows: Figure 6 As shown, the LSCP module introduces a shared convolution mechanism and separate Batch Normalization (BN) calculation. The core feature extraction, composed of 3×3 and 1×1 convolutions, is set as a shared module. All detection layers reuse the weight parameters of this set of convolutions. By sharing the feature extraction layer, the identical features are avoided from being calculated separately for each detection layer, significantly reducing redundant computation and storage overhead. Furthermore, each detection layer has independent BN and SiLU activation functions, further enhancing the non-linear expressive power of features through smooth non-linear transformations. This allows for more precise adjustment of feature distribution and improves the detection branch's ability to capture complex features.

[0080] In terms of network structure design for the keypoint branch, YOLOv11-Pose adopts a relatively simple architecture, with its core consisting of a 3×3 DWConv and a 3×3 ordinary convolution. DWConv achieves lightweighting by separating channel and spatial convolution, while the 3×3 convolution is responsible for capturing local spatial relationships. The two are concatenated to directly output the predicted keypoint coordinates and visibility. While this structure can balance accuracy and speed to some extent, it is limited by the feature representation capabilities of a single convolution combination, limiting its ability to capture occluded keypoints and small-scale targets in complex scenes. Furthermore, it lacks a targeted dimensionality adjustment mechanism, making it susceptible to affecting prediction accuracy due to feature channel redundancy or insufficiency. In contrast, the LSCP module employs a more refined convolution structure design strategy: the first 1×1 convolution reduces subsequent computation by integrating channels and dimensionality reduction, preserving core information; the middle 3×3 convolution focuses on spatial feature extraction, utilizing a larger receptive field to capture spatial relationships between keypoints and enhancing the non-linear expressive power of features through the SiLU activation function; the final 1×1 convolution increases dimensionality, accurately mapping the processed features to the required output dimension for the keypoints. This structural optimization allows the LSCP module to maintain high keypoint detection accuracy while achieving independent dimensionality reduction through two 1×1 convolution operations on the keypoint branch. This avoids the computational redundancy of 3×3 convolution under high channel counts and improves feature extraction efficiency.

[0081] In summary, the overall structure of the proposed improved human pose estimation network PLES-YOLO is shown in the figure below. Figure 7 As shown, this approach is based on the YOLOv11-Pose model. First, it replaces the original downsampling module with PSConv, expanding the receptive field while minimizing the increase in the number of parameters, effectively improving feature extraction capabilities. Second, it introduces the LSKA attention mechanism into the SPPF module to dynamically adjust the receptive field and weightedly fuse multi-scale features, preserving detailed information while reducing computational complexity. Then, by optimizing the original detector head, it achieves lightweight shared convolution, utilizing independent BN and SiLU activation functions, as well as independent dimensionality reduction processing for keypoint branches, enabling more accurate processing of features at different scales and improving keypoint localization accuracy. Finally, based on this, it proposes the EPMF module, using progressive processing and convolutional kernels of different sizes to cover the entire human body scale and achieve cross-layer feature reuse.

[0082] Ultimately, this design enables PLES-YOLO to effectively optimize both model accuracy and computational complexity. This approach significantly reduces the computational resources required for model inference, making it feasible for efficient deployment in resource-constrained environments such as mobile terminals and embedded devices, thereby enhancing the model's practicality and competitiveness in a wide range of real-world applications.

[0083] As a specific implementation of this embodiment, to verify the performance of the PLES-YOLO model, experiments were conducted on the COCO-Pose2017 and MPII datasets. The COCO-Pose2017 dataset is a subset of the COCO-2017 dataset specifically designed for human keypoint detection, containing a total of 58,945 images, with 56,599 images in the training set and 2,346 images in the validation set. The dataset includes images of single individuals, multiple individuals, and various scenes, taking into account occlusion handling and densely populated scenes. The MPII dataset contains approximately 25,000 images, covering 410 human activities, providing rich data support for the training and evaluation of the pose estimation model in different activity scenarios. The training set contains approximately 15,000 images, the validation set approximately 3,000 images, and the test set approximately 7,000 images.

[0084] The experimental environment is based on Windows 10, and the specific configuration is shown in Table 1 below:

[0085] Table 1

[0086]

[0087] Experimental Metrics: In human pose estimation tasks, the keypoint similarity between model-predicted keypoints and ground truth keypoints is not calculated solely using Euclidean distance. Instead, a certain scale is incorporated to calculate the similarity between the two points. Object Keypoint Similarity (OKS), which considers keypoint positional deviation, size factors, and visibility, is generally regarded as an important metric. Its formula is shown below:

[0088]

[0089] Where i represents all keypoint types defined for the human body; d i is the Euclidean distance between the predicted and actual locations of the i-th keypoint; s is the scale of the target region, generally used for scale normalization; k i is the weight factor for the i-th keypoint type, a predefined constant for different keypoint types, reflecting the repeatability of keypoint annotation; v i The visibility label for the i-th keypoint, v i =0 indicates no labeling, usually not within the image, v i =1 indicates that the label is not visible, usually meaning it is obscured, v i =2 indicates that the key point is labeled and visible, and is clearly visible in the image; δ means that when the key point is labeled, the function value is 1 regardless of whether it is visible or not, and the function value is 0 when the key point is not labeled.

[0090] The OKS value ranges from 0 to 1. The closer the value is to 1, the better the predicted human keypoints match the true values ​​labeled in the dataset, and the better the model's prediction performance; conversely, the lower the value, the better. In practical applications, OKS is often combined with metrics such as precision (P) and recall (R) to comprehensively evaluate the performance of human pose estimation models.

[0091] This embodiment selects several variants of Average Precision (AP), including AP50, AP75, and AP(0.50:0.95), and corresponding metrics for Average Recall (AR), AR50, AR75, and AR(0.50:0.95), as key indicators for evaluating model performance. AP50 represents the average precision at a threshold of 0.5; AP75 corresponds to the average precision at a threshold of 0.75; and AP(0.50:0.95) covers all average precision values ​​from 0.5 to 0.95. AR measures the proportion of targets correctly identified by the model among all real targets. AR50 and AR75 correspond to average recall at thresholds of 0.5 and 0.75, respectively; while AR(0.50:0.95) represents the average recall across the entire range of thresholds from 0.5 to 0.95. In addition, the number of parameters and GFLOPs were selected as key indicators for comparing model size and computational complexity.

[0092] Comparative Experiment Results: Table 2 shows the experimental results of the PLES-YOLO model on the COCO-Pose dataset compared with mainstream models such as litehrnet, simcc, rtmpose, YOLOv5-pose, YOLOv8-pose, and YOLOv11-pose.

[0093] Table 2

[0094]

[0095] The experimental results show that, compared with the baseline model YOLOv11-pose, the PLES-YOLO model has 17% fewer parameters, increases AP50 by 1.3 percentage points, AP75 by 1.5 percentage points, AP(0.50:0.95) by 1.2 percentage points, AR50 by 0.8 percentage points, AR75 by 1.5 percentage points, and AR(0.50:0.95) by 1 percentage point. These results demonstrate that the PLES-YOLO model has higher detection accuracy and better performance than the baseline model YOLOv11-pose, while also significantly reducing the number of parameters. Compared with mainstream models such as litehrnet, simcc, rtmpose, YOLOv5-pose, and YOLOv8-pose, the PLES-YOLO model has a significant advantage in accuracy while also reducing the number of parameters.

[0096] In addition, this embodiment was further tested on the MPII dataset, and the experimental results are shown in Table 3.

[0097] Table 3

[0098]

[0099]

[0100] The experimental results show that, compared with the baseline model YOLOv11-pose, the PLES-YOLO model improves AP50 by 1.2 percentage points, AR75 by 0.8 percentage points, AP(0.50:0.95) by 0.7 percentage points, AR50 by 0.7 percentage points, AP75 by 0.9 percentage points, and AP(0.50:0.95) by 1 percentage point. Analysis of these experimental results leads to the same conclusion: the PLES-YOLO model is more efficient than the baseline model YOLOv11-pose and is more suitable for deployment on devices with limited computing resources.

[0101] Ablation experiments: Ablation experiments were conducted on the COCO-Pose2017 dataset to test the effectiveness of the four modules PSConv, SPPF_LSKA, EPMF and LSCP. The training settings remained unchanged. The experimental results are shown in Table 4.

[0102] Table 4

[0103] - - - - 2.9 7.4 82.8 √ - - - 2.6 6.7 82.5 - √ - - 3.1 7.6 83.4 - - √ - 2.7 7.4 83.1 - - - √ 2.5 7.5 83.9 √ √ - - 2.8 6.9 82.7 √ - √ - 2.4 6.7 82.8 √ - - √ 2.2 6.7 83.5 - √ √ - 3.0 7.6 83.4 - √ - √ 2.8 7.7 83.7 - - √ √ 2.4 7.5 83.6 √ √ √ - 2.7 7 83.6 √ √ - √ 2.5 6.9 84.0 √ - √ √ 2.0 6.7 83.7 - √ √ √ 2.7 7.7 83.8 √ √ √ √ 2.4 6.9 84.1

[0104] The experimental results show that the number of parameters in the original YOLOv11-pose model after adding the LSCP module is...

[0105] GFLOPs decreased by 14% and 10% respectively, while AP50 only decreased slightly by 0.3 percentage points; after introducing the LSKA attention mechanism and PSConv, the number of parameters and GFLOPs increased slightly, and AP50 increased by 1.1 percentage points; after adding the EPMF module, the PLES-YOLO model reduced the number of parameters by 17% compared with the original YOLOv11-pose model, increased AP50 by 1.3 percentage points, and reduced GFLOPs by 7%.

[0106] To evaluate the performance of the improved model, a visual comparative analysis of the detection results was conducted. The comparison of the actual detection results is shown in the figure below. Figure 8 and 9 As shown. From Figure 8 As can be seen, in limb occlusion scenarios, the improved PLES-YOLO model exhibits superior keypoint detection accuracy compared to the original YOLOv11-Pose, effectively inferring the location of occluded parts, with a confidence level decrease of only 0.01. In small object detection scenarios, PLES-YOLO also significantly outperforms the baseline model, with confidence levels reaching 0.83, 0.81, and 0.89 respectively, further validating its robustness and accuracy in small object detection scenarios. Through the above analysis, the PLES-YOLO model proposed in this invention is more lightweight, more accurate, and performs better in handling complex scenes than the original YOLOv11-Pose model.

[0107] This invention proposes a lightweight improved model, PLES-YOLO, based on the YOLOv11-pose model. By introducing PSConv to replace the original downsampling module, feature extraction capabilities are significantly enhanced. Simultaneously, combining the LSKA large separable kernel attention module with the SPPF module effectively improves the model's performance when handling complex poses. Furthermore, a novel EPMF module is proposed. This module, through progressive multi-scale feature fusion and the application of DWConv, not only achieves model lightweighting but also integrates deep semantic information while preserving shallow details. Residual connections enhance the integrity of the information flow, thus more effectively handling limb occlusion problems. In the design of the detection head, this research innovatively proposes a lightweight shared convolutional pose detection head, LSCP, further reducing the number of model parameters and floating-point operations. Experimental results on the COCO-Pose2017 and MPII datasets show that, compared to the original YOLOv11-pose model, the PLES-YOLO model maintains its lightweight characteristics, is more suitable for devices with limited computing power, and improves accuracy in human pose estimation tasks, especially in complex environments such as human occlusion.

[0108] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A lightweight human pose estimation method based on PLES-YOLO, characterized in that, Includes the following steps: An improved downsampling module is used to extract features from the input image to obtain an initial feature map; Based on the initial feature map, a multi-scale feature fusion module is used to perform progressive feature enhancement to obtain multi-scale fused features; Based on the multi-scale fusion features, a spatial pyramid pooling module combined with an attention mechanism is used for feature optimization to obtain weighted multi-scale features. Based on the weighted multi-scale features, a lightweight shared convolutional detector head is used to predict the coordinates of human key points and output the final pose estimation result. The improved downsampling module adopts a windmill-shaped asymmetric convolution structure; The process of extracting features from the input image using an improved downsampling module to obtain an initial feature map includes: extracting multi-directional branch tensors using a windmill-shaped asymmetric convolutional structure with separate convolutional kernels in the horizontal and vertical directions; concatenating the multi-directional branch tensors to obtain a concatenated tensor; performing a convolution operation on the concatenated tensor with a 2x2 kernel to obtain concatenated features; normalizing the concatenated features to obtain an output tensor; and obtaining the initial feature map based on the output tensor. The output tensor is: In the formula, h', w', and c' represent the height, width, and number of channels of the output tensor of each branch, respectively; X' represents the concatenated tensor; and Y represents the output tensor. This represents a 2×2 convolution kernel with output channels c2, ⊗ is the convolution operator, h2, w2 and c2 represent the height, width and channel size of the input tensor X(h2,w2,c2) respectively, and SiLU represents the activation function; The multi-scale feature fusion module is a Progressive Multi-Scale Feature Extraction (EPMF) module. The process of progressively enhancing features using the multi-scale feature fusion module to obtain multi-scale fused features includes: performing preliminary processing on the initial feature map using a 3×3 depthwise separable convolution to obtain first-level features; uniformly splitting the first-level features into two parts according to the channel dimension, retaining the first part; processing the first part of the features using a 5×5 depthwise separable convolution to obtain second-level features; uniformly splitting the second-level features again according to the channel dimension, retaining the second part; processing the second part of the features using a 7×7 depthwise separable convolution to obtain third-level features; concatenating and integrating the first-level features, second-level features, and third-level features; adding the input features of the initial feature map to the concatenated and integrated features through residual connections to obtain multi-scale fused features; the spatial pyramid pooling module combined with the attention mechanism is the SPPF_LSKA module. The process of using a spatial pyramid pooling module incorporating an attention mechanism for feature optimization to obtain weighted multi-scale features includes: Multi-level pooling features of different scales are generated through spatial pyramid pooling; A large separable kernel attention LSKA mechanism is used to process the pooling features at each level to obtain attention-weighted multi-level pooling features. The LSKA mechanism decomposes a two-dimensional large convolution kernel by cascading horizontal and vertical one-dimensional kernels. Attention-weighted multi-level pooling features are fused to obtain weighted multi-scale features.

2. The lightweight human pose estimation method based on PLES-YOLO according to claim 1, characterized in that, The lightweight shared convolutional detection head is an LSCP module; The process of predicting human keypoint coordinates using a lightweight shared convolutional detector head includes: Common features are extracted through a core feature extraction layer composed of shared 3×3 convolutions and 1×1 convolutions; The bounding box prediction branch and key point prediction branch are applied to the general features respectively to obtain the prediction results of the human body key point coordinates.

3. A computer terminal device, characterized in that, include: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-2.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-2.

Images