Lightweight human body posture estimation method based on shared convolution and layered receptive field

Abstract

The invention belongs to the technical field of computer vision, and particularly relates to a lightweight human body posture estimation method based on shared convolution and layered receptive field, the method is realized based on an improved YOLO11n-pose network architecture, and the method comprises a backbone network, a neck network and a head network which are connected in sequence; wherein the backbone network comprises a first standard convolutional layer, a plurality of feature processing module groups, a fast spatial pyramid pooling module and a cross-stage partial space attention module which are connected in sequence; each feature processing module group is formed by sequentially connecting a second standard convolution layer and a hole re-parameterization module based on a layered receptive field; the neck network comprises a network structure for up-sampling and feature fusion; the head network comprises a lightweight detection head; the problems that an existing human body posture estimation method is difficult to deploy on edge equipment and the precision is reduced due to non-rigid deformation of a human body are solved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a lightweight human pose estimation method based on shared convolution and hierarchical receptive fields. Background Technology

[0002] Human pose estimation, a core task in computer vision, demonstrates broad application value in numerous fields such as intelligent monitoring, human-computer interaction, and sports rehabilitation by accurately perceiving the spatial position and motion state of key human body parts. For example, in intelligent monitoring scenarios, real-time warnings of abnormal behavior can be achieved based on human pose analysis; in human-computer interaction scenarios, it can provide users with natural and smooth body interaction methods; and in sports rehabilitation scenarios, pose quantification analysis can provide data support for optimizing training programs. However, in practical applications, a large number of human pose estimation tasks need to be completed on edge devices. Edge devices, however, are often significantly limited in terms of computing power and storage space due to factors such as hardware cost, physical size, and energy consumption control. Existing mainstream pose estimation algorithms, in pursuit of detection accuracy, generally employ complex network structures and contain a large number of parameters, resulting in extremely high computational overhead and storage requirements during operation. This makes them difficult to adapt to the hardware constraints of edge devices, severely hindering the large-scale promotion and application of human pose estimation technology in edge computing scenarios. Based on this, developing a lightweight human pose estimation algorithm suitable for scenarios with limited computing resources, while ensuring that the pose estimation accuracy meets actual needs, and significantly reducing the consumption of computing and storage resources by the algorithm, has become a key technical problem that urgently needs to be solved in this field. Summary of the Invention

[0003] To address the problems of existing human pose estimation methods being difficult to deploy on edge devices and having reduced accuracy due to non-rigid deformation of the human body, this invention provides a lightweight human pose estimation method based on shared convolution and hierarchical receptive fields.

[0004] This invention is achieved using the following technical solution: A lightweight human pose estimation method based on shared convolution and hierarchical receptive fields is proposed. The method is implemented based on an improved YOLO11n-pose network architecture, which includes a backbone network, a neck network, and a head network connected in sequence. The backbone network includes a first standard convolutional layer, multiple feature processing module groups, a fast spatial pyramid pooling module, and a cross-stage partial spatial attention module connected in sequence; each feature processing module group consists of a second standard convolutional layer and a dilated reparameterization module based on hierarchical receptive fields connected in sequence. The neck network includes a network structure for upsampling and feature fusion; The head network includes a lightweight detection head, which is a structural improvement on the original YOLO11n-pose network architecture detection head; The method includes the following steps: S1: Obtain the input image to be detected and input it into the backbone network; S2: The input image is preliminarily feature extracted through the first standard convolutional layer to obtain an initial feature map; S3: The initial feature map is processed sequentially through the multiple feature processing module groups to obtain the first multi-scale feature map; wherein, within each feature processing module group, the input features are first transformed through the second standard convolutional layer, and then feature information at different scales is extracted through the dilated reparameterization module. S4: The fast spatial pyramid pooling module performs pooling operations of different scales in parallel on the first multi-scale feature map and splices and fuses the results to extract multi-scale spatial features and obtain the second multi-scale feature map. S5: The second multi-scale feature map is processed by the cross-stage partial spatial attention module. Key information is focused and feature enhancement is performed through the attention mechanism to obtain an optimized feature map. S6: Input the optimized feature map output by the backbone network into the neck network, and perform multi-scale feature fusion through its internal upsampling and feature fusion network structure to obtain a fused feature map; S7: Input the fused feature map output by the neck network into the lightweight detection head of the head network for processing, and output the final human pose estimation result.

[0005] Furthermore, the cavity reparameterization module is constructed in the following manner: First, the sub-modules in the expanded residual module that are used to extract medium- and large-scale feature information are replaced with efficient feature extraction modules to obtain the replaced expanded residual module; Secondly, the bottleneck module in the dual convolutional cross-stage module is replaced with the replaced expanded residual module, thereby forming the holed reparameterization module. The submodule used to extract medium- and large-scale feature information has convolution kernel sizes of 5 and 7, respectively.

[0006] Furthermore, in step S4, the fast spatial pyramid pooling module specifically performs the following operations: At least two different scale pooling operations are performed in parallel on the input first multi-scale feature map, while preserving the original features; The pooling results at different scales are concatenated with the original features along the channel dimension; The concatenated features are compressed and fused using 1×1 convolutions to output the second multi-scale feature map.

[0007] Furthermore, in step S5, the cross-stage partial spatial attention module specifically performs the following operations: The input second multi-scale feature map is divided into a first branch and a second branch; The first branch is either mapped using an identity or passed directly. The second branch is processed sequentially through a partial spatial attention mechanism and multiple convolutional blocks; The output features of the first branch and the output features of the second branch are fused together, and the optimized feature map is output through a convolutional layer.

[0008] Furthermore, in step S6, the network structure for upsampling and feature fusion achieves multi-scale feature fusion through the following operations: Receive optimized feature maps from the backbone network output; High-level semantic features are fused with low-level detailed features through a top-down upsampling path; The semantic information of low-level features is enhanced by a bottom-up path aggregation network. The fused features are adjusted in dimension and optimized in quality by convolutional layers to output the fused feature map.

[0009] Furthermore, the structural improvement of the lightweight detection head lies in modifying the initial convolution processing path, which was independent of the localization branch and the classification branch in the original YOLO11n-pose network architecture detection head, into a shared convolution processing path that shares the same set of convolution kernel parameters.

[0010] Furthermore, the improvements to the lightweight detection head also include: introducing a scaling layer before the shared convolutional processing path to scale and adapt input features of different sizes; and using a grouping normalization layer instead of a batch normalization layer for feature normalization processing in the shared convolutional processing path.

[0011] This invention provides a lightweight human pose estimation method based on shared convolution and hierarchical receptive fields, which has the following advantages compared with existing technologies: Firstly, the hierarchical receptive field-based dilated reparameterization module proposed in this invention significantly enhances the ability to focus on small-scale information channels, strengthening the extraction and expression of detailed features. This accurately solves the bias problem in fine-grained pose estimation, while effectively compensating for the accuracy loss caused by the lightweight design of the model. An efficient feature extraction module is introduced to replace the dilated convolution structure in the original dilated residual module. Based on the principle of equivalent transformation, a single large-kernel convolution operation is decomposed into a combination of multiple sparse large-kernel convolution operations with equivalent receptive fields. This design not only significantly reduces the number of model parameters and computational complexity while maintaining the feature extraction capability, but also enhances the ability to capture local detailed features through the sparse distribution of convolution kernels, further improving the pose estimation accuracy of small-scale human body parts, thus achieving a better balance between lightweight design and high accuracy.

[0012] Secondly, this invention employs a shared convolution strategy to design a lightweight detection head, allowing the localization and classification branches to share core convolution parameters. This significantly reduces the number of parameters and computational overhead, avoiding redundancy while ensuring efficient feature extraction. A grouping normalization layer is introduced to replace the batch normalization layer. This reduces dependence on batch size, ensuring stable operation in small-batch scenarios, reducing feature distribution shifts, improving feature discrimination capabilities, and simultaneously optimizing localization and classification performance, thus balancing lightweight design with detection effectiveness. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0014] Figure 2 A schematic diagram of the DRB module of the present invention.

[0015] Figure 3 A schematic diagram of the DWR module of the present invention.

[0016] Figure 4 A schematic diagram of the C3DR module of this invention.

[0017] Figure 5 A schematic diagram of the CSCD module of the present invention. Detailed Implementation

[0018] The present invention will be further explained and described below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example

[0019] A lightweight human pose estimation method based on shared convolution and hierarchical receptive fields, as shown in the appendix. Figure 1As shown, the method is based on an improved YOLO11n-pose network architecture, which includes a backbone network, a neck network, and a head network connected in sequence.

[0020] The Backbone comprises a first standard convolutional layer, multiple feature processing module groups, a fast spatial pyramid pooling module (SPPF module), and a cross-stage partial spatial attention module (C2PSA module) connected in sequence. Each feature processing module group consists of a second standard convolutional layer and a hierarchical receptive field-based dilated reparameterization module (C3DR module) connected in sequence.

[0021] The C3DR module is constructed in the following manner: First, the sub-modules in the expanded residual module (DWR module) used to extract medium- and large-scale feature information are replaced with the efficient feature extraction module (DRB module) to obtain the replaced DWR module; Secondly, the bottleneck module in the dual convolutional cross-stage module (C3K2 module) is replaced with the replaced DWR module to form the C3DR module; The submodule used to extract medium- and large-scale feature information has convolution kernel sizes of 5 and 7, respectively.

[0022] Based on the principle of equivalent transformation, the design decomposes a single large kernel convolution operation into a combination of multiple sparse large kernel convolutions with equivalent receptive fields. This can significantly reduce the number of model parameters and computational complexity without compromising feature extraction capabilities. At the same time, the sparse distribution of convolution kernels enhances the ability to capture local detailed features, effectively solving the problem of fine-grained estimation bias caused by non-rigid deformation of human joints, and compensating for the accuracy loss caused by model lightweighting.

[0023] The Neck includes a network structure for upsampling and feature fusion.

[0024] The Head includes a lightweight detection head (CSCD detection head), which is a structural improvement on the original YOLO11n-pose network architecture detection head.

[0025] The structural improvement of the CSCD detection head is that the initial convolution processing path, which was independent for the localization branch and the classification branch in the original YOLO11n-pose network architecture detection head, is modified into a shared convolution processing path that shares the same set of convolution kernel parameters, thereby significantly reducing the number of parameters and computational overhead.

[0026] The improvements to the CSCD detection head also include: introducing a scale layer before the shared convolutional processing path to scale and adapt input features of different sizes; and using a group normalization layer (GN layer) instead of a batch normalization layer (BN layer) for feature normalization processing in the shared convolutional processing path.

[0027] The improvement of the normalization layer mentioned above is based on the following design considerations: In a shared convolutional structure, if the Batch Normalization (BN) layer is continued to be used, its normalization effect will depend on the statistics (mean and variance) of the current training batch, which is prone to instability in the small-batch or noisy data scenarios common on edge devices. The Normalization (GN) layer, by performing internal normalization through channel grouping, eliminates the dependence on batch size, thereby significantly improving the robustness and performance of the model during small-batch training and inference.

[0028] Furthermore, in the specific implementation of the pose estimation task, the number of channels in the final output of the detection head is closely related to the number of human keypoints (e.g., detecting 17 keypoints typically corresponds to 51 output channels). When using a GN layer, its group number (G) needs to be divisible by the number of input channels (C). If this condition cannot be met (e.g., 51 channels are difficult to divisible by the commonly used group number), then standard convolution operations are retained in that specific convolutional layer instead of using the GN layer to ensure the correct operation of the network. In this design, the shared convolutional path satisfies the condition for using the GN layer, thus successfully replacing the BN layer with the GN layer, reducing the number of parameters while enhancing the localization and classification stability of the detection head in complex scenes.

[0029] The method includes the following steps: S1: Obtain the input image to be detected and input it into the Backbone.

[0030] S2: The input image is initially extracted through the first standard convolutional layer to obtain an initial feature map.

[0031] S3: The initial feature map is processed sequentially through the multiple feature processing module groups to obtain the first multi-scale feature map. Within each feature processing module group, the input features are first transformed by the second standard convolutional layer, and then the C3DR module extracts feature information at different scales. The C3DR module, through its internal branches with convolutional kernels of different sizes (5 and 7), focuses on and fuses features in medium and large-scale receptive fields, thereby enhancing the ability to extract and express features of different scale parts of the human body.

[0032] S4: Process the first multi-scale feature map using the SPPF module to extract richer spatial context information. Specifically, perform the following operations: At least two different scale pooling operations are performed in parallel on the input first multi-scale feature map, while preserving the original features; The pooling results at different scales are concatenated with the original features along the channel dimension; The concatenated features are compressed and fused using 1×1 convolutions to output the second multi-scale feature map. This process achieves efficient multi-scale spatial feature fusion.

[0033] S5: The second multi-scale feature map is processed by the C2PSA module to focus key information and enhance feature quality. Specifically, the following operations are performed: The input second multi-scale feature map is divided into a first branch and a second branch; The first branch is either mapped using an identity or passed directly. The second branch is processed sequentially through a partial spatial attention (PSA) mechanism and multiple convolutional blocks, where the PSA mechanism helps the model focus on important spatial regions; The output features of the first branch and the output features of the second branch are fused together, and the optimized feature map is output through a convolutional layer, thereby achieving feature enhancement and key information focusing.

[0034] S6: Input the optimized feature map output by the Backbone into the Neck, and perform multi-scale feature fusion through its internal upsampling and feature fusion network structure to obtain a fused feature map.

[0035] Specifically, the network structure performs the following operations: The system receives optimized feature maps from the backbone network output; it then fuses high-level semantic features with low-level detail features through a top-down upsampling path; simultaneously, it combines a bottom-up path to aggregate the network and enhance the semantic information of low-level features; finally, it performs dimensionality adjustment and quality optimization on the fused features through convolutional layers and outputs the fused feature map, thereby improving the network's ability to detect targets of different sizes.

[0036] S7: Input the fused feature map output by the Neck into the CSCD detection head of the Head for processing, and output the final human pose estimation result.

[0037] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A lightweight human pose estimation method based on shared convolution and hierarchical receptive fields, characterized by: The method is based on an improved YOLO11n-pose network architecture, which includes a backbone network, a neck network, and a head network connected in sequence. The backbone network includes a first standard convolutional layer, multiple feature processing module groups, a fast spatial pyramid pooling module, and a cross-stage partial spatial attention module connected in sequence; each feature processing module group consists of a second standard convolutional layer and a dilated reparameterization module based on hierarchical receptive fields connected in sequence. The neck network includes a network structure for upsampling and feature fusion; The head network includes a lightweight detection head, which is a structural improvement on the original YOLO11n-pose network architecture detection head; The method includes the following steps: S1: Obtain the input image to be detected and input it into the backbone network; S2: The input image is preliminarily feature extracted through the first standard convolutional layer to obtain an initial feature map; S3: The initial feature map is processed sequentially through the multiple feature processing module groups to obtain the first multi-scale feature map; wherein, within each feature processing module group, the input features are first transformed through the second standard convolutional layer, and then feature information at different scales is extracted through the dilated reparameterization module. S4: The fast spatial pyramid pooling module performs pooling operations of different scales in parallel on the first multi-scale feature map and splices and fuses the results to extract multi-scale spatial features and obtain the second multi-scale feature map. S5: The second multi-scale feature map is processed by the cross-stage partial spatial attention module. Key information is focused and feature enhancement is performed through the attention mechanism to obtain an optimized feature map. S6: Input the optimized feature map output by the backbone network into the neck network, and perform multi-scale feature fusion through its internal upsampling and feature fusion network structure to obtain a fused feature map; S7: Input the fused feature map output by the neck network into the lightweight detection head of the head network for processing, and output the final human pose estimation result.

2. The lightweight human pose estimation method based on shared convolution and hierarchical receptive fields according to claim 1, characterized in that: The cavity reparameterization module is constructed in the following manner: First, the sub-modules in the expanded residual module that are used to extract medium- and large-scale feature information are replaced with efficient feature extraction modules to obtain the replaced expanded residual module; Secondly, the bottleneck module in the dual convolutional cross-stage module is replaced with the replaced expanded residual module, thereby forming the holed reparameterization module. The submodule used to extract medium- and large-scale feature information has convolution kernel sizes of 5 and 7, respectively.

3. The lightweight human pose estimation method based on shared convolution and hierarchical receptive fields according to claim 1, characterized in that: In step S4, the fast spatial pyramid pooling module specifically performs the following operations: At least two different scale pooling operations are performed in parallel on the input first multi-scale feature map, while preserving the original features; The pooling results at different scales are concatenated with the original features along the channel dimension; The concatenated features are compressed and fused using 1×1 convolutions to output the second multi-scale feature map.

4. The lightweight human pose estimation method based on shared convolution and hierarchical receptive fields according to claim 1, characterized in that: In step S5, the cross-stage spatial attention module specifically performs the following operations: The input second multi-scale feature map is divided into a first branch and a second branch; The first branch is either mapped using an identity or passed directly. The second branch is processed sequentially through a partial spatial attention mechanism and multiple convolutional blocks; The output features of the first branch and the output features of the second branch are fused together, and the optimized feature map is output through a convolutional layer.

5. The lightweight human pose estimation method based on shared convolution and hierarchical receptive fields according to claim 1, characterized in that: In step S6, the network structure for upsampling and feature fusion achieves multi-scale feature fusion through the following operations: Receive optimized feature maps from the backbone network output; High-level semantic features are fused with low-level detailed features through a top-down upsampling path; The semantic information of low-level features is enhanced by a bottom-up path aggregation network. The fused features are adjusted in dimension and optimized in quality by convolutional layers to output the fused feature map.

6. The lightweight human pose estimation method based on shared convolution and hierarchical receptive fields according to claim 1, characterized in that: The structural improvement of the lightweight detection head lies in modifying the initial convolution processing path, which was independent of the localization branch and the classification branch in the original YOLO11n-pose network architecture detection head, into a shared convolution processing path that shares the same set of convolution kernel parameters.

7. The lightweight human pose estimation method based on shared convolution and hierarchical receptive fields according to claim 6, characterized in that: The improvements to the lightweight detection head also include: introducing a scaling layer before the shared convolutional processing path to scale and adapt input features of different sizes; and using a grouping normalization layer instead of a batch normalization layer for feature normalization processing in the shared convolutional processing path.

Images