Lightweight attitude estimation method based on dynamic resolution and structural re-parameterization

Abstract

The invention provides a lightweight attitude estimation method based on dynamic resolution and structural re-parameterization. The lightweight attitude estimation method comprises the following steps: step 1, recognizing a human body core area, accurately quantifying a motion blur degree, dynamically correcting a decision threshold by combining a human body proportion, adaptively selecting resolution based on multi-dimensional parameters, and outputting frame data of adaptive resolution; 2, constructing a backbone network adaptive to attitude estimation, completing feature extraction, sub-network connection and reasoning optimization, and outputting a refined feature map; 3, based on the refined feature map output in the step 2, joint point heat map prediction, position positioning and result optimization are completed through a multi-stage posture estimation sub-network, and finally a human body posture estimation result matched with an original input frame is generated. According to the method, key joint points of the human body can be accurately detected and positioned, limb structures are reasonably connected, the reasoning speed can be remarkably increased, resource consumption is reduced, and the method can be successfully adapted to low-computing-power scenes such as embedded equipment and mobile terminals.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to a lightweight pose estimation method based on dynamic resolution and structural reparameterization. Background Technology

[0002] Human pose estimation, a core task in computer vision, aims to detect and locate key human joints from images or videos, and logically connect these joints based on human physiological structure to present a complete human pose. It plays a crucial role in real-time applications such as intelligent video surveillance, human-computer interaction, motion-sensing games, and security systems. The classic model OpenPose achieves real-time multi-person pose detection through a multi-stage prediction framework and partial affinity fields (PAFs). Its technical process includes extracting multi-scale features through a VGG19 backbone network to generate a basic feature map. This feature map is processed by a multi-stage dual-branch prediction module, outputting a confidence map representing the location of key points and partial affinity fields encoding the degree of association between joints. Finally, greedy reasoning is used to parse and obtain the two-dimensional key points and limb connection information of all individuals. However, the VGG-19 backbone network used by OpenPose has inherent drawbacks such as high computational redundancy and high memory consumption, severely limiting its practical deployment in resource-constrained environments such as mobile terminals and embedded devices.

[0003] To address the aforementioned deployment bottlenecks, various lightweight improvement schemes have emerged in existing technologies. These mainly include replacing the original backbone network with a lightweight network structure. For example, SqueezeNet uses a modular design called "Fire Module" to reduce the number of parameters; the MobileNet series uses depthwise separable convolutions to reduce computational complexity; ShuffleNet enhances cross-group information exchange through group convolutions and channel shuffling operations; and GhostNet expands features through inexpensive linear operations to reduce computational costs. In addition, feature reuse techniques such as DenseNet enhance feature transfer efficiency through cross-layer connections, thereby compressing the parameter scale. While these methods have achieved model lightweighting to some extent, they still have significant limitations: SqueezeNet's model structure design leads to decreased parallelism and poor real-time performance; MobileNetV1's single-path direct connection structure constrains feature representation capabilities, while MobileNetV2 introduces additional computational costs and latency; ShuffleNet's multi-branch structure increases memory access costs; linear transformations in GhostNet may introduce significant computational latency in embedded devices; and while feature reuse techniques reduce computational overhead, they are prone to losing spatial detail information, which is detrimental to accurate localization. Therefore, existing methods generally struggle to achieve an effective balance between detection accuracy, computational efficiency, and memory usage, failing to fully meet the needs of real-time attitude estimation in low-computing-power scenarios. Summary of the Invention

[0004] Purpose of the invention: The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing a lightweight pose estimation method based on dynamic resolution and structural reparameterization. The dynamic resolution adaptation module balances computational efficiency and detection accuracy, while the structural reparameterization backbone network preserves spatial details and improves inference speed, providing high-quality feature input for subsequent pose estimation subnetworks, thus meeting the application requirements of real-time deployment (such as pedestrian pose detection in surveillance videos and human pose recognition in autonomous driving scenarios).

[0005] Define the following terms:

[0006] The object refers to the input data, module components, network structure units, or parameter indicators corresponding to the human pose estimation system. Specifically, it includes:

[0007] 1) Input data: Original video frames (such as RGB format frames, grayscale frames), and video frames with adjusted resolution (high resolution frames, low resolution frames).

[0008] 2) Module components: Dynamic Resolution Adaptation Module (DRA module), Motion Blur Evaluation Network, Resolution Decision Mechanism, CPM (Convolutional Pose Machine) module;

[0009] 3) Network structure units: RepVGG-B1 architecture stage0~stage4, RepVGGBlock (including multi-branch structure during training and single-path structure during inference), depthwise separable convolution (including depthwise convolution and pointwise convolution);

[0010] 4) Parameter indicators: fuzziness scalar (range 0~1), resolution size (e.g. 384×288, 192×144), feature map dimension (e.g. 224×224×3, 28×28×128), loss function parameters (smoothing constraint coefficient λ).

[0011] Dynamic Resolution Adaptation (DRA) module: As the input preprocessing module of the model, it dynamically adjusts the input frame resolution according to scene characteristics (blurriness, human body proportion) through the linkage mechanism of "human body region extraction, motion blur assessment, dynamic threshold correction, and multi-level resolution decision". This enables high-blurriness frames to use high resolution to maintain accuracy and low-blurriness frames to use low resolution to reduce redundancy.

[0012] Reparameterized backbone network: Based on RepVGG-B1, it is reconstructed in stages (preserving stage0-stage2), post-processed for features (channel adaptation, texture refinement) and optimized for inference (multi-branch to single-path) to meet the feature requirements of human pose estimation for "high semantics + spatial details" while satisfying the network structure for deployment in low computing power scenarios.

[0013] RepVGGBlock: The basic building block of the RepVGG architecture. During training, it contains a multi-branch structure with 3×3 convolutions, 1×1 convolutions, and identity mappings (enhancing representation capabilities). During inference, it is reparameterized into a single-path 3×3 convolution (improving computational efficiency).

[0014] The method of the present invention includes the following steps:

[0015] Step 1, Dynamic resolution adaptation processing: By identifying the core human body region, accurately quantifying the degree of motion blur, and dynamically correcting the decision threshold based on the proportion of the human body, the resolution is adaptively selected based on multi-dimensional parameters, and frame data with the adapted resolution is output to provide optimized input for the subsequent backbone network;

[0016] Step 2, Reparameterization of the backbone network: Construct a backbone network adapted for pose estimation, complete feature extraction, sub-network connection and inference optimization, and output a refined feature map (typically 28×28×128 in size).

[0017] Step 3, Generate pose estimation results: Based on the refined feature map output in Step 2, the multi-stage pose estimation sub-network completes joint heatmap prediction, location localization and result optimization, and finally generates human pose estimation results that match the original input frame.

[0018] Step 1 includes:

[0019] Step 1.1, Human body region recognition: By detecting human body regions and calculating the proportion of human body, the core human body region is extracted and the background is weakened;

[0020] Step 1.2, Motion blur assessment: Extract blur features of the human body region through a lightweight three-level convolutional network and jointly optimize the training to achieve end-to-end blur prediction and quantify the degree of motion blur of the human body region in the original frame.

[0021] Step 1.3, Resolution dynamic decision-making mechanism execution: Based on ambiguity scalar, human body proportion and threshold, adaptive resolution selection;

[0022] Step 1.4, Resolution Scaling: Use bilinear interpolation to achieve frame size conversion to adapt to the resolution.

[0023] Step 1.1 includes:

[0024] Step 1.1.1, Human Region Extraction and Background Weakening: The lightweight object detection network YOLO-Nano, with only 1.9MB of parameters, is used to detect human bodies in the original video frames, locating the bounding boxes of all human bodies within the frames. The detected human bounding boxes are then cropped, and the pure human region image is extracted as input for subsequent blur assessment. The background region outside the human bounding boxes is weakened by downsampling at a 1 / 8 ratio, eliminating background blur interference and reducing redundant computation in subsequent feature extraction stages.

[0025] Step 1.1.2, Human body proportion calculation: Calculate the human body proportion ρ, which is the ratio of the total area of ​​all human body bounding boxes to the total area of ​​the original video frame, ρ∈[0,1], which is used for subsequent resolution decision threshold correction.

[0026] Step 1.2 includes:

[0027] Step 1.2.1, configure the lightweight three-level convolutional network structure: The lightweight three-level convolutional network uses the cropped human body region as the processing object, avoiding background blur interference and ensuring that the output score directly reflects the blur level of the region where the joint is located. The overall three-level progressive structure achieves a balance between lightweight computation and evaluation accuracy.

[0028] The first stage is a standard 3×3 convolutional layer, which extracts shallow features. Its kernel size is 3×3, which matches the scale of local features at the edges of human joints, effectively capturing shallow textures related to blur, such as joint contours and motion blur. The stride is 2, which, through 2x spatial downsampling, compresses the spatial dimension of subsequent calculations to reduce computational cost while selectively preserving high-frequency information of motion blur, avoiding oversampling that could lead to the loss of key features. The output has 16 channels, fully covering different types of shallow blur features within the human body region. Edge padding is 1, ensuring that blur features at human edge joints are not truncated during convolution, avoiding blur assessment bias caused by missing edge information.

[0029] The second level is a depthwise separable convolutional layer, used to deepen fuzzy features. It consists of two parts: depthwise convolution and pointwise convolution. The depthwise convolution kernel size is 3×3, and the number of input channels is consistent with the number of output channels of the first level (i.e., 16 channels). Each channel is independently configured with one convolution kernel. This design of independent processing of each channel can focus on the local fuzzy features of a single joint corresponding to each channel, avoiding mutual interference between fuzzy features of different joints, and achieving accurate extraction of local fuzzy information. The pointwise convolution kernel size is 1×1, and the number of output channels is set to twice the number of input channels (i.e., 32 channels). The 1×1 convolution kernel can realize cross-channel feature fusion, associating and integrating the local fuzzy features of the 16 channels extracted by the depthwise convolution, mining the correlation of fuzzy features of different joints, and forming global features that reflect the fuzzy laws of the overall human body movement. At the same time, the computational cost of this structure is only 1 / 8 of that of the traditional 3×3 standard convolution, reducing network computing power consumption and adapting to the deployment requirements of low computing power scenarios.

[0030] The third stage consists of a 1×1 convolutional layer, a global average pooling layer, and a sigmoid activation layer, which completes the quantitative output of fuzziness. The 1×1 convolutional layer has 32 input channels (the same as the output of the second stage) and only 1 output channel. Single-channel compression eliminates channel dimensional redundancy and integrates the global fuzzy features of 32 channels through linear transformation, generating a single-channel fuzzy feature map. The global average pooling layer aggregates the single-channel fuzzy feature map across all spatial dimensions, calculates the average value of all pixel values, and transforms the two-dimensional spatial features into a one-dimensional scalar. This process effectively eliminates the interference of local pixel noise on fuzziness evaluation, ensuring that the output scalar accurately reflects the overall fuzziness of the human body region. The sigmoid activation layer constrains the one-dimensional scalar within the range of 0 to 1 through nonlinear mapping, ultimately outputting a fuzziness scalar. The closer the s value is to 1, the higher the motion fuzziness of the human body joint region; the closer it is to 0, the clearer the region. This output range matches the subjective perception of "fuzziness level," providing a precise and quantifiable basis for threshold comparison and scene classification in the subsequent resolution decision mechanism.

[0031] Step 1.2.2, Feature Extraction and Blur Calculation: Using the cropped human body region image (H×W×3) as input, feature extraction and fuzziness quantification are completed sequentially through a three-level network structure. The specific process includes:

[0032] Step 1.2.2.1: The input human body region image (H×W×3) is fed into a 3×3 standard convolutional layer. The 3×3 standard convolutional layer, through a convolution operation with a stride of 2, compresses the spatial resolution of the input image to half its original value, while simultaneously extracting the edge textures of the human body contour and key joints, outputting a shallow feature map. (H1×W1×16).

[0033] Step 1.2.2.2: Feature map output after a 3×3 standard convolutional layer The process then proceeds to the depthwise separable convolutional layer. This layer employs two steps—depthwise convolution and pointwise convolution—to further capture motion blur features and correlate the blurry relationships between different joints. Specifically, this includes:

[0034] (1) Depthwise convolution calculation: Its core is to perform independent convolution on each channel of the input feature map without cross-channel information fusion. The calculation formula is as follows:

[0035] ,

[0036] in, The input feature map for depthwise convolution has dimensions H1×W1×16; It is a special operation symbol for depthwise convolution, which performs convolution operations only on a 3×3 local region within a single channel and does not involve information exchange between different channels; The kernel is a depthwise convolution kernel, with one 3×3 kernel corresponding to each channel. Its function is to independently extract the local blurred features of joints within a single channel. It is the bias term of the depthwise convolution, with a dimension of 16. Each channel corresponds to one bias, which is used to adjust the numerical range of the single-channel convolution output to avoid the feature value from shifting. It is the output feature map of the depthwise convolution, with dimensions still H1×W1×16. It can completely preserve the independent joint local blur information of each channel and there is no cross-channel interference. h, w, and c represent the height, width, and number of channels of the output feature map of the depthwise convolution, respectively.

[0037] (2) Pointwise convolution calculation: Pointwise convolution is calculated using... (H1×W1×16) is the input, and cross-channel fusion is achieved through a 1×1 convolution kernel to associate the fuzzy correlations of different joints. The calculation formula is as follows: ,

[0038] in, It is the input feature map of pointwise convolution, that is, the output of depthwise convolution. , For input channel index; It is the symbol for pointwise convolution, which means that cross-channel information fusion is achieved through a 1×1 convolution kernel without changing the spatial resolution of the feature map; It is a pointwise convolution kernel with dimensions 1×1×16×32, where As the output channel index, the number of output channels is set to twice the number of input channels. Through the mapping relationship between 16 input channels and 32 output channels, the fuzzy features of different joints can be associated. It is the bias term of pointwise convolution, with a dimension of 32. Each output channel corresponds to one bias, which is used to adjust the numerical range after cross-channel fusion. It is the output feature map of pointwise convolution with dimensions H1×W1×32. It has integrated fuzzy association information across joints and can provide global fuzzy features for subsequent fuzzy metric resolution.

[0039] Step 1.2.2.3: After processing with depthwise separable convolutional layers, the feature map has dimensions H1×W1×32. The 32-dimensional channels are compressed to 1 dimension using a 1×1 convolutional kernel to eliminate channel redundancy and reduce computational complexity, outputting a single-channel blurred feature map. (Dimensions H1×W1×1), where the pixel value at each spatial location (h,w) corresponds to the fuzzy feature intensity at that location. Here, H1 and W1 represent the height and width of the feature map, respectively.

[0040] Then, Input a global average pooling layer, and use the formula The average of all spatial pixels in the single-channel feature map is taken to aggregate the two-dimensional spatial features into a one-dimensional scalar S. The scalar S can comprehensively reflect the overall blur level of the human body region, while eliminating the interference of local pixel noise on the evaluation results.

[0041] Finally, the scalar S is input into the Sigmoid activation layer, through the formula... The S is nonlinearly mapped to the range of 0 to 1, and the final output is the ambiguity scalar s. This quantization result provides a basis for accurate scene judgment in the subsequent resolution decision mechanism.

[0042] Step 1.2.3, Network Training Optimization: A joint optimization using the mean squared error loss function and the total variation regularization term is employed to simultaneously achieve the dual objectives of accurate ambiguity prediction and stable, unfluctuating output. The formula for this joint optimization is... for:

[0043] ,

[0044] in It is a ground truth label for ambiguity generated based on the Laplacian gradient variance. The Laplacian operator can capture the edge gradient of the image. The larger the variance, the clearer the image edge (lower ambiguity); the smaller the variance, the higher the ambiguity. This is the blur prediction value output by a lightweight three-level convolutional network. The lightweight three-level convolutional network learns the features of the input image and outputs a score of the blur level of the original video frame. This is the mean squared error loss term, where N is the sample size; for each sample, first calculate the "predicted value". with truth value The difference is calculated by first finding the mean squared error, then squaring the difference (to amplify the penalty for error), and finally summing the results for all samples and dividing by the sample size N to obtain the mean squared error. Its purpose is to force the model to learn "how to accurately predict ambiguity," making the predicted values ​​as close as possible to the true degree of ambiguity. This is the foundation for ensuring accurate dynamic resolution decision-making. λ represents the gradient (change) of the ambiguity prediction value; λ is the smoothing constraint coefficient. The total variation regularization term is obtained by summing the absolute values ​​of the gradients of the predicted values ​​of all samples and then multiplying them by λ. This term avoids the situation where the ambiguity score fluctuates greatly in the same type of scene in order to fit the noise in the training data, thus preventing overfitting and improving the model's generalization ability in new scenes.

[0045] Step 1.3 includes:

[0046] Step 1.3.1: To ensure the universality and adjustability of the decision-making logic, the following core parameters are preset:

[0047] Basic ambiguity threshold The baseline threshold used to distinguish between "high blur" and "low blur" scenes;

[0048] Proportion adjustment coefficient α: used to control the degree of influence of human body proportion on the decision threshold;

[0049] Ambiguity classification threshold: set to ,satisfy It is used to classify ambiguity into four levels: extremely high ambiguity, high ambiguity, medium ambiguity, and low ambiguity.

[0050] Human body proportion grading threshold Used to classify the proportion of the human body as "high proportion" ( ) and "low percentage" ), suitable for both small and large goal scenarios;

[0051] Multi-level resolution set: set as Each resolution is distributed in a gradient according to a preset ratio (e.g.) ),satisfy .

[0052] Step 1.3.2, Dynamically Adjust the Threshold. Since a fixed threshold is prone to misjudgment in scenarios with "small targets (low proportion)" or "large targets (high proportion)," such as small human figures requiring high resolution to preserve details even with slightly lower blur, and large human figures requiring excessive resolution leading to redundancy even with slightly higher blur, the basic threshold is adjusted based on the human figure proportion ρ. To obtain the dynamic threshold The formula is:

[0053] ,

[0054] When the proportion of human body is high It increases with increasing ρ to avoid the loss of joint details due to the misuse of low resolution when the human body has rich details; when the proportion is low, near This ensures that even if the ambiguity is slightly below τ, a higher resolution is prioritized to preserve the key point information of small targets. By adaptively adjusting the threshold based on the proportion of the human body, the scene adaptability of the resolution decision can be enhanced, small targets can be avoided being missed, and the accuracy of pose estimation in complex environments can be improved.

[0055] Step 1.3.3, Dynamic resolution selection:

[0056] A resolution decision matrix is ​​established using "ambiguity classification + human body proportion classification" as two dimensions to ensure accurate matching between resolution and scene features, specifically including:

[0057] Extremely high blur, high proportion of scenes: and Select the highest resolution. It is designed for "humans moving at close range and at high speed" (such as runners) and maximizes the preservation of joint details (such as wrist swing trajectory).

[0058] High-blur, high-percentage scenes or extremely high-blur, low-percentage scenes: and ,or and Choose the second highest resolution. Balancing "detail preservation" with "computational efficiency" (such as a human body walking briskly over a medium distance, or a human body moving rapidly over a long distance);

[0059] Medium blur, high percentage scene or high blur, low percentage scene: and ,or and Choose a medium resolution. It is designed for "slow-moving human bodies over medium distances" (such as walkers), achieving the optimal balance between accuracy and efficiency.

[0060] Low-blur, high-percentage scenes or medium-blur, low-percentage scenes: and ,or and Choose the second lowest resolution. Reduce computational redundancy (e.g., near-stationary human body, medium-range slow-moving small target human body);

[0061] Low-blur, low-percentage scenes: and Expandable selection of the lowest resolution To maximize computational efficiency (e.g., for small, stationary human targets at a distance).

[0062] Step 2 includes:

[0063] Step 2.1, Stage-based feature extraction: Reconstruct the stage structure of RepVGG-B1. To avoid irreversible loss of spatial information of joints due to depth downsampling, discard stage3~stage4 and retain stage0~stage2 to achieve hierarchical feature transformation.

[0064] Step 2.2, Connection processing with the pose estimation sub-network: Channel adaptation and texture refinement are performed on the output features of the backbone network to ensure compatibility with subsequent pose estimation sub-networks:

[0065] Step 2.3, Structure reparameterization and inference optimization: Convert the multi-branch structure during training into a single-path structure during inference.

[0066] Step 2.1 includes:

[0067] Step 2.1.1, stage0, initial feature mapping:

[0068] The feature extraction unit of stage0 consists of a 3×3 convolutional layer and a BN (batch normalization) layer. The parameters of the convolutional layer are: kernel size 3×3, stride 2, padding 1, and output channels 64. The receptive field of the 3×3 convolutional kernel is adapted to the local scale of the human body contour and joint edges, effectively extracting low-frequency texture features (such as the human torso contour and the approximate edges of the limbs). The BN layer improves the stability of subsequent feature extraction by normalizing the feature distribution.

[0069] Step 2.1.2, stage 1, feature channel preservation:

[0070] The feature extraction unit in stage 1 consists of three stacked RepVGGBlocks (RepVGG blocks, the basic network structure unit of the RepVGG architecture). Each RepVGGBlock has 64 input and output channels and a stride of 1, thus maintaining feature map resolution and preserving details. During training, RepVGGBlocks employ a multi-branch structure, including 3×3 convolutions, 1×1 convolutions, and identity mappings: 1×1 convolutions capture fine-grained features (such as local textures of small joints like fingers and wrists), identity mappings protect gradient propagation and prevent gradient vanishing, and 3×3 convolutions strengthen local feature associations. These three elements work together to improve the initial detection accuracy of joints.

[0071] The feature map resolution is maintained at 112×112×64, and the local feature association is enhanced by a multi-branch structure during training; the multi-branch structure during training includes 3×3 convolution, 1×1 convolution, and identity mapping.

[0072] Step 2.1.3, Stage 2, Semantic Feature Enhancement:

[0073] The feature extraction unit in stage 2 consists of four RepVGGBlocks. The first RepVGGBlock uses a stride of 2 for downsampling, while the other three use a stride of 1. Stage 2 increases the number of feature channels from 64 to 128 and achieves a 4-fold downsampling compared to stage 1. The large receptive field of the 3×3 convolutional main branch is fully utilized in this stage, focusing on spatial layout modeling of large-scale joints such as the torso and limbs to form global pose features, providing semantic support for the overall association of joints.

[0074] Step 2.2 includes:

[0075] Step 2.2.1, Channel dimension normalization:

[0076] By using a single 1×1 convolutional layer without Batch Normalization (BN) or activation functions, the 128-channel feature map output from stage 2 is adjusted to increase the number of channels to 512. The core functions of this operation are: first, to eliminate channel redundancy by integrating the semantic features of the 128 channels through linear transformation, reducing subsequent computational complexity; and second, to match the input channel requirements of the subsequent CPM (Convolutional Pose Machine) module, avoiding feature transfer loss due to channel incompatibility.

[0077] Step 2.2.2, Distribution Alignment and Feature Refinement:

[0078] The adapted feature map is input into the CPM (Convolutional Pose Machine) module to complete feature distribution alignment and texture refinement. First, a 1×1 convolution is used to align the feature distribution, mapping the feature values ​​of different channels to a uniform range to avoid affecting the subsequent prediction accuracy due to numerical differences. Then, three depthwise separable convolutional layers with a kernel size of 3×3 and a stride of 1 are used to further refine the feature texture, and finally output a feature map that can be seamlessly connected with the pose estimation sub-network.

[0079] Step 2.3 includes:

[0080] Step 2.3.1, Inference Mode Conversion Configuration: Call the RepVGG official library's structure reparameterization interface to define a conversion function. The input is the training state RepVGGBlock, and the output is the inference state single-path convolutional structure. Specifically, this includes the following steps:

[0081] Step 2.3.1.1, clarify the input specifications: limit the input to the training state RepVGGBlock instance, including a complete multi-branch structure: 3×3 convolution, 1×1 convolution, identity mapping, and the parameters of each branch match the corresponding network stage (e.g., stage1 channel 64, stage2 channel 128), and the output channels are consistent, laying the foundation for parameter fusion.

[0082] Step 2.3.1.2, Design the conversion logic: Based on the RepVGG official interface encapsulation logic, first verify the branch integrity and parameter dimension to eliminate abnormal input; then convert the multi-branch into a single 3×3 convolutional layer according to the RepVGG-B1 rule, and force the convolutional layer to pad with 1 and stride with 1 to ensure that the resolution of the input and output feature maps remains unchanged.

[0083] Step 2.3.1.3, verify the transformation results: First, verify the consistency of dimensions by using test features to confirm that the spatial dimension and number of channels of the output of the training state and the inference state are consistent; second, verify the integrity of parameters by checking whether the weights and biases of the convolutional layers in the inference state are complete.

[0084] Step 2.3.1.4, Batch integration of networks: Following the original training state structure order, convert RepVGGBlock of stage1 (3 layers) and stage2 (4 layers) into inference state convolutional layers one by one and replace them, keeping the forward propagation path unchanged, so as to provide a unified structural basis for subsequent parameter fusion.

[0085] Step 2.3.2: Parameter Fusion Processing. Considering the sensitivity of pose estimation tasks to the spatial location of joints and the need to adapt to complex scenes, the mathematical formulas in the RepVGG parameter fusion process are specifically improved. By introducing joint scale factors, constrained spatial transformations, and weighting mechanisms, the fused parameters are ensured to retain the feature representation capabilities of the multi-branch training state while enhancing joint localization accuracy and scene robustness. Specifically, the following steps are included:

[0086] In step 2.3.2.1, in the training state of RepVGGBlock, both the 3×3 convolutions and 1×1 convolutions are cascaded with independent BN layers. Direct fusion can easily lead to the loss of small joint features due to neglecting the differences in joint size. Therefore, a joint scale factor is introduced into the fusion formula. (Calculated based on OKS (Object Keypoint Similarity) of keypoint annotations in the COCO dataset, reflecting the size variance of different joints, such as small joints) (With a larger value), the bias after fusion is adaptively adjusted to enhance the small joint feature response. The fusion formula is modified as follows:

[0087] Convolution weights after fusion: ,

[0088] Post-fusion convolution bias: ,

[0089] in, , These are the convolutional layer weights and biases corresponding to the i-th output channel, respectively; , These are the scaling factor and bias of the BN layer corresponding to the i-th output channel, respectively, used to standardize the feature distribution; , These are the mean and variance of the BN layer corresponding to the i-th output channel, respectively, which characterize the statistical regularity of the features during the training process; It is a small constant to prevent the denominator from being zero; Here, j is the joint scale factor, and j is the joint category index, ensuring that small joint features are not suppressed after fusion. The adjustment coefficient balances the effect of the joint scale factor on the bias. , These are the fused convolution weights and biases corresponding to the i-th output channel, respectively.

[0090] Step 2.3.2.2, Branch Structure Unification: The training state RepVGGBlock contains three types of branches: 3×3 convolution, 1×1 convolution, and identity mapping. Directly converting them to 3×3 convolution can easily introduce edge artifacts or gradient vanishing, affecting the localization accuracy of the joint heatmap. Therefore, a constrained conversion strategy is designed for the two types of non-3×3 branches, as follows:

[0091] (1) For 1×1 convolution branches: constrained zero padding;

[0092] When expanding a 1×1 convolutional kernel to a 3×3 convolutional kernel, only the outer ring of the kernel is zero-padded to avoid diluting the effective features inside and reduce the interference of edge artifacts on the localization of key points. The conversion formula is as follows: ,

[0093] in, These are the weights of the original 1×1 convolution branch; For constrained zero-padding, zero values ​​are padded only on the top, bottom, left, and right sides of the 1×1 convolutional kernel to form a 3×3 convolutional kernel, while the original weight values ​​are retained in the core region (center 1×1). These are the weights of the expanded 3×3 convolution kernel.

[0094] (2) Identity mapping branch: Key point location mask enhancement

[0095] When transforming the identity mapping into a 3×3 convolution kernel, a joint position mask and a small constant are introduced to strengthen gradient protection in the joint region and improve the joint recovery capability in occluded scenes. The transformation formula is as follows:

[0096] in, The weights of the 3×3 convolution kernel after the identity mapping are transformed; (i,j) are the spatial coordinates of the convolution kernel; I is the identity matrix, with the central element being 1 and the rest initially being 0, ensuring the basic function of the identity mapping (direct transmission of input features). This is a mask for the key points, with the same spatial dimension as the convolution kernel. The mask value for the key point region is 1, which enhances the feature transfer weights in that region. It is a small constant, providing additional gradient protection for key regions in occluded scenarios to avoid feature propagation interruption caused by gradient vanishing.

[0097] Step 2.3.2.3, Multi-branch parameter fusion: Add the unified 3×3 convolution kernel weights and biases separately to generate the parameters of a single-path 3×3 convolution.

[0098] Traditional multi-branch fusion uses equal weight addition, which easily overlooks the need to balance the "global pose structure (such as torso association) and local joint accuracy (such as wrist localization)" in pose estimation. Therefore, dynamic weight coefficients (1×1 convolution branch weights) and identity mapping branch weights are introduced, and the weights are dynamically optimized through backpropagation of the pose estimation loss function. The formula is modified as follows:

[0099] Fusion convolution weights : ;

[0100] Convolutional bias after fusion : ;

[0101] in, , These are the weights and biases after the original 3×3 convolutional branches are fused together, and they are responsible for capturing global pose semantic features. , These are the weights and biases fused after expanding the 1×1 convolutional branches, responsible for capturing fine-grained joint features; , These are the weights and biases fused after the transformation of the identity mapping branch, respectively, which are responsible for protecting gradient propagation; , The dynamic weighting coefficients are optimized through backpropagation of the attitude estimation loss.

[0102] Through the above adaptive modifications, the parameter fusion process adapts to the key point localization requirements of pose estimation. While simplifying the inference structure, it ensures that key point features are not lost and scene robustness is not reduced, providing high-quality feature input for the subsequent pose estimation sub-network.

[0103] The present invention also provides an electronic device, including a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method.

[0104] The present invention also provides a storage medium storing a computer program or instructions that, when the computer program or instructions are run on a computer, execute the steps of the method described.

[0105] Against this technical background, the present invention addresses the problems of high computational redundancy, large memory consumption, and sensitivity to motion blur in the deployment of existing lightweight pose estimation algorithms on mobile and embedded devices. It innovatively proposes a lightweight human pose estimation algorithm that combines dynamic resolution adaptation and structural reparameterization mechanism to significantly improve inference speed and reduce peak memory consumption, thereby effectively supporting the real-time application requirements in low computing power environments.

[0106] Beneficial Effects: This invention rationally integrates the dynamic resolution adaptation module and the structural reparameterization backbone network optimization scheme into the original OpenPose model. The dynamic resolution adaptation module accurately judges the scene blur level through motion blur assessment and selectively chooses high and low resolution modes, preserving key details of human joints in high-blur scenes while reducing computational redundancy in low-blur scenes. The structural reparameterization backbone network reconstructs the stage structure based on RepVGG-B1, combining the characteristics of multi-branch enhancement of feature representation during training and single-path structure simplification of computation during inference. Simultaneously, it achieves seamless integration with the pose estimation subnetwork through channel adaptation and texture refinement. The improved model effectively solves the problems of high computational redundancy and high memory consumption caused by the VGG-19 backbone network in the original OpenPose model. This allows the improved model to maintain good human pose estimation accuracy, accurately detect and locate key human joints, and reasonably connect limb structures. At the same time, it can significantly improve inference speed and reduce resource consumption. It can be successfully adapted to low-computing scenarios such as embedded devices and mobile terminals, and meet the real-time pose detection requirements of intelligent video surveillance, human-computer interaction, and autonomous driving. It also has stronger robustness to motion-blurred scenes, further expanding the deployment scope and practical value of the OpenPose model in complex real-world application scenarios. Attached Figure Description

[0107] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0108] Figure 1 This is a schematic diagram of the overall architecture of a lightweight human pose estimation algorithm based on dynamic resolution adaptation and structural reparameterization.

[0109] Figure 2This is a schematic diagram of the dynamic resolution adaptation module.

[0110] Figure 3 This is a schematic diagram of the RepVGGBlock structure.

[0111] Figure 4 This is a structural reparameterized backbone network structure diagram.

[0112] Figure 5 This is a diagram showing the effect of human posture estimation. Detailed Implementation

[0113] This embodiment provides a lightweight attitude estimation method based on dynamic resolution and structural reparameterization. Taking an indoor dining scenario as an example, the data is set as shown in Table 1 below.

[0114] Table 1 Setting Data Table

[0115]

[0116] The overall architecture of this method is as follows: Figure 1 As shown, the specific implementation steps for each stage are as follows:

[0117] Step 1: Based on the motion blur characteristics of video frames and the difference in human body proportions, dynamically adjust the resolution to reduce computational redundancy, as shown in the structure below. Figure 2 As shown. Detailed steps are as follows:

[0118] Step 1.1, Human Region Recognition. The lightweight object detection network YOLO-Nano is used to detect humans in the original video frames (1280×720×3), locating the bounding boxes (BBoxes) of four human bodies. The pure human body regions are cropped and extracted as input for subsequent blur assessment. The background regions outside the BBoxes are downsampled by a 1 / 2 ratio to weaken them and eliminate background blur interference. By calculating the ratio of the total area of ​​all human body BBoxes to the total area of ​​the frame, the human body proportion ρ = 0.31 (…). This is a high-percentage scenario.

[0119] Step 1.2, Shallow Feature Extraction. Input the original video frame I (size 1280×720×3), and extract shallow features using a 3×3 standard convolutional layer (stride 2, output channels 16). The convolution formula is as follows: .in , is a standard 3×3 convolution kernel (output channels are 16), s represents the stride (calculated every 2 pixels to achieve downsampling), and p is the edge padding (to ensure that edge information of the feature map is not lost after convolution). This is the shallow feature map output by the convolution. The original frame is reduced to 1 / 4 size due to a stride of 2, and the number of channels increases from 3 (RGB) to 16 (extracting 16 types of shallow features, such as contours and edges). The output feature map... The resolution is 640×360×16. Background noise is filtered out, and human contour features (such as the body contour of a seated person) are initially revealed. The peak response value of the feature map reaches 0.7 (only areas with response values ​​greater than 0.3 are retained, and noise is filtered out).

[0120] Step 1.3, depthwise and pointwise convolution processing. For shallow feature maps... (640×360×16) Feature optimization is performed in two steps: first, each channel is processed individually using depthwise convolution (focusing on blur-related details), and then channel information is fused using pointwise convolution (compressing redundant dimensions). Details are as follows:

[0121] Step 1.3.1, Depthwise Convolution. Convolution is performed individually on each input channel to avoid interference between channels and focus on the blurred features of dynamic human regions (such as an arm holding a cup). The key formula is: Fuzzy features. Among them, The kernel is a 3×3 depth convolution kernel (with 16 channels, the same as the number of input channels, and each kernel corresponds to 1 input channel). The feature map output by the deep convolution has its number of channels changed from 16 to 32 (each input channel generates 2 types of features to enhance blurred details).

[0122] Step 1.3.2, Pointwise Convolution. A 1×1 convolution is used to fuse multi-channel features, compressing the number of channels to 1, eliminating irrelevant information, and obtaining a single-channel blurred feature map. The key formula is: .in, Here, is a 1×1 pointwise convolution kernel (output channel number = 1). This is a single-channel blurred feature map output from point-by-point convolution. The output... With dimensions of 640×360×1, the pixel value variance of static areas (such as a dining table) is <0.1 (uniform distribution, no blur); the pixel value variance of dynamic areas (such as an arm holding a glass) is >0.5 (large response fluctuations, significant blur features), verifying that the ambiguity assessment focuses on human dynamics.

[0123] Step 1.4, Motion Blur Evaluation. The single-channel blurred feature map is processed using Global Average Pooling (GAP). The result (640×360×1) is compressed into a 1D scalar, and then the Sigmoid activation function is used to map the result to the 0-1 range to obtain the final ambiguity score. The formula for Global Average Pooling (GAP) is:

[0124]

[0125] Where H=640 and W=360 are respectively Height and width; for In the first The value of a pixel.

[0126] The Sigmoid activation formula is:

[0127]

[0128] Output fuzziness score .

[0129] Step 1.5, Dynamic Resolution Decision. Based on the ambiguity scalar and the proportion of the human body, the optimal resolution is selected through a dynamic decision-making mechanism. According to the formula... Adjust the threshold and substitute. Calculated .because This is identified as a low-dynamic scene, and the resolution can be reduced to decrease computational redundancy. (Ambiguity scalar) , in and Between (medium blur scene), and (For high-percentage scenarios), based on the resolution decision matrix, select a medium-level resolution. .

[0130] Step 1.6, Resolution Scaling. Based on the determination of a blurred scene with a high proportion of human figures in Step 1.5, bilinear interpolation is used to scale the original frame to a medium resolution. The bilinear interpolation formula is as follows:

[0131]

[0132] in, For the adapted frames in The value of a pixel; In the original frame The four nearest pixels ( , , , ); Interpolation weight coefficients (based on) The distance to neighboring pixels is calculated, with closer pixels receiving a higher weight to ensure smooth pixel transitions.

[0133] Output (256×192×3), the resolution was reduced from 1280×720 to 256×192 (only 1 / 5 of the original size), memory usage was greatly reduced, and the pixel continuity was intact with no obvious distortion.

[0134] Step 2: Structural reparameterization feature extraction. Based on the adapted frame. (256×192×3), the RepVGG backbone network extracts "high semantic + spatial detail" features, while simultaneously converting the multi-branch structure during training into a single-path structure during inference, thus improving inference efficiency. The RepVGGBlock structure is as follows: Figure 3 As shown.

[0135] Step 2.1, phased feature extraction, structure as follows: Figure 4 As shown. Based on the RepVGG-B1 framework, only Stage 0 to Stage 2 are retained to achieve hierarchical feature transformation (the input is the adaptation frame). (256×192×3):

[0136] Step 2.1.1, Stage 0: Initial feature mapping. A 2x downsampling is performed using 3×3 convolution + batch normalization (BN) to extract low-frequency textures (such as human body contours).

[0137] Output (128×96×64);

[0138] Size change: 256×192→128×96 (step 2 downsampling);

[0139] Channel change: 3→64 (extracting 64 initial features to enhance contour discrimination).

[0140] Step 2.1.2, Stage 1: Feature Channel Preservation. Stack 3 RepVGGBlocks (multi-branch during training: 3×3 convolution + 1×1 convolution + identity mapping) to maintain the resolution and enhance the association of local features (such as the connection between the human torso and limbs).

[0141] Output: (128×96×64);

[0142] Size remains unchanged: 128×96 (step 1);

[0143] Channel remains unchanged: 64;

[0144] Step 2.1.3, Stage 2: Semantic Feature Enhancement. Stack 4 RepVGGBlocks, with a stride of 2 for the first block (downsampling) and a stride of 1 for the rest, increasing the number of channels to 256 and enhancing high-level semantic features (such as global position of joints and limb associations).

[0145] Output: (32×24×256);

[0146] Size change: 128×96→32×24 (step 2 downsampling, achieving 4 times downsampling compared to Stage 1);

[0147] Channel change: 64→256 (extracting more semantic features and encoding the association strength of limb affinity fields (PAFs), such as the connection between the arm and torso of the person raising the glass).

[0148] Step 2.2, Connection processing with the pose estimation subnetwork. To match the input requirements of the subsequent CPM (Convolutional Pose Machine) subnetwork, the connection is processed... By performing "channel dimension normalization + feature refinement" processing, seamless connection is achieved.

[0149] Step 2.2.1, Channel Dimension Normalization. This is achieved through 1×1 convolutions without batch normalization or activation functions. By standardizing the number of feature channels, eliminating redundant dimensions, and reducing subsequent computational complexity, this approach matches the input channel requirements of the CPM module, avoiding feature transfer loss caused by channel incompatibility. The 1×1 convolution parameter has 512 output channels, a stride of 1, and zero padding. The dimensions are 32×24×512.

[0150] Step 2.2.2, Feature Refinement and Distribution Alignment. This is achieved through a two-step operation using the CPM module. Optimize features to ensure compatibility with the Initial Stage:

[0151] (1) Feature distribution alignment: Unify the range of feature values ​​by 1×1 convolution (e.g., map feature values ​​to [-1,1]) to avoid the influence of numerical differences on subsequent predictions;

[0152] (2) Feature texture refinement: By using three depthwise separable convolutions (DSC, kernel size 3×3, stride 1, padding 1), we focus on the details of the joint edges (such as the outline of the wrist and ankle) while reducing the computational cost.

[0153] Output feature map (32×24×128). The number of channels was reduced from 512 to 128 to match the input requirements of the initial stage of pose estimation. The feature texture resolution was kept at 32×24 to ensure the accuracy of joint localization and to achieve seamless connection between the backbone network and sub-networks.

[0154] Step 2.3, Structure Reparameterization and Inference Optimization. The RepVGG official library interface is called to convert the multi-branch RepVGGBlock during training into a single-path 3×3 convolution during inference, reducing memory usage, improving inference speed, and adapting to low-computing-power scenarios.

[0155] Step 2.3.1: Through linear transformation, the parameters (convolution weights W, bias b, BN mean) of the convolutional and batch normalization layers in each RepVGGBlock are transformed. ,variance Scaling factor The parameters are fused into a single biased convolutional layer. The fused weights are... The bias after fusion is . To prevent tiny values ​​with a denominator of 0, For BN bias, This is the joint scale factor.

[0156] Step 2.3.2, Branch Structure Unification. The 1×1 convolutional branch and the identity mapping branch in RepVGGBlock are unified into 3×3 convolutional kernels to prepare for subsequent parameter fusion, as follows:

[0157] (1) 1×1 convolution branch: The 1×1 convolution kernel is expanded into a 3×3 convolution kernel by zero-padding (the padding value is 0, which does not change the convolution effect).

[0158] (2) Identity mapping branch: transformed into a 3×3 convolution kernel with "center element = 1 and surrounding 8 elements = 0" (equivalent to direct output of input features without additional calculation).

[0159] Step 2.3.3, Multi-branch parameter fusion. The convolution weights and biases of the three unified 3×3 convolution branches (original 3×3 convolution, expanded 1×1 convolution, and transformed identity mapping) are added together to generate the parameters of a single-path 3×3 convolution.

[0160] Step 3: Using the adapted feature map output in Step 2 The input image is (32×24×128). A multi-stage pose estimation sub-network completes joint detection, limb connection, and result mapping, ultimately generating a labeled human pose image (1280×720×3). Figure 5 As shown in the diagram, the rendering clearly presents the joint positions and limb connections of four people (2 seated, 1 raising a glass, and 1 sideways): the key joints such as the elbows and wrists of the seated person are accurately located; the raised arm posture of the person raising a glass is naturally connected to the limbs; and the torso outline of the sideways person is consistent with the limb connections. Even if there is partial occlusion in the scene (such as the head of the seated person on the left obscuring the hip joint of the person raising a glass), thanks to the joint position mask enhancement design during the structural reparameterization process, the potential position of the obscured joints can still be accurately inferred. Under the low computing power conditions of resolution compression and CPU inference alone, a clear visual effect and accurate posture restoration are still maintained, intuitively demonstrating the excellent balance between lightweight deployment and posture estimation accuracy of this invention.

[0161] The experimental dataset used was COCO 2017, covering single-player and multi-player scenes, complex backgrounds, and diverse pose variations. The training environment consisted of a CPU (Intel® Core™ i7-14700KF) and a GPU (GeForce RTX 3090). To demonstrate the advantages of lightweight networks in low-computing-power scenarios, the testing phase used only the CPU as the computing resource, without enabling GPU acceleration. This setting allows for more intuitive verification of the model's actual performance in a general hardware environment. Comparison algorithms included the classic OpenPose model, lightweight improved models (SqueezeNet-OpenPose, ShuffleNet-OpenPose), and the mainstream lightweight pose estimation model, Lightweight OpenPose.

[0162] In terms of accuracy, the algorithm achieved an average precision (AP) of 49.2% and an average recall (AR) of 53.2%. While slightly lower than the original OpenPose model (AP 53.5%, AR 56.0%), it significantly outperformed other lightweight improvements such as SqueezeNet-OpenPose (AP 43.81%, AR 47.93%) and ShuffleNet-OpenPose (AP 45.06%, AR 48.85%), and also surpassed Lightweight OpenPose (AP 48.5%, AR 51.8%), demonstrating good balance between accuracy and performance.

[0163] In terms of lightweight characteristics, this method exhibits significant advantages. The number of parameters is only 6.8 MB, far lower than OpenPose's 62.5 MB; the number of multiply-accumulate operations (MADD) is 23.2 GMAdd, and the number of floating-point operations (FLOPs) is 8.7 GFlops, which are significantly reduced compared to OpenPose's 286.9 GMAdd and 116.2 GFlops; the memory read / write operation (MemR+W) is 547.8 MB, also lower than OpenPose's 674.3 MB, fully validating its low resource consumption design advantage.

[0164] In terms of inference efficiency, this algorithm achieved an inference speed of 17.18 FPS and an average execution time of 31.7 ms in a pure CPU environment, far exceeding OpenPose's 1.48 FPS and 674.3 ms, representing a speed improvement of approximately 11.6 times. It also outperformed ShuffleNet-OpenPose (16.23 FPS) and Lightweight OpenPose (2.38 FPS), highlighting its high real-time performance on low-computing-power platforms. The results are shown in Table 2.

[0165] Table 2 Performance Comparison of Different Models

[0166]

[0167] Table 3. Comparison of Complexity of Different Backbone Network Models

[0168]

[0169] In summary, this invention, through a series of lightweight design and optimization strategies, significantly reduces model complexity and computational overhead while maintaining high accuracy, thereby significantly improving operational efficiency. It is suitable for application scenarios with strict requirements for real-time performance and deployment resources, such as intelligent monitoring and human-computer interaction.

[0170] This invention provides a lightweight attitude estimation method based on dynamic resolution and structural reparameterization. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A lightweight attitude estimation method based on dynamic resolution and structural reparameterization, characterized in that, Includes the following steps: Step 1, Dynamic resolution adaptation processing: By identifying the core human body region, accurately quantifying the degree of motion blur, and dynamically correcting the decision threshold based on the proportion of the human body, the resolution is adaptively selected based on multi-dimensional parameters, and frame data with the adapted resolution is output to provide optimized input for the subsequent backbone network; Step 2, Reparameterization of the backbone network: Construct a backbone network adapted for pose estimation, complete feature extraction, subnetwork connection and inference optimization, and output a refined feature map; Step 3, Generate pose estimation results: Based on the refined feature map output in Step 2, the multi-stage pose estimation sub-network completes joint heatmap prediction, location localization and result optimization, and finally generates human pose estimation results that match the original input frame.

2. The method according to claim 1, characterized in that, Step 1 includes: Step 1.1, Human body region recognition: By detecting human body regions and calculating the proportion of human body, the core human body region is extracted and the background is weakened; Step 1.2, Motion blur assessment: Extract blur features of the human body region through a lightweight three-level convolutional network and jointly optimize the training to achieve end-to-end blur prediction and quantify the degree of motion blur of the human body region in the original frame. Step 1.3, Resolution dynamic decision-making mechanism execution: Based on ambiguity scalar, human body proportion and threshold, adaptive resolution selection; Step 1.4, Resolution Scaling: Use bilinear interpolation to achieve frame size conversion to adapt to the resolution.

3. The method according to claim 2, characterized in that, Step 1.1 includes: Step 1.1.1, Human Region Extraction and Background Weakening: The lightweight object detection network YOLO-Nano is used to detect human bodies in the original video frames, locate the bounding boxes of all human bodies in the frames, and crop the detected human body bounding boxes to extract pure human body region images as input for subsequent blur evaluation; the background region outside the human body bounding boxes is weakened by downsampling at a ratio of 1 / 8. Step 1.1.2, Human body proportion calculation: Calculate the human body proportion ρ, which is the ratio of the total area of ​​all human body bounding boxes to the total area of ​​the original video frame, ρ∈[0,1].

4. The method according to claim 3, characterized in that, Step 1.2 includes: Step 1.2.1, Configure the lightweight three-level convolutional network structure: The lightweight three-level convolutional network uses the cropped human body region as the processing object, avoids background blur interference, and ensures that the output score directly reflects the blur degree of the region where the joint is located. The overall three-level progressive structure achieves a balance between lightweight computation and evaluation accuracy. The first level is a standard 3×3 convolutional layer, which realizes shallow feature extraction. The kernel size is 3×3, the stride is 2, and the number of output channels is 16, which can completely cover different types of shallow blurry features in the human body area; the edge padding is 1. The second level is a depthwise separable convolutional layer, used to deepen fuzzy features. It consists of two parts: depthwise convolution and pointwise convolution. The kernel size of the depthwise convolution is 3×3, the number of input channels is the same as the number of output channels of the first level, and each channel is configured with an independent convolution kernel. The kernel size of the pointwise convolution is 1×1, and the number of output channels is set to twice the number of input channels. The third stage consists of a 1×1 convolutional layer, a global average pooling layer, and a sigmoid activation layer, which completes the fuzziness scalar output. The 1×1 convolutional layer has the same 32 input channels as the second stage output, and the number of output channels is set to 1. The global average pooling layer aggregates the single-channel fuzzy feature map across the entire spatial dimension, calculates the average value of all pixel values, and transforms the two-dimensional spatial features into a one-dimensional scalar. The sigmoid activation layer constrains the one-dimensional scalar within the range of 0 to 1 through a non-linear mapping, and finally outputs the fuzziness scalar. Step 1.2.2, Feature Extraction and Blur Calculation: Using the cropped human body region image as input, feature extraction and fuzziness quantification are completed sequentially through a three-level network structure. The specific process includes: Step 1.2.2.1: The input human body region image is fed into a 3×3 standard convolutional layer. The 3×3 standard convolutional layer compresses the spatial resolution of the input image to half of its original value through a convolution operation with a stride of 2. At the same time, it extracts the edge texture of the human body contour and key joints, and outputs a shallow feature map. ; Step 1.2.2.2: Feature map output after a 3×3 standard convolutional layer The process then proceeds to the depthwise separable convolutional layer. This layer, through two steps of depthwise convolution and pointwise convolution, further captures motion blur features and correlates the blurry relationships between different joints. Specifically, this includes: Depthwise convolution calculation: Each channel of the input feature map is convolved independently, without cross-channel information fusion. The calculation formula is as follows: ， in, The input feature map for depthwise convolution; It is the symbol for depthwise convolution; Use depthwise convolution kernels; It is the bias term of depthwise convolution; is the output feature map of the depthwise convolution, where h, w, and c represent the height, width, and number of channels of the output feature map of the depthwise convolution, respectively; Pointwise convolution calculation: Pointwise convolution is based on... As input, cross-channel fusion is achieved through a 1×1 convolution kernel, relating the fuzzy relationships between different joints. The calculation formula is as follows: , in, It is the input feature map of pointwise convolution. For input channel index; It is the symbol for pointwise convolution; For pointwise convolution kernels, where For output channel index; It is the bias term of pointwise convolution; It is the output feature map of pointwise convolution; Step 1.2.2.3: After processing with depthwise separable convolutional layers, the feature map has dimensions H1×W1×32. The 32-dimensional channels are compressed to 1 dimension using a 1×1 convolutional kernel, outputting a single-channel blurred feature map. Where H1 and W1 represent the height and width of the feature map, respectively; Then, Input a global average pooling layer, and use the formula The average of all spatial pixels in the single-channel feature map is taken to aggregate the two-dimensional spatial features into a one-dimensional scalar S. Finally, the scalar S is input into the Sigmoid activation layer, through the formula... The nonlinear mapping of S is applied to the range of 0 to 1, and the final output is the ambiguity scalar s. Step 1.2.3, Network Training Optimization: A joint optimization using the mean squared error loss function and the total variation regularization term is employed to simultaneously achieve the dual objectives of accurate ambiguity prediction and stable, unfluctuating output. The formula for this joint optimization is... for: ， in These are truth labels for fuzziness generated based on Laplacian gradient variance. This is a blur prediction value output by a lightweight three-level convolutional network. The lightweight three-level convolutional network learns the features of the input image and outputs a score of the blur level of the original video frame. This is the mean squared error loss term, where N is the number of samples; λ is the gradient of the ambiguity prediction value; λ is the smoothing constraint coefficient. This is the regularization term for the total variation.

5. The method according to claim 4, characterized in that, Step 1.3 includes: Step 1.3.1, preset the following core parameters: Basic ambiguity threshold The baseline threshold used to distinguish between high-blur and low-blur scenes; Proportion adjustment coefficient α: used to control the degree of influence of human body proportion on the decision threshold; Ambiguity classification threshold: set to ,satisfy It is used to classify ambiguity into four levels: extremely high ambiguity, high ambiguity, medium ambiguity, and low ambiguity. Human body proportion grading threshold : When the proportion of the human body is high, it is classified as high; otherwise, it is classified as low. Multi-level resolution set: set as Each resolution is distributed in a gradient according to a preset ratio to satisfy... ; Step 1.3.2, Dynamically correct the threshold: The base threshold is corrected based on the proportion of the human body ρ. To obtain the dynamic threshold The formula is: ； Step 1.3.3, Dynamic resolution selection: Establishing the resolution decision matrix specifically includes: Extremely high blur, high proportion of scenes: and Select the highest resolution. It is suitable for close-range, fast-moving human bodies; High-blur, high-percentage scenes or extremely high-blur, low-percentage scenes: and ,or and Choose the second highest resolution. ; Medium blur, high percentage scene or high blur, low percentage scene: and ,or and Choose a medium resolution. ; Low-blur, high-percentage scenes or medium-blur, low-percentage scenes: and ,or and Choose the second lowest resolution. ; Low-blur, low-percentage scenes: and Select the lowest resolution. .

6. The method according to claim 5, characterized in that, Step 2 includes: Step 2.1, Stage-based feature extraction: Reconstruct the stage structure of RepVGG-B1, discard stage3~stage4, retain stage0~stage2, and realize hierarchical feature transformation; Step 2.2, Connection processing with the pose estimation sub-network: Channel adaptation and texture refinement are performed on the output features of the backbone network to ensure compatibility with subsequent pose estimation sub-networks: Step 2.3, Structure reparameterization and inference optimization: Convert the multi-branch structure during training into a single-path structure during inference.

7. The method according to claim 6, characterized in that, Step 2.1 includes: Step 2.1.1, stage0, initial feature mapping: The feature extraction unit of stage0 consists of a 3×3 convolutional layer and a BN layer. The parameters of the convolutional layer are: kernel size 3×3, stride 2, padding 1, and output channels 64. Step 2.1.2, stage 1, feature channel preservation: The feature extraction unit of stage 1 consists of 3 stacked RepVGGBlocks, each RepVGGBlock has 64 input and output channels and a stride of 1; The feature map resolution is maintained at 112×112×64, and the local feature association is enhanced by a multi-branch structure during training; the multi-branch structure during training includes 3×3 convolution, 1×1 convolution, and identity mapping. Step 2.1.3, Stage 2, Semantic Feature Enhancement: The feature extraction unit of stage 2 consists of 4 RepVGGBlocks, with a stride of 2 for the first block and a stride of 1 for the remaining blocks. Stage 2 increases the number of feature channels from 64 to 128 and performs a 4-fold downsampling relative to stage 1.

8. The method according to claim 7, characterized in that, Step 2.2 includes: Step 2.2.1, Channel dimension normalization: By using a 1×1 convolutional layer without BN or activation function, the 128-channel feature map output from stage 2 is adjusted to increase the number of channels to 512. Step 2.2.2, Distribution Alignment and Feature Refinement: The adapted feature map is input into the Convolutional Pose Machine (CPM) module to complete feature distribution alignment and texture refinement: first, feature distribution alignment is achieved through 1×1 convolution, and then the feature texture is further refined through 3 depthwise separable convolutional layers with a kernel size of 3×3 and a stride of 1. Finally, the feature map is output that can be seamlessly connected with the pose estimation sub-network.

9. The method according to claim 8, characterized in that, Step 2.3 includes: Step 2.3.1, Inference Mode Transformation Configuration: Call the RepVGG official library's structure reparameterization interface to define a transformation function. The input is the training state RepVGGBlock, and the output is the inference state single-path convolutional structure; specifically, it includes the following steps: Step 2.3.1.1, define the input specifications: limit the input to the training state RepVGGBlock instance, including a complete multi-branch structure: 3×3 convolution, 1×1 convolution, identity mapping, and the parameters of each branch match the corresponding network stage and the output channels are consistent; Step 2.3.1.2, Design the conversion logic: Based on the RepVGG official interface, encapsulate the logic, first verify the integrity of the branches and the parameter dimensions, and exclude abnormal inputs; then convert the multi-branch into a single 3×3 convolutional layer according to the RepVGG-B1 rule, and force the convolutional layer to be padded with 1 and stride with 1. Step 2.3.1.3, verify the transformation results: First, verify the consistency of dimensions by using test features to confirm that the spatial dimension and number of channels of the output of the training state and the inference state are consistent; second, verify the integrity of parameters by checking whether the weights and biases of the convolutional layers in the inference state are complete. Step 2.3.1.4, Batch integration of networks: Following the original training state structure order, convert RepVGGBlock of stage1 and stage2 into inference state convolutional layers one by one and replace them, while keeping the forward propagation path unchanged; Step 2.3.2: Parameter Fusion Processing: The mathematical formulas in the RepVGG parameter fusion process are improved, specifically including the following steps: Step 2.3.2.1: In the training state RepVGGBlock, both the 3×3 convolutions and 1×1 convolutions are cascaded with independent BN layers, and a keypoint scale factor is introduced into the fusion formula. The bias after fusion is adaptively adjusted to enhance the small joint feature response. The fusion formula is modified as follows: Convolution weights after fusion: , Post-fusion convolution bias: , in, , These are the convolutional layer weights and biases corresponding to the i-th output channel, respectively; , These are the scaling factor and bias of the BN layer corresponding to the i-th output channel, respectively; , These are the mean and variance of the BN layer corresponding to the i-th output channel, respectively; It is a tiny constant; is the joint scale factor, and j is the joint category index; This is the adjustment coefficient; , These are the fused convolutional weights and biases corresponding to the i-th output channel, respectively; Step 2.3.2.2, Branch Structure Unification: The training state RepVGGBlock contains three types of branches: 3×3 convolution, 1×1 convolution, and identity mapping. A constrained transformation strategy is designed for the two types of non-3×3 branches, specifically including: For 1×1 convolution branches: constrained zero-padding; When expanding a 1×1 convolutional kernel to a 3×3 convolutional kernel, only zero padding is applied to the outer ring of the kernel. The conversion formula is as follows: ; in, These are the weights of the original 1×1 convolution branch; As a constrained zero-padding operation, zero values ​​are padded only on the top, bottom, left, and right sides of the 1×1 convolutional kernel to form a 3×3 convolutional kernel, while the original weight values ​​are retained in the center 1×1 kernel. The weights are the expanded 3×3 convolution kernel weights; For the identity mapping branch: key point location mask enhancement; When transforming the identity mapping into a 3×3 convolution kernel, a key point location mask and a small constant are introduced, and the transformation formula is as follows: ; in, represents the weights of the 3×3 convolution kernel after the transformation identity mapping; (i,j) represents the spatial coordinates of the convolution kernel; I is the identity matrix, with the central element being 1 and the rest initially set to 0; This is a mask for the key point locations; It is a small constant; Step 2.3.2.3, Multi-branch parameter fusion: Add the unified 3×3 convolution kernel weights and biases separately to generate the parameters of a single-path 3×3 convolution; By introducing dynamic weight coefficients and identity mapping branch weights, the weights are dynamically optimized through backpropagation of the pose estimation loss function. The formula is modified as follows: Fusion convolution weights : ; Convolutional bias after fusion : ; in, , These are the weights and biases after the original 3×3 convolution branches are merged; , These are the weights and biases of the fused components after expanding the 1×1 convolutional branches; , These are the weights and biases of the fused components after the identity mapping branch transformation, respectively; , These are dynamic weighting coefficients.

10. A storage medium, characterized in that, It stores a computer program or instructions that, when run on a computer, perform the steps of the method as described in any one of claims 1 to 9.

Images