A method, system, electronic device, and storage medium for human pose estimation.

Abstract

This invention discloses a human pose estimation method, system, electronic device, and storage medium, relating to the fields of computer vision and digital image processing technology. The method includes: acquiring target human bounding box data and inputting the target human bounding box data into a feature extraction network to determine fine-grained features; the feature extraction network uses a pyramid Transformer as the backbone network, and the intermediate feature maps generated in the backbone network are processed by a local information enhancement module; a fine-grained fusion mechanism is used to perform cross-layer feature correction on the fine-grained features to obtain fused features; a coordinate information interaction module is used to perform cross-axis information complementation on the fused features to obtain coordinate information based on the relative positions of various joints and parts of the human body in space; and the coordinate information of key points of the human body is combined and connected to obtain the human pose estimation result. This invention can improve the accuracy of human pose estimation.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and digital image processing technology, and in particular to a human pose estimation method, system, electronic device, and storage medium. Background Technology

[0002] In the context of the artificial intelligence era, studying the motion state of humans as subjects of multimedia information has become an important and far-reaching task. Human pose estimation, a crucial research direction in computer vision, aims to predict the two-dimensional positions of key human skeletal points (such as joints and limbs) in a given image or video, and to recover the human pose based on the geometric relationships between these key points. However, complex pose variations, occlusion, truncation, insufficient lighting, image blurring, and low-resolution human instances pose significant challenges to human pose estimation.

[0003] In early research, human pose estimation typically employed coordinate regression-based methods to directly regress the coordinates of keypoints. However, this approach struggles to effectively model the structural relationships between keypoints, resulting in poor performance. In recent years, heatmap-based regression has become mainstream. These methods smooth the training process and significantly improve performance by assigning probability values ​​to each pixel location. However, heatmap-based regression suffers from quantization errors, particularly in low-resolution images or small-scale scenarios. To address this quantization error, numerous innovative methods have been proposed. One effective approach redefines human pose estimation as two independent classification tasks on the horizontal and vertical coordinates. This method not only eliminates the expensive upsampling operation in heatmap regression but also achieves superior performance in small- to medium-scale scenarios. However, this approach does not fully consider the inherent correlations between coordinate axes; the interaction between information from different coordinate axes is crucial for human pose estimation.

[0004] Visual Transformers have redefined the performance benchmark for vision tasks through a global self-attention mechanism. While some methods based on the Visual Transformer architecture have achieved state-of-the-art results, their computational overhead and large parameter counts hinder deployment in resource-constrained environments. To address this, the Pyramid Transformer balances efficiency and accuracy through progressive feature pyramid compression and spatially reduced attention. However, these pure transformer architectures lack the inductive bias inherent in convolutional neural networks, particularly their ability to capture local spatial hierarchies and scale-invariant patterns, which are crucial for solving fine-grained pose details. Summary of the Invention

[0005] The purpose of this invention is to provide a human posture estimation method, system, electronic device, and storage medium, aiming to solve or improve at least one of the above-mentioned technical problems.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A human pose estimation method includes:

[0008] Obtain the target human bounding box data;

[0009] The target human bounding box data is input into a feature extraction network to determine fine-grained features; the feature extraction network uses a pyramid Transformer as the backbone network, and the intermediate feature maps generated in the backbone network are processed by a local information enhancement module;

[0010] A fine-grained fusion mechanism is used to perform cross-layer feature correction on fine-grained features to obtain fused features;

[0011] The coordinate information interaction module is used to perform cross-axis information complementation on the fused features to obtain coordinate information based on the relative positions of various joints and parts of the human body in space;

[0012] The human posture estimation result is obtained by combining and connecting the coordinate information of key human body points; the key human body points include nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle.

[0013] Optionally, the step of inputting the target human body bounding box data into a feature extraction network to determine fine-grained features specifically includes:

[0014] The target human bounding box data is input into the backbone network and processed through four stages to form a feature pyramid with resolution decreasing from high to low. Each stage uses spatial reduction blocks to progressively compress the feature map resolution and expand the number of channels according to a set value. The resolution of each stage is sequentially equal to the resolution of the input image. and

[0015] The feature pyramid is input into the local information enhancement module for processing, and the enhanced features are output:

[0016] f out =Conv(Concat(f1′,f3+f2′)).

[0017] Where Conv represents convolution processing, Concat represents merging two tensors along the channel layer, f1′ represents small-scale features, f2′ represents medium-to-large-scale features, and f3 represents medium-to-large-scale features refined by context-dependent weights.

[0018] Optionally, the step of using a fine-grained fusion mechanism to perform cross-layer feature correction on fine-grained features to obtain fused features specifically includes:

[0019] The features output by the backbone network and the features output by the local information enhancement module are added along the channel dimension, and then one-dimensional context information is determined through global pooling operation to generate channel descriptors;

[0020] The channel descriptors are fed into a multilayer perceptron for fusion calculation to obtain fused features:

[0021] F″=λ1·F4+λ2·f out .

[0022] Where F″ represents the feature F4 output by the fusion backbone network and the feature f output by the local information enhancement module. out The features are λ1 and λ2, which are two two-dimensional weights.

[0023] Optionally, the step of using the coordinate information interaction module to perform cross-axis information complementation on the fused features to obtain coordinate information based on the relative spatial positions of various joints and parts of the human body specifically includes:

[0024] The fused features are processed through three one-dimensional convolutional layers of different scales to obtain the axial information on the horizontal axis and the axial information on the vertical axis.

[0025] Cross-operation is used to interact the axial information on the horizontal axis and the axial information on the vertical axis to obtain the information F after interaction along the horizontal axis. H Information F after interaction along the vertical axis V And calculate the coordinate information based on the relative positions of various joints and parts of the human body in space:

[0026] F out =Conv 1×1 (F V )+Conv 1×1 (F H )+F″.

[0027] Where F″ represents the fusion feature, Conv 1×1 This represents one-dimensional convolution.

[0028] Optionally, the step of combining and connecting the coordinate information of key human body points to obtain the human pose estimation result specifically includes:

[0029] The size output by the coordinate information interaction module is... Coordinate information F out Flattened into an embedding vector Where N represents the number of human body keypoints, H represents the feature map height, and W represents the feature map width. Represents the real number field;

[0030] The embedding vector of each human body key point is subjected to two linear projections to encode the horizontal and vertical coordinate information. The encoding results are then input into a coordinate classifier to decode the horizontal and vertical coordinates of the human body key points.

[0031] The human pose estimation result is obtained by combining and connecting the coordinate pairs of N human body key points.

[0032] The present invention also provides a human pose estimation system, comprising:

[0033] The data acquisition unit is used to acquire the target human body bounding box data;

[0034] The feature extraction unit is used to input the target human bounding box data into the feature extraction network to determine fine-grained features; the feature extraction network uses a pyramid Transformer as the backbone network, and processes the intermediate feature maps generated in the backbone network through a local information enhancement module;

[0035] The feature correction unit is used to perform cross-layer feature correction on fine-grained features using a fine-grained fusion mechanism to obtain fused features;

[0036] The information complementation unit is used to perform cross-axis information complementation on the fused features using the coordinate information interaction module to obtain coordinate information based on the relative positions of various joints and parts of the human body in space.

[0037] The human posture estimation unit is used to combine and connect the coordinate information of human body key points to obtain human posture estimation results; the human body key points include nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle.

[0038] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to cause the electronic device to perform the human posture estimation method described above.

[0039] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the human pose estimation method as described above.

[0040] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0041] This invention discloses a human pose estimation method, system, electronic device, and storage medium. The method includes acquiring target human bounding box data and inputting the target human bounding box data into a feature extraction network to determine fine-grained features. The feature extraction network uses a pyramid Transformer as the backbone network, and the intermediate feature maps generated in the backbone network are processed by a local information enhancement module. A fine-grained fusion mechanism is used to perform cross-layer feature correction on the fine-grained features to obtain fused features. A coordinate information interaction module is used to perform cross-axis information complementation on the fused features to obtain coordinate information based on the relative positions of various joints and parts of the human body in space. The coordinate information of key points of the human body is combined and connected to obtain the human pose estimation result. This invention can improve the accuracy of human pose estimation. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart of the human pose estimation method based on local information enhancement and coordinate interaction of the present invention;

[0044] Figure 2 This is a schematic diagram of the human pose estimation method based on local information enhancement and coordinate interaction in this embodiment; wherein, (a) is a schematic diagram of the local information enhancement module LIEM; (b) is a schematic diagram of the coordinate information interaction module CIIM; (c) is a schematic diagram of the fine-grained fusion mechanism FIFM; and (d) is a schematic diagram of the overall model architecture.

[0045] Figure 3 This is a schematic diagram showing the comparison of the visualization results between the present invention and the benchmark algorithm on the COCO validation dataset in this embodiment;

[0046] Figure 4 This diagram illustrates the computational performance comparison between the present invention and other algorithms on the COCO validation and COCO test-dev datasets in this embodiment; where (a) shows the comparison results on COCO validation; and (b) shows the comparison results on COCO test-dev.

[0047] Figure 5 This is a schematic diagram comparing the changes in loss values ​​during training between the present invention and the benchmark method;

[0048] Figure 6This is a schematic diagram illustrating the visualization results of the present invention in some dense scenarios on the COCO validation dataset in this embodiment. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] The purpose of this invention is to provide a human posture estimation method, system, electronic device, and storage medium, aiming to solve or improve at least one of the above-mentioned technical problems.

[0051] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] like Figure 1 As shown, the present invention provides a human pose estimation method, including:

[0053] Step 1: Obtain the target human body bounding box data.

[0054] Step 2: Input the target human bounding box data into the feature extraction network to determine fine-grained features; the feature extraction network uses the Pyramid Transformer as the backbone network, and the intermediate feature maps generated in the backbone network are processed by the local information enhancement module.

[0055] Step 3: Use a fine-grained fusion mechanism to perform cross-layer feature correction on the fine-grained features to obtain fused features.

[0056] Step 4: Use the coordinate information interaction module to perform cross-axis information complementation on the fused features to obtain coordinate information based on the relative positions of various joints and parts of the human body in space.

[0057] Step 5: Combine and connect the coordinate information of the key points of the human body to obtain the human posture estimation result; the key points of the human body include the nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle.

[0058] As a specific implementation method, the following specific processing procedure is provided.

[0059] In this embodiment, a pyramidal Transformer is used as the backbone network. The intermediate feature maps generated in the backbone network are processed by a Local Information Enhancement (LIEM) module, which is implemented using a convolutional neural network. By mining local contextual features, the robustness of the model to occlusion is enhanced. Next, fine-grained features are processed through a fine-grained fusion mechanism (FIFM). The features from the original backbone and those processed by the LIEM are then corrected across layers, enhancing local features while preserving the global background. Finally, the fused information is interacted along the coordinate axes to establish cross-axis multi-resolution feature associations, achieving information complementarity between coordinate axes. In this way, the model exhibits better robustness under occlusion and improves the accuracy of keypoint detection.

[0060] The main steps are as follows:

[0061] Step 1: Use the human bounding box detector to obtain human bounding box data, preprocess the data, and use a series of data augmentation strategies to augment the training data, including at least the following strategies: random scaling factor, random rotation, random horizontal flip, random color enhancement, random occlusion, and additional half-body augmentation.

[0062] Step 2: The preprocessed image data is input into the backbone network of the Pyramid Transformer for multi-scale feature extraction. The Pyramid Transformer backbone network consists of four stages. Each stage progressively compresses the resolution of the feature maps and expands the number of channels through spatial reduction blocks, forming a feature pyramid from high to low resolution. The feature maps output by each stage are denoted as F1, F2, F3, and F4, respectively, with resolutions of [missing information - likely a percentage of the input image resolution]. The backbone network generates the F1 feature with the highest resolution and contains more detailed information. This feature is then fed into the local information enhancement module for processing. This module consists of a series of convolutions and depthwise convolutions. Specifically, the F1 feature is first fed into a convolutional layer, where the feature size is halved and the number of channels is doubled. Then, it is divided into two parts along the channel direction:

[0063]

[0064] Next, f1 and f2 are fed into two different branches. The left branch is used to process small-scale information:

[0065] f1′=Conv(DWConv(Conv(f1))).

[0066] DWConv stands for Depthwise Convolution, which reduces computation while maintaining performance. The right branch extracts mid-scale and large-scale information. Specifically, we obtain context-dependent weights by performing global average pooling, convolution, and depthwise convolution on the features.

[0067]

[0068] Where H and W represent the height and width of the feature map, respectively, and σ represents the sigmoid function. Next, local information at different scales is obtained using DWConv with different convolutional kernel sizes:

[0069]

[0070] For the kernel size of depthwise convolutions, we set them to 5×5, 7×7, and 9×9 respectively. Then, we computed more accurate multi-scale local information by element-wise multiplying the context-dependent weights and multi-scale local information:

[0071] f3 = weight·f2′.

[0072] Next, residual connections are introduced into the unrefined multi-scale local information, enabling the network to learn identity mappings and avoid learning unnecessary complex representations. Finally, the output of the local information enhancement module can be represented as:

[0073] f out =Conv(Concat(f1′,f3+f2′)).

[0074] Here, `Concat` represents merging two tensors along the channel layer. Through this operation, the local information enhancement module can effectively extract small-scale local information, which is crucial for the eye and key points requiring precise localization within a small spatial scale. Simultaneously, the design of the local information enhancement module also considers the ability to recognize mesoscale and large-scale information. This means the model can not only capture fine-grained local changes but also maintain sensitivity to overall structure and large-scale features, enabling it to handle complex tasks while considering features at different levels, thereby improving its comprehensive understanding of information at various scales.

[0075] Step 3: A fine-grained fusion mechanism is used to fuse information at different granularities. This mechanism accepts information from different granular levels and assigns different weights to global and local information based on their importance. Specifically, for the backbone network's output feature F4 and the enhanced feature f... out Add along the channel dimension, then obtain one-dimensional context information through a global pooling operation:

[0076]

[0077] Where c represents the channel dimension, and H and W represent the height and width of the feature map, respectively. Global average pooling integrates information from feature maps of different granularities and generates a channel descriptor F′, which is then fed into a multilayer perceptron for computation.

[0078] α = MLP(F′).

[0079] The MLP(·) consists of two fully connected layers and a ReLU activation function. The purpose of using a multilayer perceptron is twofold: first, to reduce the number of feature maps in the channel dimension to match the granularity; and second, to increase the channel dimension to generate fusion weights.

[0080]

[0081] Where σ represents the Sigmoid function, and λ1 and λ2 are two two-dimensional weights. Features of different granularities are calculated using their corresponding weights, ultimately yielding the fusion result.

[0082] F″=λ1·F4+λ2·f out .

[0083] Where F″ represents the fusion of F4 and f out The resulting F″ will be fed into the subsequent network structure. The FIFM mechanism adaptively assigns weights to different granularities based on global information and channel correlation in different granularity information spaces, ultimately generating more effective features.

[0084] Step 4: Achieve cross-axis information complementarity through the coordinate information interaction module. Human pose estimation involves accurately predicting the positions of various key points on the human body. In this process, the relationship between the x-axis and y-axis coordinates in the two-dimensional coordinate system is crucial. The relative positions of various joints and parts of the human body in space are influenced not only by their individual coordinate values ​​but also by the geometric constraints and relationships between them. Therefore, accurately capturing and modeling the intrinsic connections between these coordinates can effectively improve the model's understanding and prediction accuracy of human pose. Specifically, the computational process of this module is as follows: The feature map F″ generated using the fine-grained fusion mechanism is first processed through three one-dimensional convolutional layers of different scales to extract the axial information of the image. These convolutional layers capture features at different scales, extract the spatial relationships between key points along the axes, and calculate their outputs through the fine-grained fusion mechanism.

[0085] F 01 =FIFM(Conv 1×7 (Norm(F″)),Conv 1×11 (Norm(F″))),

[0086] F x =FIFM(F 01 Conv 1×21 (Norm(F″))).

[0087] Among them, F 01 It is the result of the first phase of integration, F x This refers to the axial information on the horizontal axis; FIFM is a fine-grained fusion mechanism. Correspondingly, the axial information on the vertical axis can be represented as:

[0088] F 11 =FIFM(Conv 7×1 (Norm(F″)),Conv 11×1 (Norm(F″))),

[0089] F y =FIFM(F 11 Conv 21×1 (Norm(F″))).

[0090] To make the information F on the horizontal axis x and information F on the vertical axis y To achieve this interaction, we perform cross operations on them. Specifically, we use F... x As the key matrix and value matrix, F is used y As a query matrix:

[0091] F V =MHCA y (F y ,F x ,F x ).

[0092] Among them, MHCA y (·,·,·) represents multi-head cross-attention along the horizontal axis. The operation on the vertical axis is similar:

[0093] F H =MHCA x (F x ,F y ,F y ).

[0094] Among them, MHCA x (·,·,·) represents multi-head cross-attention along the vertical axis. Information F is obtained after interaction along the horizontal and vertical axes. H and F V Then, the output of the coordinate information interaction module can be expressed as:

[0095] F out =Conv 1×1(F V )+Conv 1×1 (F H )+F″.

[0096] Step 5, the size from the coordinate information interaction module is... The output F out Flattened into an embedding vector Here, N represents the number of keypoints in the dataset. Next, linear projection is applied twice to the embedding vector of each keypoint to encode its horizontal and vertical coordinates, respectively. The encoded results are then input into a coordinate classifier to decode the horizontal and vertical coordinates of the keypoints. The coordinate pairs of the N keypoints are then combined. Taking the COCO dataset as an example, the model outputs 17 key points, which correspond to important joints and parts of the human body: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle. Then, following the human anatomical structure, the left shoulder is connected to the left elbow, the left elbow is connected to the left wrist, and so on, forming a complete skeletal framework from the head to the limbs, thus completing the human posture estimation.

[0097] To overcome the unavoidable occlusion problem in complex real-world environments, this invention leverages the excellent global dependency modeling capabilities of the Pyramid Transformer network. However, since it lacks the inductive bias inherent in convolutional neural networks, this invention proposes a local information enhancement module and a fine-grained fusion mechanism to reconstruct spatial continuity in the backbone network feature map, effectively alleviating the shortcomings of the self-attention mechanism in modeling local details. Simultaneously, a coordinate information interaction module is proposed to establish cross-axis multi-resolution feature associations, thereby achieving more accurate predictions in human pose estimation tasks.

[0098] Based on the above technical solution, the following embodiments are provided.

[0099] like Figure 1 The flowchart shown is a representation of the human pose estimation method based on local information enhancement and coordinate interaction according to the present invention. Figure 2 The diagram illustrates the human pose estimation method based on local information enhancement and coordinate interaction according to the present invention. It includes a Local Information Enhancement (LIEM) module, a Coordinate Information Interaction (CIIM) module, and a Fine-Grained Fusion Mechanism (FIFM). The description is as follows:

[0100] 1) First, a human bounding box detector is used on the input image to obtain human bounding box data. The data is then preprocessed and random data augmentation is performed.

[0101] 2) The preprocessed image is input into the backbone network of the Pyramid Transformer for feature extraction, resulting in feature maps F1, F2, F3, and F4. Feature map F1 is then fed into the local information enhancement module to obtain enhanced texture information f. out .

[0102] 3) Enhanced texture information f out By dynamically weighting the output feature map F4 of the backbone network through a fine-grained fusion mechanism, a feature map F″ containing global context and local information is obtained.

[0103] 4) The feature map F″, which contains global context and local information, is fed into the coordinate information interaction module to model the implicit relationship between coordinates, and finally the feature map F to be predicted is obtained. out .

[0104] 5) The feature map F to be predicted out Flattened into an embedding vector For each keypoint's embedding vector, a linear projection is applied twice to encode its horizontal and vertical coordinates. The encoding results are then input into a coordinate classifier to decode the keypoint's horizontal and vertical coordinates.

[0105] To further explain the above, the method was trained and tested on three datasets: COCO validation, COCO test-dev, and MPII. The specific dataset processing flow is as follows:

[0106] The human body outline was cropped from the image and then scaled to 256×192 and 388×244. A series of data augmentation strategies were used to enhance the training data: random scaling factors [0.6, 1.4], random rotation [-80, +80], random horizontal flipping, random color enhancement, random occlusion, and additional half-body augmentation. The total training epochs were 300, with a batch size of 256. The Adam optimizer was used, with an initial learning rate of 5e-4. The learning rate decreased to 5e-5 after 170 epochs and to 5e-6 after 200 epochs.

[0107] The experimental results of this embodiment were quantitatively compared with other methods. The human pose estimation method based on local information augmentation and coordinate interaction achieved high-quality detection performance on three datasets: COCO validation, COCO test-dev, and MPII. Specifically, on COCO validation, the keypoint detection quality was significantly improved compared with the Simple baselines model, the High-resolution Network (HRNet-W48) model, the TokenPose model, the HRFormer model, the ViTPose model, the UniFormer model, the SHaRPose model, the HRVMamba model, and the PRTR model (see Table 1).

[0108] Table 1 shows the comparison results with other methods on the COCOvalidation dataset.

[0109]

[0110]

[0111] On COCO test-dev, the keypoint detection quality is significantly improved compared with the High-resolution Network (HRNet-W48), TFPose, TokenPose, DistilPose, CONet, CPN, SimpleBaseline, PRTR, SRPose, HRVMamba, and VMamba models (see Table 2).

[0112] Table 2 shows the comparison results with other methods on the COCOtest-dev dataset.

[0113]

[0114]

[0115] On MPII, the keypoint detection quality is significantly improved compared to high-resolution network models (HRNet-W48), SimpleBaseline models, PRTR models, TranPose models, TokenPose models, SimCC models, RTMPose models, LMFormer models, and CBLCapsNet models (see Table 3). The methods in this invention, based on local information augmentation and coordinate interaction, effectively help the model achieve good keypoint detection performance.

[0116] Table 3 shows the comparison results with other methods on the MPII dataset.

[0117]

[0118]

[0119] Specific visualization results are as follows: Figure 3 , Figure 4 , Figure 5 and Figure 6 As shown in the results, both quantitative and qualitative analysis demonstrate that the prediction results of this invention achieve better detection results in occluded areas compared to the benchmark method. Therefore, this invention employs the aforementioned human pose estimation method based on local information enhancement and coordinate interaction. It utilizes local information enhancement and fine-grained fusion mechanisms to obtain dynamically weighted global context information and local texture information, and achieves cross-axis multi-resolution feature association through a coordinate information interaction module, thereby improving the accuracy of keypoint detection in human pose estimation tasks.

[0120] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0121] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for estimating human pose, characterized in that, include: Obtain the target human bounding box data; The target human body bounding box data is input into a feature extraction network to determine fine-grained features; The feature extraction network uses the Pyramid Transformer as the backbone network, and the intermediate feature maps generated in the backbone network are processed by the local information enhancement module. A fine-grained fusion mechanism is used to perform cross-layer feature correction on fine-grained features to obtain fused features; The coordinate information interaction module is used to perform cross-axis information complementation on the fused features to obtain coordinate information based on the relative positions of various joints and parts of the human body in space; The human posture estimation result is obtained by combining and connecting the coordinate information of key human body points; the key human body points include nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle. The step of inputting the target human body bounding box data into a feature extraction network to determine fine-grained features specifically includes: The target human bounding box data is input into the backbone network and processed through four stages to form a feature pyramid with resolution decreasing from high to low. Each stage uses spatial reduction blocks to progressively compress the feature map resolution and expand the number of channels according to a set value. The resolution of each stage is sequentially equal to the resolution of the input image. and The feature pyramid is input into the local information enhancement module for processing, and the enhanced features are output: f out =Conv(Concat(f1′,f3+f2′)). Where Conv represents convolution processing, Concat represents merging two tensors along the channel layer, f1′ represents small-scale features, f2′ represents medium-to-large-scale features, and f3 represents medium-to-large-scale features refined by context-dependent weights. The method of using a fine-grained fusion mechanism to perform cross-layer feature correction on fine-grained features to obtain fused features specifically includes: The features output by the backbone network and the features output by the local information enhancement module are added along the channel dimension, and then one-dimensional context information is determined through global pooling operation to generate channel descriptors; The channel descriptors are fed into a multilayer perceptron for fusion calculation to obtain fused features: F″=λ1·F4+λ2·f out . Where F″ represents the feature F4 output by the fusion backbone network and the feature f output by the local information enhancement module. out The features after λ1 and λ2 are two two-dimensional weights. The method for determining the two weights is to adaptively assign weights to different granularities based on the global information and channel correlation of different granularity information spaces. The method of using a coordinate information interaction module to perform cross-axis information complementation on the fused features to obtain coordinate information based on the relative spatial positions of various joints and parts of the human body specifically includes: The fused features are processed through three one-dimensional convolutional layers of different scales to obtain the axial information on the horizontal axis and the axial information on the vertical axis. Cross-operation is used to interact the axial information on the horizontal axis and the axial information on the vertical axis to obtain the information F after interaction along the horizontal axis. H Information F after interaction along the vertical axis V And calculate the coordinate information based on the relative positions of various joints and parts of the human body in space: F out =Conv 1×1 (F V )+Conv 1×1 (F H )+F″. Where F″ represents the fusion feature, Conv 1×1 This represents one-dimensional convolution.

2. The human posture estimation method according to claim 1, characterized in that, The process of combining and connecting the coordinate information of key human body points to obtain the human pose estimation result specifically includes: The size output by the coordinate information interaction module is [value]. Coordinate information F out Flattened into an embedding vector Where N represents the number of human body key points, H represents the feature map height, and W represents the feature map width. Represents the real number field; The embedding vector of each human body key point is subjected to two linear projections to encode the horizontal and vertical coordinate information. The encoding results are then input into a coordinate classifier to decode the horizontal and vertical coordinates of the human body key points. The human pose estimation result is obtained by combining and connecting the coordinate pairs of N human body key points.

3. A human pose estimation system, using the method as described in any one of claims 1-2, characterized in that, include: The data acquisition unit is used to acquire the target human body bounding box data; The feature extraction unit is used to input the target human bounding box data into the feature extraction network to determine fine-grained features; the feature extraction network uses a pyramid Transformer as the backbone network, and processes the intermediate feature maps generated in the backbone network through a local information enhancement module; The feature correction unit is used to perform cross-layer feature correction on fine-grained features using a fine-grained fusion mechanism to obtain fused features; The information complementation unit is used to perform cross-axis information complementation on the fused features using the coordinate information interaction module to obtain coordinate information based on the relative positions of various joints and parts of the human body in space. The human posture estimation unit is used to combine and connect the coordinate information of human body key points to obtain human posture estimation results; the human body key points include nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle.

4. An electronic device, characterized in that, The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform the human pose estimation method according to any one of claims 1-2.

5. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the human pose estimation method as described in any one of claims 1-2.

Images