Human body posture estimation method and system, human body posture estimation model

Abstract

The present invention relates to the field of machine vision, and provides a method and system for estimating human posture, and a human posture estimation model, wherein the method comprises: collecting a static image of a human body; processing the static image of a human body based on a pre-built posture estimation model, wherein the posture estimation model comprises a multi-view feature module for feature calibration and enhancement, a hierarchical feature fusion module for feature fusion, and a multi-scale deep supervision module, wherein the multi-scale deep supervision module comprises several branches, and the multi-scale deep supervision module comprehensively calculates the loss function of the posture estimation model based on the loss values ​​of the several branches; and determines the human posture corresponding to the static image of the human body based on the feature point map output by the posture estimation model. The method is used to solve the defects of the posture estimation method in the prior art in that the use scenarios are limited and the performance is poor. The scheme of the present invention can improve the performance, the accuracy of the reconstruction results, and the detail performance.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a human body posture estimation method and system, and a human body posture estimation model. Background Art

[0002] Human posture estimation is to distinguish the position, angle and scale relationship of various parts of the human body in a static image containing a person.

[0003] However, in order to improve the performance of the model, the human posture estimation method in the prior art needs to stack a large number of feature extraction modules and continuously increase the network depth, which requires a large amount of computing resources, which may limit its application in real-time or resource-constrained scenarios. In addition, the feature fusion method in the prior art cannot fully utilize the information from different scales and perspectives, resulting in limited performance of the results. If the input image quality is poor or there are problems such as occlusion, the performance of the posture estimation method in the prior art may be significantly affected. Summary of the invention

[0004] The present invention provides a human body posture estimation method and system, and a human body posture estimation model, so as to solve the defects of the posture estimation method in the prior art in that the usage scenarios are limited and the performance is poor. The scheme of the present invention can improve the performance and improve the accuracy and detail expression of the reconstruction results.

[0005] The present invention provides a method for estimating a human body posture, comprising:

[0006] Collect static images of human body;

[0007] Processing a static image of a human body based on a pre-built posture estimation model, wherein the posture estimation model includes a multi-view feature module for feature calibration and enhancement, a hierarchical feature fusion module for feature fusion, and a multi-scale deep supervision module, wherein the multi-scale deep supervision module includes several branches, and the multi-scale deep supervision module comprehensively calculates a loss function of the posture estimation model based on loss values ​​of the several branches;

[0008] Based on the feature point map output by the posture estimation model, the human posture corresponding to the human static image is determined.

[0009] According to the human body posture estimation method provided by the present invention, the posture estimation model includes a plurality of multi-view feature modules, a plurality of hierarchical feature fusion modules and a plurality of multi-scale deep supervision modules, and the plurality of multi-view feature modules, the plurality of hierarchical feature fusion modules and the plurality of multi-scale deep supervision modules are arranged in any order to form the posture estimation model.

[0010] According to the human body posture estimation method provided by the present invention, the posture estimation model includes a first part, a second part and a third part which are connected in series in sequence, wherein the first part is connected in series in sequence with two multi-view feature modules, a hierarchical feature fusion module and a multi-scale depth supervision module;

[0011] The second part sequentially connects four multi-view feature modules, a hierarchical feature fusion module and a multi-scale deep supervision module;

[0012] The third part connects two multi-view feature modules, a hierarchical feature fusion module and a multi-scale deep supervision module in series.

[0013] According to the human body posture estimation method provided by the present invention, the multi-view feature module is used for:

[0014] Split the input into a residual branch and a feature extraction branch;

[0015] For the feature extraction branch, the high-resolution features are scaled and matched with the low-resolution features; weighted based on the attention mechanism of spatial and channel dimensions; and feature extraction is performed through continuous dilated convolution;

[0016] The residual branch and the feature extraction branch that completes feature extraction are rearranged in the channel dimension.

[0017] According to the human posture estimation method provided by the present invention, the hierarchical feature fusion module includes a fusion module and a multi-view feature weighting module, and the fusion module is used for:

[0018] Perform identity mapping operation on high-resolution branches;

[0019] The low-resolution branch is first modeled for local features, and then globally modeled through a lightweight transformer module;

[0020] The multi-view feature weighting module is used to:

[0021] The high-resolution branch and the low-resolution branch are weighted by multi-view features to enhance information fusion.

[0022] According to the human posture estimation method provided by the present invention, the multi-scale deep supervision module is used to calculate the loss function, and the loss function includes the loss of several branches of the multi-scale deep supervision module and the loss caused by fusing several branches;

[0023] The multi-scale deep supervision module is used to:

[0024] Independently calculate the difference between each branch and the true label, and perform a weighted summation of the losses of several branches;

[0025] The results of several branch fusions are compared with the true labels to determine the difference between the fused results and the true labels shown.

[0026] The present invention also provides a human body posture estimation system, comprising:

[0027] An image acquisition unit, used for acquiring static images of a human body;

[0028] An image processing unit, used for processing a static image of a human body based on a pre-built posture estimation model, wherein the posture estimation model includes a multi-view feature module for feature calibration and enhancement, a hierarchical feature fusion module for feature fusion, and a multi-scale deep supervision module, wherein the multi-scale deep supervision module includes several branches, and the multi-scale deep supervision module comprehensively calculates a loss function of the posture estimation model based on loss values ​​of the several branches;

[0029] The posture determination unit is used to determine the human posture corresponding to the human static image based on the feature point map output by the posture estimation model.

[0030] The present invention also provides a human body posture estimation model, which is applied to any of the above human body posture estimation methods, comprising:

[0031] Multi-view feature module for feature calibration and enhancement;

[0032] Hierarchical feature fusion module, used for feature fusion;

[0033] Multi-scale deep supervision module for calculating loss functions.

[0034] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned human body posture estimation methods is implemented.

[0035] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned human body posture estimation methods is implemented.

[0036] The present invention also provides a computer program product, comprising a computer program, which implements any of the above-mentioned human body posture estimation methods when executed by a processor.

[0037] In the human posture evaluation method provided by the present invention, a posture estimation model can be pre-constructed and trained. The model can be configured with several branches in a multi-scale deep supervision module, and the loss values ​​of the several branches are integrated to comprehensively calculate the loss value of the posture estimation model, that is, the loss function. In this way, the loss function can be calculated more accurately. Furthermore, when human posture estimation is performed based on the posture estimation model obtained by the training, the performance can be improved, and the accuracy and detail expression of the reconstruction results can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 is a schematic diagram of a flow chart of a method for estimating a human body posture provided by an embodiment of the present invention;

[0040] Figure 2 It is one of the structural schematic diagrams of the posture estimation model provided by the embodiment of the present invention;

[0041] Figure 3 This is the second structural diagram of the posture estimation model provided by the embodiment of the present invention;

[0042] Figure 4 This is the third structural diagram of the posture estimation model provided by the embodiment of the present invention;

[0043] Figure 5 This is a fourth structural diagram of the posture estimation model provided by an embodiment of the present invention;

[0044] Figure 6 is a schematic diagram of the structure of a human body posture estimation system provided by an embodiment of the present invention;

[0045] Figure 7 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The backbone networks used in the human posture estimation methods in the prior art are mostly based on high-resolution output and context feature information aggregation networks, such as HRNet, U-Net, etc., which are based on CNN, Transformer or hybrid architectures for model design, involving feature extraction modules, attention mechanisms, feature fusion, image enhancement, error processing, loss function calculation and other related technologies, and update network parameters through back propagation and optimization algorithms. However, most of the above prior arts have the following defects:

[0048] 1. High computational complexity: In order to improve the performance of the model, a large number of feature extraction modules need to be stacked and the network depth needs to be continuously increased, which requires a large amount of computing resources, which may limit its application in real-time or resource-constrained scenarios;

[0049] 2. Limited feature fusion strategies: Feature fusion methods in existing technologies cannot fully utilize information from different scales and perspectives, resulting in limited performance of the results;

[0050] 3. High requirements for input data quality: If the input image quality is poor or there are problems such as occlusion, the performance of existing technologies may be significantly affected;

[0051] 4. Generalization ability needs to be improved: Existing technologies may perform well in specific data sets or scenarios, but their generalization ability in other unknown environments needs to be further improved.

[0052] Based on this, the solution of the present application provides a human body posture estimation method and system, and a human body posture estimation model as shown below, which can reduce computational complexity, improve feature fusion strategy, enhance model robustness, and improve model generalization ability.

[0053] Figure 1 It is a flow chart of a method for estimating a human body posture provided by an embodiment of the present invention.

[0054] like Figure 1 As shown, this embodiment provides a method for estimating a human body posture, including:

[0055] Step 101, collecting a static image of a human body;

[0056] Step 102, processing a human body static image based on a pre-built posture estimation model, wherein the posture estimation model includes a multi-view feature module for feature calibration and enhancement, a hierarchical feature fusion module for feature fusion, and a multi-scale deep supervision module, wherein the multi-scale deep supervision module includes several branches, and the multi-scale deep supervision module comprehensively calculates a loss function of the posture estimation model based on loss values ​​of the several branches;

[0057] Step 103: Determine the human body posture corresponding to the human body static image based on the feature point map output by the posture estimation model.

[0058] In an exemplary embodiment, the posture estimation model includes several multi-view feature modules, several hierarchical feature fusion modules and several multi-scale deep supervision modules. The several multi-view feature modules, several hierarchical feature fusion modules and several multi-scale deep supervision modules are arranged in any order to form the posture estimation model.

[0059] In the human posture evaluation method provided by the present invention, a posture estimation model can be pre-constructed and trained. The model can be configured with several branches in a multi-scale deep supervision module, and the loss values ​​of the several branches are integrated to comprehensively calculate the loss value of the posture estimation model, that is, the loss function. In this way, the loss function can be calculated more accurately. Furthermore, when human posture estimation is performed based on the posture estimation model obtained by the training, the performance can be improved, and the accuracy and detail expression of the reconstruction results can be improved.

[0060] In addition, the solution of this embodiment also has the following beneficial effects:

[0061] 1. Improved real-time performance: By optimizing the algorithm and network structure, the amount of calculation is reduced, so that the model can run in real-time or near real-time scenarios to meet the needs of actual applications;

[0062] 2. Improved feature map quality: The improved feature fusion strategy can make better use of multi-scale and multi-view information, thereby improving the accuracy and detail expression of human pose estimation tasks;

[0063] 3. Enhanced model robustness: By introducing appropriate regularization techniques, data augmentation, or adversarial training, the model can be made more robust to low-quality inputs and complex environments;

[0064] 4. Improved generalization ability: By improving model design and training methods, the model's generalization ability in different data sets and unknown environments is improved, making it more suitable for various practical application scenarios.

[0065] Figure 2 It is one of the structural schematic diagrams of the posture estimation model provided by the embodiment of the present invention.

[0066] like Figure 2 As shown, in the exemplary embodiment, the posture estimation model includes a first part, a second part and a third part which are connected in series in sequence, wherein the first part is connected in series in sequence with two multi-view feature modules, a hierarchical feature fusion module and a multi-scale deep supervision module;

[0067] The second part sequentially connects four multi-view feature modules, a hierarchical feature fusion module and a multi-scale deep supervision module;

[0068] The third part connects two multi-view feature modules, a hierarchical feature fusion module and a multi-scale deep supervision module in series.

[0069] like Figure 2 As shown, the posture estimation model provided by this embodiment may also include a multi-scale deep supervision module located at the head end of the model, that is, the static image of the human body collected in the scheme of the present application is first subjected to loss calculation by a multi-scale deep supervision module and then subjected to feature enhancement and feature fusion processing by all the multi-view feature modules, hierarchical feature fusion modules and multi-scale deep supervision modules in the first part, the second part and the third part before being output.

[0070] like Figure 2 As shown, the first part of the solution of this embodiment may include two multi-view feature modules, a hierarchical feature fusion module and a multi-scale depth supervision module connected in sequence;

[0071] The second part may include four sequentially connected multi-view feature modules, a hierarchical feature fusion module, and a multi-scale deep supervision module;

[0072] The third part may include two sequentially connected multi-view feature modules, a hierarchical feature fusion module, and a multi-scale deep supervision module.

[0073] In practical applications, the multi-view feature modules in the first, second and third parts mentioned above can all be used for feature calibration and enhancement, the hierarchical feature fusion modules can all be used for feature fusion, and the multi-scale deep supervision modules can all be used to calculate loss values. The purpose of calculating the loss value is to evaluate the performance of the model in comprehensively utilizing multi-scale features. In the solution of this application, the loss value is calculated multiple times based on the multi-scale deep supervision module, so that the model loss can be evaluated at various stages of model training to ensure that the model can effectively learn feature representations at different scales.

[0074] Figure 3 This is the second structural diagram of the posture estimation model provided by the embodiment of the present invention.

[0075] like Figure 3 As shown, in an exemplary embodiment, the multi-view feature module is used to:

[0076] Split the input into a residual branch and a feature extraction branch;

[0077] For the feature extraction branch, the high-resolution features are scaled and matched with the low-resolution features; weighted based on the attention mechanism of spatial and channel dimensions; and feature extraction is performed through continuous dilated convolution;

[0078] The residual branch and the feature extraction branch that completes feature extraction are rearranged in the channel dimension.

[0079] In practical applications, such as Figure 3 As shown in the figure, in the multi-view feature module, the input features can be comprehensively and effectively modeled from three different perspectives: cross-scale, space, and channel. Specifically, in the multi-view feature module, the input features are first split into two parts along the channel dimension, the residual branch and the feature extraction branch. In the feature extraction branch, the high-resolution features are first scaled by the multi-view feature weighting module to match the size of the low-resolution features. Subsequently, the high-resolution features and the low-resolution features are spliced ​​in the channel dimension to form a fused feature map. Next, the spliced ​​features are weighted using the attention mechanism in the spatial and channel dimensions to highlight important information. After that, continuous dilated convolution is used for feature extraction. The receptive field is effectively expanded by adjusting the dilation coefficient, thereby improving the model performance. After feature extraction, the result is scaled back to the original resolution and multiplied element-by-element with the input features to achieve feature calibration and enhancement. Finally, the calibrated features in the feature extraction branch are spliced ​​with the residual branch and rearranged in the channel dimension to complete feature modeling across three perspectives: cross-scale, space, and channel. This process aims to make full use of the information of the input features and improve the expression ability and performance of the model.

[0080] In other words, when constructing a multi-view feature module, the following steps can be included:

[0081] 1. Feature splitting and retention: First, the input features are split into two parts along the channel dimension. One part is directly retained as the residual branch for subsequent feature fusion; the other part enters the feature extraction branch for further processing.

[0082] 2. Scale scaling and feature concatenation: In the feature extraction branch, the high-resolution features are first scaled to match the size of the low-resolution features. Then, the scaled high-resolution features are concatenated with the low-resolution features in the channel dimension to form a fused feature map containing multi-scale information.

[0083] 3. Attention weighting: In order to highlight the important information in the fused feature map, we use the attention mechanism in the spatial and channel dimensions to weight the concatenated features. In this way, the model can pay more attention to the feature parts that are beneficial to the task.

[0084] 4. Continuous dilated convolution and feature extraction: Next, we use continuous dilated convolution to extract weighted features. By properly setting the dilation coefficient, continuous dilated convolution can effectively expand the receptive field, thereby capturing more contextual information and improving the performance of the model.

[0085] 5. Feature calibration and enhancement: After feature extraction, we scale the extracted results back to the original resolution and perform element-by-element multiplication with the input features to achieve feature calibration and enhancement. This step helps preserve the details in the original features and improves the expressiveness of the features.

[0086] 6. Feature fusion and rearrangement: The calibrated features are concatenated with the residual branch and rearranged in the channel dimension. In this way, the feature modeling process across scale, space and channel perspectives is completed.

[0087] Figure 4 This is the third structural diagram of the posture estimation model provided by the embodiment of the present invention.

[0088] Figure 5 This is the fourth structural diagram of the posture estimation model provided by the embodiment of the present invention.

[0089] like Figure 4 and Figure 5 As shown, in an exemplary embodiment, the hierarchical feature fusion module includes a fusion module and a multi-view feature weighting module, and the fusion module is used to:

[0090] Perform identity mapping operation on high-resolution branches;

[0091] The low-resolution branch is first modeled for local features, and then globally modeled through a lightweight transformer module;

[0092] The multi-view feature weighting module is used to:

[0093] The high-resolution branch and the low-resolution branch are weighted by multi-view features to enhance information fusion.

[0094] in Figure 5 F sq (.) indicates the downsampling operation, F ex (.,.) represents channel weighting, F scale (.,.) represents a scaling operation.

[0095] In practical applications, the hierarchical feature fusion module is dedicated to achieving feature fusion between branches at each scale to optimize model performance. Figure 4 and Figure 5As shown, in the hierarchical feature fusion module, in order to efficiently retain the high-resolution detail features and realize lightweight processing, the high-resolution branch can be subjected to an identity mapping operation to ensure lossless transmission of information. For the low-resolution branch, the scheme of this embodiment can adopt a hybrid architecture design of CNN and Transformer. Specifically, local feature modeling can be performed first, and then the global features can be captured through a lightweight Transformer module, that is, global modeling is performed. In this process, the multi-layer perceptron (MLP) is specially designed as a ResMLP with a residual structure to enhance the feature extraction capability. Next, we splice the results of the global modeling with the initial input features to achieve effective fusion of features. Finally, each low-resolution branch and the high-resolution branch of the identity mapping undergo a multi-view feature weighting operation to further enhance the information fusion between the hierarchical features. Through this strategy, the hierarchical feature fusion module not only reduces the computational complexity, but also successfully achieves effective modeling of long-distance relationships while paying attention to local details, thereby significantly improving the feature expression capability of the model.

[0096] The solution of this embodiment also provides a strategy for constructing a hierarchical feature fusion module, including:

[0097] 1. Identity mapping of high-resolution branches: In order to efficiently retain high-resolution detail feature information and achieve lightweight processing, an identity mapping operation is performed on the high-resolution branch to ensure lossless transmission of information to the subsequent fusion stage.

[0098] 2. Hybrid architecture design for low-resolution branches: For low-resolution branches, a hybrid architecture design of CNN and Transformer is adopted. First, CNN is used to model local features, and then a lightweight Transformer module is used to capture global features, that is, global modeling is performed. This hybrid architecture helps the model to effectively model long-distance relationships while paying attention to local details.

[0099] 3. Feature fusion and weighted operation: Based on the hybrid architecture, we concatenate the results of global modeling with the initial input features to achieve effective feature fusion. Then, each low-resolution branch and the high-resolution branch of the identity map undergo a multi-view feature weighting operation to further enhance the information fusion between hierarchical features. This weighted operation helps the model better utilize feature information from different scales and perspectives.

[0100] In an exemplary embodiment, the multi-scale deep supervision module is used to calculate a loss function, and the loss function includes the losses of several branches of the multi-scale deep supervision module and the losses caused by fusing several branches;

[0101] The multi-scale deep supervision module is used to:

[0102] Independently calculate the difference between each branch and the true label, and perform a weighted summation of the losses of several branches;

[0103] The results of several branch fusions are compared with the true labels to determine the difference between the fused results and the true labels shown.

[0104] in Figure 2 Loss fuse represents the loss function after fusion, as well as Represent the loss function of each branch respectively.

[0105] In practical applications, during the training process of the posture estimation model, a multi-scale deep supervision strategy can be used to calculate the loss function to ensure that the model can effectively learn feature representations at different scales. In this embodiment, the loss function can be shown as follows:

[0106]

[0107] in represents the loss of four branch features, l fuse and the above Loss fuse Consistent, indicating the loss after fusion of the four branch features, and w fuse Represents the weight corresponding to each loss term.

[0108] Specifically, the loss function consists of two parts: one is the loss of the four branch features l branch , and the second is the loss l after integrating the features of these four branches fuse For each branch feature loss l branch In the solution of this embodiment, the difference between each scale branch and the corresponding true label can be calculated separately, so that each branch can learn effective feature representation independently. By weighted summing the losses of branches of different scales, we can get the total loss of each branch. As for the loss after feature fusion, fuse , we consider comparing the result of fusing the four branch features with the true labels. By calculating the difference between the fused features and the true labels, we can evaluate the performance of the model in comprehensively utilizing multi-scale features.

[0109] In practical applications, after calculating and determining the loss function of the posture estimation model, the model can be further optimized based on the loss function. Specifically, the network parameters can be continuously updated through optimization algorithms such as gradient descent to minimize the loss function L. During the optimization process, we select appropriate hyperparameter settings such as optimization algorithm, learning rate, and regularization strategy to ensure the effectiveness of the training process and the generalization ability of the model.

[0110] The performance of the human pose estimation method provided by the solution of the present application is verified below. During the verification process, experimental verification can be carried out on the COCO, MPII and CrowdPose datasets respectively. The LHRNet model in the following table is the human pose estimation model provided by the solution of the present application. According to the different depths of each branch model, the LHRNet model can be divided into three versions, namely LHRNet-S, LHRNet-B and LHRNet-L. At the same time, the large-scale network and lightweight network in the prior art can be used as the control group for simultaneous experimental verification, which is also carried out on the COCO, MPII and CrowdPose datasets respectively.

[0111] The following Table 1 shows the performance comparison on the COCO validation set:

[0112]

[0113]

[0114] Table 2 below shows the performance comparison on the COCO test set:

[0115]

[0116]

[0117] Table 3 below shows the performance comparison on the MPII validation set, parameters and computational complexity at a depth resolution of 256×256:

[0118]

[0119] Table 4 below shows the performance comparison on the CrowdPose test set:

[0120]

[0121]

[0122] From the above tables, we can see that on the COCO validation set, the smallest variant of LHRNet outperforms the lightweight models (MobileNetv2, ShuffleNetv2, and Small HRNet) by 0.7, 5.4, and 10.1 points, respectively, while requiring significantly fewer parameters and GFLOPs. In particular, the largest LHRNet variant achieves an AP score of 72.4 on the COCO test set, although there is a certain performance gap compared to the large-scale network. It is worth noting that its parameters and GFLOPs are only 16% and 18% of HRNet-W32, and 7% and 9% of DARK, respectively. In addition, on the CrowdPose dataset, LHRNet-B outperforms the comparison models (EfficientHRNet, LitePose, and Scaled-HigherHRNet) with an AP score of 63.6, while taking advantage of lower input resolution, parameter amount, and GFLOPs. This study provides a novel design approach for designing lightweight and efficient feature extraction network architectures, and provides valuable theoretical and technical support for the deployment of lightweight models.

[0123] The human body posture estimation system provided by the present invention is described below. The human body posture estimation system described below and the human body posture estimation method described above can be referenced to each other.

[0124] Figure 6 Schematic diagram of the structure of a human body posture estimation system provided by an embodiment of the present invention.

[0125] like Figure 6 As shown, the human body posture estimation system provided in this embodiment includes:

[0126] An image acquisition unit 601 is used to acquire a static image of a human body;

[0127] An image processing unit 602 is used to process a static image of a human body based on a pre-built posture estimation model, wherein the posture estimation model includes a multi-view feature module for feature calibration and enhancement, a hierarchical feature fusion module for feature fusion, and a multi-scale deep supervision module, wherein the multi-scale deep supervision module includes a plurality of branches, and the multi-scale deep supervision module comprehensively calculates a loss function of the posture estimation model based on loss values ​​of the plurality of branches;

[0128] The posture determination unit 603 is used to determine the human posture corresponding to the human static image based on the feature point map output by the posture estimation model.

[0129] The specific implementation method of the human body posture estimation system provided in this embodiment can be implemented with reference to the above embodiment, and will not be described in detail here.

[0130] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the human body posture estimation method, which includes:

[0131] Collect static images of human body;

[0132] Processing a static image of a human body based on a pre-built posture estimation model, wherein the posture estimation model includes a multi-view feature module for feature calibration and enhancement, a hierarchical feature fusion module for feature fusion, and a multi-scale deep supervision module, wherein the multi-scale deep supervision module includes several branches, and the multi-scale deep supervision module comprehensively calculates a loss function of the posture estimation model based on loss values ​​of the several branches;

[0133] Based on the feature point map output by the posture estimation model, the human posture corresponding to the human static image is determined.

[0134] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0135] On the other hand, the present invention also provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the human body posture estimation method provided by the above methods, the method comprising:

[0136] Collect static images of human body;

[0137] Processing a static image of a human body based on a pre-built posture estimation model, wherein the posture estimation model includes a multi-view feature module for feature calibration and enhancement, a hierarchical feature fusion module for feature fusion, and a multi-scale deep supervision module, wherein the multi-scale deep supervision module includes several branches, and the multi-scale deep supervision module comprehensively calculates a loss function of the posture estimation model based on loss values ​​of the several branches;

[0138] Based on the feature point map output by the posture estimation model, the human posture corresponding to the human static image is determined.

[0139] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the human body posture estimation method provided by the above methods, the method comprising:

[0140] Collect static images of human body;

[0141] Processing a static image of a human body based on a pre-built posture estimation model, wherein the posture estimation model includes a multi-view feature module for feature calibration and enhancement, a hierarchical feature fusion module for feature fusion, and a multi-scale deep supervision module, wherein the multi-scale deep supervision module includes several branches, and the multi-scale deep supervision module comprehensively calculates a loss function of the posture estimation model based on loss values ​​of the several branches;

[0142] Based on the feature point map output by the posture estimation model, the human posture corresponding to the human static image is determined.

[0143] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0144] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A human body posture estimation method, characterized in that: include: Collect static images of human body; Processing the human body static image based on a pre-built posture estimation model, wherein the posture estimation model includes a multi-view feature module for feature calibration and enhancement, a hierarchical feature fusion module for feature fusion, and a multi-scale deep supervision module, wherein the multi-scale deep supervision module includes several branches, and the multi-scale deep supervision module comprehensively calculates the loss function of the posture estimation model based on the loss values ​​of the several branches; Determining a human body posture corresponding to the human body static image based on a feature point graph output by the posture estimation model; The posture estimation model comprises a first part, a second part and a third part which are connected in series in sequence, wherein the first part is connected in series in sequence with two multi-view feature modules, one hierarchical feature fusion module and one multi-scale depth supervision module; The second part sequentially connects four of the multi-view feature modules, one of the hierarchical feature fusion modules and one of the multi-scale depth supervision modules; The third part sequentially connects two of the multi-view feature modules, one of the hierarchical feature fusion modules and one of the multi-scale depth supervision modules; The multi-view feature module is used for: Split the input into a residual branch and a feature extraction branch; For the feature extraction branch, the high-resolution features are scaled and matched with the low-resolution features; weighted based on the attention mechanism of spatial and channel dimensions; Feature extraction through continuous dilated convolution; Rearranging the residual branch and the feature extraction branch that completes feature extraction in the channel dimension; The hierarchical feature fusion module includes a fusion module and a multi-view feature weighting module, and the fusion module is used to: Perform identity mapping operation on high-resolution branches; The low-resolution branch is first modeled for local features, and then globally modeled through a lightweight transformer module; The multi-view feature weighting module is used for: Perform multi-view feature weighting on the high-resolution branch and the low-resolution branch to enhance information fusion; The multi-scale deep supervision module is used to calculate a loss function, where the loss function includes the losses of several branches of the multi-scale deep supervision module and the losses caused by fusing the several branches; The multi-scale deep supervision module is used to: Independently calculate the difference between each branch and the true label, and perform a weighted summation of the losses of several branches; The fused results of the several branches are compared with the true labels to determine the difference between the fused results and the true labels.

2. A human posture estimation system, characterized in that: The method for estimating a human body posture according to claim 1 comprises: An image acquisition unit, used for acquiring static images of a human body; An image processing unit, used for processing the human body static image based on a pre-built posture estimation model, wherein the posture estimation model includes a multi-view feature module for feature calibration and enhancement, a hierarchical feature fusion module for feature fusion, and a multi-scale deep supervision module, wherein the multi-scale deep supervision module includes a plurality of branches, and the multi-scale deep supervision module comprehensively calculates a loss function of the posture estimation model based on loss values ​​of the plurality of branches; The posture determination unit is used to determine the human body posture corresponding to the human body static image based on the feature point map output by the posture estimation model.

3. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the human body posture estimation method as claimed in claim 1 is implemented.

4. A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the human body posture estimation method as claimed in claim 1.

Images