Human pose estimation method

Abstract

A human pose estimation method includes receiving an image frame, extracting first-stage image features based on the image frame at least in a first stage, extracting second-stage image features based on the first-stage image features at least in a second stage, extracting subsequent-stage image features based on the second-stage image features at least in a subsequent stage, upsampling the subsequent-stage image features, upsampling the second-stage image features, combining the first-stage image features, the upsampled second-stage image features, and the upsampled subsequent-stage image features, and outputting feature maps based on an output of the combining.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to pose estimation, and more particularly to human pose estimation. BACKGROUND

[0002] Human pose estimation detects a set of keypoints representing joints or body parts in images and videos. Joints or body parts are jointly described in a predefined format to depict a human pose. Pose estimation has various applications in motion detection and recognition, pedestrian tracking, augmented reality, and animation, etc. It provides information for understanding human orientation, motion, behavior, and activity, etc. Due to the huge variations in human visual appearance, scale, body movement, and partial occlusion, accurate pose estimation is challenging. Multi-person pose estimation requires detecting multiple humans in an image simultaneously, further complicating the estimation. Currently, human pose estimation cannot achieve multi-person pose estimation on resource-limited devices. SUMMARY

[0003] An example method of human pose estimation includes receiving an image frame, extracting first-stage image features based on the image frame at least in a first stage, extracting second-stage image features based on the first-stage image features at least in a second stage, extracting subsequent-stage image features based on the second-stage image features at least in a subsequent stage, upsampling the subsequent-stage image features, upsampling the second-stage image features, jointly the first-stage image features, the upsam ped second-stage image features, and the upsam ped subsequent-stage image features, and outputting feature maps based on an output of the joint.

[0004] Another example method of human pose estimation includes receiving a feature map by an initial stage, convolving the feature map based on an initial part association map branch of the initial stage, determining an initial part association map based on an output of the initial part association map branch, convolving the feature map based on an initial heat map branch of the initial stage, determining an initial heat map based on an output of the initial heat map branch, jointly the initial part association map branch, the initial heat map branch, and the feature map to output an initial stage joint, receiving the initial stage joint by a refinement stage, convolving the initial stage joint based on a refinement part association map branch of the refinement stage, determining a refinement part association map based on an output of the refinement part association map branch, convolving the initial stage joint by a refinement heat map branch of the refinement stage, and determining a refinement heat map based on an output of the refinement heat map branch.

[0005] Yet another example method of human pose estimation includes receiving a feature map through an initial stage, performing a convolution on the feature map based on a first part affinity map branch of the initial stage, determining a first stage part affinity map based on an output of the first part affinity map branch, jointly the first stage part affinity map branch and the feature map to output a first stage part affinity map joint, performing a convolution on the first stage part affinity map joint to output a second stage part affinity map convolution, determining a second stage part affinity map based on an output of the second stage part affinity map convolution, jointly the second stage part affinity map branch and the feature map to output a second stage part affinity map joint, receiving the second stage part affinity map joint through a refinement stage, performing a convolution on the second stage part affinity map joint to output a third stage part affinity map convolution, determining a third stage part affinity map based on an output of the third stage part affinity map convolution, jointly the third stage part affinity map and the feature map to output a joint third stage part affinity map, performing a convolution on the joint third stage part affinity map to output a refined heat map prediction, and determining a refined heat map based on an output of the refined heat map prediction stage. BRIEF DESCRIPTION OF DRAWINGS

[0006] Figure 1 is a first example system diagram according to one embodiment of the present disclosure;

[0007] Figure 2 is a second example system diagram according to one embodiment of the present disclosure;

[0008] Figure 3 is an example structure according to one embodiment of the present disclosure;

[0009] Figure 4 is an example head net architecture according to one embodiment of the present disclosure;

[0010] Figure 5 is another example head net architecture according to one embodiment of the present disclosure;

[0011] Figure 6 is an example refinement stage replacement unit according to one embodiment of the present disclosure;

[0012] Figure 7 is an example model result according to one embodiment of the present disclosure;

[0013] Figure 8 is a first example method of human pose estimation backbone according to one embodiment of the present disclosure;

[0014] Figure 9 is a second example method of human pose estimation according to one embodiment of the present disclosure; and

[0015] Figure 10This is a third exemplary method for human posture estimation according to one embodiment of the present disclosure. Detailed Implementation

[0016] The embodiments listed below are for illustrative purposes only and are not intended to limit the scope of the apparatus and method. Modifications to equivalent forms of the apparatus and method should be classified within the scope of the claims.

[0017] Certain terms are used in the following description and claims to refer to specific system components. As those skilled in the art will understand, different companies may use different names to refer to components and / or methods. This application is not intended to distinguish between components and / or methods that have different names but the same function.

[0018] In the following discussion and in the claims, the terms "including" and "comprising" are used in an open-ended manner and can therefore be interpreted as meaning "including, but not limited to...". Furthermore, the term "couple" or "couples" is intended to mean either an indirect connection or a direct connection. Thus, if a first device is coupled to a second device, the connection can be either a direct connection or an indirect connection via other devices and connections.

[0019] Figure 1 An exemplary hybrid computing system 100 is depicted, which can be used to implement neural networks associated with the operation of one or more parts or steps of a process. In this example, the processors associated with the hybrid system include a field-programmable gate array (FPGA) 122, a graphics processing unit (GPU) 120, and a central processing unit (CPU) 118.

[0020] CPU 118, GPU 120, and FPGA 122 all possess the capability to provide neural networks. A CPU is a general-purpose processor capable of performing many different functions; its versatility allows it to execute multiple different tasks, but its processing of multiple data streams is limited, and its capabilities relative to neural networks are also limited. A GPU is a graphics processing unit with many small processing cores capable of processing parallel tasks sequentially. An FPGA is a field-programmable device that has the ability to be reconfigured and execute any function that can be programmed into a CPU or GPU in a hard-wired manner. Because FPGA programming is in the form of circuits, it is many times faster than a CPU and much faster than a GPU.

[0021] The system can include other types of processors, such as: an accelerated processing unit (APU), which includes a CPU with on-chip GPU elements; and a digital signal processor (DSP), which is designed for high-speed numeric data processing. Application-specific integrated circuits (ASICs) can also perform the hardwired functions of FPGAs; however, the lead time to design and produce an ASIC is on the order of a year or so, as opposed to the rapid turn-around implementation possible when programming an FPGA.

[0022] The graphics processor unit 120, the central processing unit 118, and the field programmable gate array 122 are connected and connected to the memory interface and controller 112. The FPGA is connected to the memory interface through a programmable logic circuit to memory interconnect 180. This additional device is used because the FPGA operates at very large bandwidth, and to minimize the circuitry of the FPGA used to perform memory tasks. The memory interface and controller 112 is additionally connected to the persistent memory disk 110, the system memory 114, and the read-only memory (ROM) 116.

[0023] Figure 1 The system of FIG. 1 can be used to program and train the FPGA. The GPU can handle unstructured data well and can be used to train. Once the data has been trained, a deterministic inference model is available, and the CPU can program the FPGA using the model data determined by the GPU.

[0024] The memory interface and controller is connected to a central interconnect 124, which is additionally connected to the GPU 120, the CPU 118, and the FPGA 122. The central interconnect 124 is additionally connected to the input and output interface 128 and the network interface 126.

[0025] Figure 2 An exemplary depicts a second hybrid computing system 200 that can be used to implement a neural network associated with the operations of one or more portions or steps of the process 1000. In this example, the processors associated with the hybrid system include a field programmable gate array (FPGA) 210 and a central processing unit (CPU) 220.

[0026] The FPGA is electrically connected to an FPGA controller 212 that is connected to a direct memory access (DMA) 218. The DMA is connected to an input buffer 214 and an output buffer 216, both of which are coupled to the FPGA to buffer input data to the FPGA and buffer data from the FPGA for output, respectively. The DMA 218 includes two first-in-first-out (FIFO) buffers, one for the host CPU and one for the FPGA, which allows data to be written to and read from the appropriate buffer.

[0027] On the CPU side of the DMA is a master switch 228 that transfers data and commands to the DMA. The DMA is also connected to an SDRAM controller 224 that allows data to be transferred to and from the FPGA and CPU 220, which is also connected to an external SDRAM 226 and the CPU 220. The master switch 228 is connected to a peripheral interface 280. A flash controller 222 controls the persistent memory and is connected to the CPU 220.

[0028] Existing multi-person pose estimation methods can be classified into two categories: top-down or bottom-up methods. Top-down methods rely on object detectors to detect human bodies and have a runtime that increases proportionally to the number of human bodies in the image, making real-time use of top-down methods impractical in crowded scenes. Bottom-up methods detect body parts and assemble them together to form individual human bodies, allowing the runtime to be largely unaffected by the number of people in the image. Existing bottom-up methods use resource-intensive models, which do not allow them to be deployed on resource-limited embedded systems to achieve real-time performance.

[0029] The solution presented in this disclosure uses a bottom-up approach that includes a parameter-free representation called Part Affinity Fields (PAFs) to associate body parts with individuals and body part locations in the image, also known as heatmaps. The association of part affinity fields and heatmaps provides two branches of a sequential prediction process.

[0030] The solution presented in this disclosure is a two-dimensional multi-person human pose estimation method that estimates two-dimensional human poses from color or infrared input images. The bottom-up framework for multi-person pose estimation jointly learns body part locations and their association with individuals. The base model is smaller in size, fuses multi-level feature maps, and reduces the number of stages to make the model usable for resource-limited products. The solution presented in this disclosure can use a model for embedded systems that reduces resources by 80% compared to existing models while achieving similar real-time performance.

[0031] The two-dimensional multi-person human pose estimation method presented in this disclosure uses image frames from a monocular camera, such as a red-blue-green (RGB) camera-based system or an infrared (IR) camera-based system, to estimate two-dimensional human poses. The bottom-up framework for multi-person pose estimation jointly learns part locations and their association. The image frames can be a single frame output from a camera or at least one image frame from a video camera.

[0032] Residual neural networks (ResNet) are a model that utilizes pyramidal cells in the cerebral cortex that use skip connections to skip over some layers. One example system uses a ResNet-vl-50 type residual neural network to extract image features from image frames, which is much more lightweight than other models used for bottom-up determination methods. The addition of residual connections allows ResNet to achieve representational capabilities with deeper and more lightweight network structures. The output of the ResNet for classification tasks is downsampled by a factor of 32, which is more suitable for image classification tasks. The coarse feature maps after downsampling can not have enough detail to accurately localize body parts, so various arrangements are investigated.

[0033] One example model includes combining outputs from different layers that include both low-level spatial details and high-level semantic data. As shown in Figure 3 The output of block 1, the output of block 2 after 2x upsampling, and the output of block 4 after 4x upsampling can be concatenated together as a feature map. Experimental results show that this example model has the ability to fuse pyramid features from multiple scales, rather than simply using upsampled or expanded convolutional features at the end of the model.

[0034] Figure 3 A first example model including a ResNet engine 310 is depicted. This example includes a ResNet-vl-50 engine 310 that includes four blocks 312, 314, 316, and 318. Image data in the form of image frames is input into block 1 312, which is fed to block 2 314 and forward fed to a concatenation stage 324. Data from block 2 314 is sent to block 3 316 and forward fed to the concatenation stage 324 after being upsampled by a factor of two at block 322. Block 3 316 receives data from block 2 314 and sends data to block 4 318. Data output by block 4 318 is sent to the concatenation stage 324 after being upsampled by a factor of four in block 320. The concatenated output of block 1, block 2, and block 4 is used to construct a feature map 326.

[0035] Resource-intensive models used by the prior art can be replaced with models that are compatible with limited resources. To fuse image features at multiple scales, the outputs of multiple layers of the model, such as the outputs of block 1, block 2, and block 4 in a ResNet-vl-50 model, are concatenated after being upsampled to capture high-level semantic information and low-level spatial details.

[0036] In the solution proposed in this disclosure, batch normalization parameters can be trained based on a large image dataset. In one example, a pose estimation model is trained using a ResNet engine, with weights pre-trained based on a large image dataset used for initialization. In this example, the model is fine-tuned based on a small image training dataset. The weights in the head structure are self-learned. Freezing the batch normalization parameters in ResNet by using the batch normalization parameters trained based on a large image dataset during training without fine-tuning improves the model performance and outperforms existing bottom-up models.

[0037] Figure 4 An example is depicted with a parallel prediction branch 400 that includes an initial stage 412 and a refinement stage 414.

[0038] In the initial stage 412, the feature map 410 is input to two parallel branches, branch 1 416 and branch 2 420. Branch 1 416 is a part affinity field prediction branch and includes at least one 3x3 convolutional block and at least one 1x1 convolutional block to produce a part affinity field (PAF) map 418. Branch 2 420 is a heatmap prediction branch and includes at least one 3x3 convolutional block and at least one 1x1 convolutional block to produce a heatmap 422. A joint stage 424 receives the feature map 410, the PAF map 418, and the heatmap 422.

[0039] In the refinement stage 414, the output of the joint stage 424 is input to two parallel branches, branch 1 426 and branch 2 430. Branch 1 426 is a part affinity field prediction branch and includes at least one convolutional block C that includes one 1x1 convolutional layer followed by two 3x3 convolutional layers and at least one final 1x1 convolutional stage to produce a part affinity field (PAF) map 428. Branch 2 430 is a heatmap prediction branch that includes at least one convolutional block C that includes one 1x1 convolutional layer followed by two 3x3 convolutional layers and at least one final 1x1 convolutional stage to produce a heatmap 432.

[0040] The head structure of one example model includes at least two parallel prediction branches. One branch predicts the location of body parts in the form of heatmaps. The other branch predicts the association of body parts in the form of part affinity field (PAF) maps. As shown, Figure 4 The network includes an initial stage followed by a refinement stage, as shown. The prediction of heatmaps and PAF maps can be refined after multiple iterations, as implemented with multiple refinement stages, as the location of some body parts can provide contextual information for the location of other body parts. In one example, one initial stage followed by one refinement stage can achieve acceptable results with 80% reduction in resources.

[0041] In another example solution, a sequential structure is used. Since PAFs implicitly include body part position information, PAF refinement can be sufficient for accurate body part estimation. Body part confidence map (CM) refinement, which can also be referred to as a heat map, can be performed in a stage after PAF prediction. Thus, as shown in FIG. 5, this example uses a structure of two PAF initial stages and one PAF refinement stage, followed by a confidence map prediction. Figure 5

[0042] Figure 5 An example is depicted with a sequential prediction branch 500 having an initial stage 512 and a refinement stage 514.

[0043] In the initial stage 512, the feature map 510 is input to a first part association field prediction branch 516. The first part association field prediction branch 516 includes at least one 3x3 convolutional block and at least one 1x1 convolutional block, resulting in a first PAF map 518. A first joint stage 534 receives the first PAF map 518 and the feature map 510 and outputs to a second part association field prediction branch 520. The second part association field prediction branch 520 includes at least one 3x3 convolutional block and at least one 1x1 convolutional block, resulting in a second PAF map 522. A second joint stage 524 receives the second PAF map 522 and the feature map 510 and outputs an initial result to the refinement stage 514.

[0044] In the refinement stage 514, the output of the second joint stage 524 is input to a third part association field prediction branch 526. The third part association field prediction branch 526 includes at least one convolutional block C, which includes one 1x1 convolutional layer followed by two 3x3 convolutional layers and at least one final 1x1 convolutional layer, resulting in a third PAF map 528. A third joint stage 536 receives the third PAF map 528 and the feature map 510 and outputs to a heat map prediction branch 530, which is also referred to as a body part confidence map. The heat map prediction branch 530 includes at least one convolutional block C, which includes one 1x1 convolutional layer followed by two 3x3 convolutional layers and at least one final 1x1 convolutional layer, resulting in a heat map 532.

[0045] In existing models, the convolutional unit C of the refinement stage performs a 7x7 convolution. Large receptive fields can be used for body part prediction to provide contextual information such as spatial distribution of other body parts. In one example of the solution proposed in this disclosure, the convolutional structure includes at least one 1x1 convolutional layer followed by two 3x3 convolutional layers. The last 3x3 convolutional layer uses a 2x dilation to ensure that the receptive field matches the current 7x7 convolution. The solution proposed in this disclosure has a structure that provides similar performance to the original model using half the resources.

[0046] ​Figure 6 An example 600 of resource reduced convolution is depicted. The existing 7x7 convolutional unit can be replaced by a convolution 1x1 layer 610 that connects the input. The output of the convolution 1x1 layer 610 of channel number 128 is fed forward to a multiply-accumulate layer 616. A first 3x3 convolutional layer 612 receives the output of the convolution 1x1 layer 610 and sends an output of channel number 128 to a second 3x3 convolutional layer 614 of channel number 128, dilated by a factor of 2. The multiply-accumulate layer 616 receives the output of the second 3x3 convolutional layer 614 and the output of the 1x1 convolutional layer 610.

[0047] By reducing the number of refinement stages to 1, the head structure of the proposed solution of the present disclosure becomes more compact. Furthermore, the second example approach using a sequential structure of 3 PAF stages followed by 1 confidence map (e.g., heat map) stage can provide comparable performance to the first example approach using a parallel dual-branch structure. In the proposed solution of the present disclosure, the existing 7x7 convolutional unit can be replaced by a 3x3 convolutional structure with residual connections that relies less on resources.

[0048] Figure 7 Results 700 of existing pose prediction based on an existing resource intensive Visual Geometry Group Network (VGGNet) model and the proposed ResNet resource limited model of the present disclosure are depicted. The results tend to show that reducing the model resources by 80% does not result in a decrease in model performance compared to the existing model and can show an improvement in performance of the proposed ResNet resource limited model of the present disclosure.

[0049] Figure 8 An example depicts a human pose estimation method that includes receiving 810 an image frame, extracting 812 first stage image features based on the image frame in at least a first stage, and extracting 814 second stage image features based on the first stage image features in at least a second stage. The method also includes extracting 816 subsequent stage image features based on the second stage image features in at least a subsequent stage, upsampling 818 the subsequent stage image features, and upsampling 820 the second stage image features. The method also includes jointly 822 the first stage image features, the upsampled second stage image features, and the upsampled subsequent stage image features, and outputting 824 a feature map based on the joint output.

[0050] Figure 9An example depicts a human pose estimation method that includes receiving 910 a feature map through an initial stage, convolving 912 the feature map based on an initial part association map branch of the initial stage, determining 914 an initial part association map based on an output of the initial part association map branch, and convolving 916 the feature map based on an initial heat map branch of the initial stage. The method also includes determining 918 an initial heat map based on an output of the initial heat map branch, joining 920 the initial part association map branch, the initial heat map branch, and the feature map to output an initial stage join, and receiving 922 the initial stage join through a refinement stage. The method includes convolving 924 the initial stage join via a refinement part association map branch of the refinement stage, determining 926 a refinement part association map based on an output of the refinement part association map branch, convolving 928 the initial stage join via a refinement heat map branch of the refinement stage, and determining 930 a refinement heat map based on an output of the refinement heat map branch.

[0051] Figure 10 An example depicts a human pose estimation method that includes receiving 1010 a feature map through an initial stage, convolving 1012 the feature map based on a first part association map branch of the initial stage, determining 1014 a first stage part association map based on an output of the first part association map branch, and joining 1016 the first stage part association map branch and the feature map to output a first stage part association map join. The method also includes convolving the first stage part association map join to output a second stage part association map convolution 1018, determining 1020 a second stage part association map based on an output of the second stage part association map convolution, joining 1022 a second stage part association map branch and the feature map to output a second stage part association map join, and receiving 1024 the second stage part association map join through a refinement stage. The method includes convolving 1026 the second stage part association map join to output a third stage part association map convolution, determining 1028 a third stage part association map based on an output of the third stage part association map convolution, joining 1030 the third stage part association map and the feature map to output a joined third stage part association map, convolving 1032 the joined third stage part association map to output a refinement heat map prediction, and determining 1034 a refinement heat map based on an output of the refinement heat map prediction stage.

[0052] Those skilled in the art will appreciate that the various illustrative blocks, modules, elements, components, methods, and algorithms described herein can be implemented as electronic hardware, computer software, or combinations of both. To illustrate the interchangeability of hardware and software, various illustrative blocks, modules, elements, components, methods, and algorithms have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application. Various components and blocks can be arranged differently or eliminated from the configuration described above, or a different set of configurations from those described above can be used, without departing from the spirit of the subject technology.

[0053] It should be understood that the particular order in which the steps of processes disclosed are presented is exemplary. Based on design preferences, it is understood that the specific order of steps in processes can be rearranged, or other specific orders can be utilized. Some steps can be performed simultaneously. The accompanying method claims present elements of the various steps in the example order, and are not meant to be limited to the specific order or hierarchy presented.

[0054] The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. The previous description provides various examples of the subject technology, and the subject technology is not limited to these examples. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown herein, but is to be accorded the full scope consistent with the language claims, wherein reference to an element in the singular is not intended to mean "one and only one" unless specifically so stated, but rather "one or more." Unless specifically stated otherwise, the term "some" refers to one or more. Male gender pronouns, such as his, include female and neutral gender pronouns, such as her and its, and female gender pronouns, such as her, include male and neutral gender pronouns, such as his and its. If headings and subheadings are used, they are for convenience only and do not limit the application. The phrases "configured to," "operable to," and "programmed to" do not mean any specific tangible or intangible modification to the subject, but rather merely mean that the subject is enabled, by way of hardware or software instruction, to perform the recited operation or function. For example, a processor configured to monitor and control operations or components can also mean a processor that is programmed to monitor and control operations or a processor that is operable to monitor and control operations. Likewise, a processor configured to execute code can be interpreted as a processor that is programmed to execute code or a processor that is operable to execute code.

[0055] Phrases such as “aspect” do not imply that a particular aspect is essential to the subject technology, or that such aspect applies across all configurations of the subject technology. Disclosures relating to an aspect can apply to a configuration or one or more configurations. An aspect can provide one or more examples. Phrases such as “aspect” can refer to one or more aspects, and vice versa. Phrases such as “embodiment” do not imply that a particular embodiment is essential to the subject technology, or that such embodiment applies across all configurations of the subject technology. Disclosures relating to an embodiment can apply to an embodiment or one or more embodiments. An embodiment can provide one or more examples. Phrases such as “embodiment” can refer to one or more embodiments, and vice versa. Phrases such as “configuration” do not imply that a particular configuration is essential to the subject technology, or that such configuration applies across all configurations of the subject technology. Disclosures relating to a configuration can apply to a configuration or one or more configurations. A configuration can provide one or more examples. Phrases such as “configuration” can refer to one or more configurations, and vice versa.

[0056] The word “example” is used herein to mean “serving as an example or illustration.” Any aspect or design described herein as “example” is not necessarily to be construed as preferred or advantageous over other aspects or designs.

[0057] Structural equivalents and functional equivalents of elements of various aspects described throughout this disclosure that are known or that will become known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended to be disclosed to the public before the time of the present application, whether or not expressly stated herein. Furthermore, with respect to the use of the terms “include,” “have,” and the like in the specification and / or claims, such terms are intended to be inclusive in a manner similar to the use of the term “comprise” as an open-ended transition word.

[0058] References to “one implementation,” “an implementation,” “some implementations,” “various implementations,” etc., indicate that the particular element or characteristic is included in at least one implementation of the present application. These phrases do not necessarily refer to the same implementation, although they can. It is possible for an implementation to include one or more features while excluding others. The disclosure is not limited to the implementations described herein, but the intent is to cover all changes, modifications, equivalents, and alternatives falling within the scope of the claims.

[0059] It should be understood that the present disclosure teaches only one example of an illustrative implementation and that many variations of the application can be readily devised by those skilled in the art following the teachings herein, and that the scope of the application is to be determined by the appended claims.

Claims

1. A method for human pose estimation, characterized in that, include: Feature maps are received during the initial phase; The feature map is convolved based on the first part association map branch of the initial stage; The first-stage part association diagram is determined based on the output of the first part association diagram branch. The first-stage part association map branch and the feature map are combined to output the first-stage part association map union; The first stage part association graph is convolved to output the second stage part association graph convolution; The second-stage part association map is determined based on the output of the convolution of the second-stage part association map. The second-stage part association map and the feature map are combined to output a combined second-stage part association map; The second stage of part association diagrams is received during the refinement stage; The second-stage part association maps are convolved together to output the third-stage part association map convolution; The third-stage part association map is determined based on the output of the convolution of the third-stage part association map. The third-stage part association map and the feature map are combined to output a combined third-stage part association map; The combined third-stage part correlation map is convolved to output a refined heatmap prediction; as well as The refined heatmap is determined based on the output of the refined heatmap prediction stage.

2. The human posture estimation method according to claim 1, characterized in that, The feature map is determined from the bottom up.

3. The human posture estimation method according to claim 1, characterized in that, The heat map predicts the location of body parts, and the body part association map predicts the association of the body parts.

4. The human posture estimation method according to claim 1, characterized in that, The convolution of the feature map was performed using ResNet-v1-50.

5. The human posture estimation method according to claim 1, characterized in that, Also includes: The body part location reliability map is determined based on the detailed part association map.

6. The human pose estimation method according to claim 1, characterized in that, The first stage, the second stage, and the third stage are combined using the residual connections of the feature map.

7. The human posture estimation method according to claim 1, characterized in that, The convolution of the feature map includes at least one 3x3 convolution and at least one 1x1 convolution.

Images