Unlock instant, AI-driven research and patent intelligence for your innovation.

Multi-view feature fusion method and system for 3D human body posture estimation

A human body posture and feature fusion technology, applied in neural learning methods, calculations, computer components, etc., can solve problems such as partial occlusion of different viewing angles, failure to construct learning models, and neglect of local spatial correlation, etc., to achieve high flexibility.

Pending Publication Date: 2022-07-15
HUNAN UNIV
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

One of the core difficulties faced by this type of technical solution is: partial occlusion caused by factors such as human body posture and foreground environment in different viewing angles.
This method does not make full use of multi-view image features when inferring 3D human poses, and its performance is highly dependent on the 2D human pose estimator. At the same time, this processing method cannot construct an end-to-end learning model.
Method 2 has the following two important defects: 1) It does not allow the model to select effective features for fusion; 2) It ignores the correlation of the local space of each channel feature map

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-view feature fusion method and system for 3D human body posture estimation
  • Multi-view feature fusion method and system for 3D human body posture estimation
  • Multi-view feature fusion method and system for 3D human body posture estimation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0040] In this embodiment, a multi-view feature fusion method based on a hybrid attention mechanism is disclosed, such as figure 1 shown.

[0041] The main goal of this embodiment is to obtain the position of the 3D human body posture in absolute world coordinates, that is, the set of three-dimensional coordinates of each joint point of the human body posture A specific number is assigned to each joint point, and the reconstructed joint points are connected in sequence to form a three-dimensional human skeleton.

[0042] The steps of the multi-view feature fusion method based on the hybrid attention mechanism of this embodiment are as follows:

[0043] S1. Obtain target images from different perspectives that require attitude estimation;

[0044] During the specific implementation, the target image that needs attitude estimation can be obtained through a device such as a camera, and the camera can be placed at different positions to obtain images of different perspectives. In...

Embodiment 2

[0084] For the constructed multi-view feature fusion model based on hybrid attention mechanism, the example process in practical application is as follows: image 3 , the process is as follows:

[0085] S1. Obtain the data of the original image, and then preprocess the original image data to obtain the preprocessed image;

[0086] S2, performing data enhancement on the preprocessed image to obtain an enhanced image;

[0087] S3. Input the preprocessed image as an input into the model constructed above to obtain the predicted 3D human posture representation, Figure 4 is the visualization result of the model;

[0088] allowable Figure 4 See the comparison effect of the 2D human body posture predicted by the present invention and the ground truth (ground truth), and the predicted 3D human body posture;

[0089] S4. Visually display the output on the user's mobile phone or computer screen.

Embodiment 3

[0091] The present invention also provides a computer system, including a memory, a processor, and a computer program stored in the memory and running on the processor, where the processor implements the steps of any of the foregoing embodiments when the processor executes the computer program.

[0092] In summary, the present invention learns the channel correlation of multi-view depth feature sets through SENet, uses a neural network module to learn the local space correlation of channel feature maps, and generates weight features in the form of learning masks for each channel feature map element, And it is integrated into a unified 3D human posture representation according to the perspective, which has the characteristics of self-adaptation and high flexibility.

[0093] The invention can solve the "partial occlusion" problem in the technical solution of 3D human posture estimation with the idea of ​​feature fusion, and can be easily embedded into the multi-view 3D human pos...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a multi-view feature fusion method and system for 3D (three-dimensional) human body posture estimation. The method comprises the following steps: acquiring target images of different views needing posture estimation; after a target image is subjected to image preprocessing, the target image is input into the trained MVP-att posture estimation model, and a 3D human body posture estimation result is output; the MVP-att attitude estimation model is obtained by training through the following steps: sampling a plurality of target images input in a multi-view manner through an encoder, and extracting a plurality of depth feature maps expressed by 2D human body postures of single views; the input feature conversion module is used for realizing decoupling of depth feature maps of multiple views and camera postures; and a multi-view feature fusion mechanism module based on a mixed attention mechanism is used to automatically select effective depth features from the decoupled depth features, and the effective depth features are fused into a unified 3D human body posture representation according to view angles. According to the invention, any number of multi-view depth features can be aggregated into 3D human body posture representation.

Description

technical field [0001] The invention relates to the field of 3D human body posture estimation, in particular to a multi-view feature fusion method and system for 3D human body posture estimation based on a mixed attention mechanism. Background technique [0002] Multi-view 3D human pose estimation is a hot branch of research in the field of computer vision with the rise of deep learning algorithms in recent years. As a basic technology, it serves a wide range of downstream applications such as film and television animation production, virtual reality, and medical rehabilitation. The traditional marker-based optical motion capture technology has achieved excellent results in the problem of 3D human pose estimation, but the deployment process of this solution is cumbersome and the site requirements are harsh, which greatly limits the use of 3D human pose estimation in virtual reality, Further promotion of low-precision, high-flexibility scenarios such as action analysis. [0...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06V10/80G06K9/62G06V40/20G06V20/64G06V10/82G06N3/08
CPCG06N3/08G06N3/045G06F18/253
Inventor 肖德贵魏钰麒李健芳
Owner HUNAN UNIV