Three-dimensional hand posture estimation and action recognition algorithm based on deep learning

Through the Transformer framework and multi-scale features, the hierarchical mobile window strategy is constructed, combined with the posture smoothing network, the complexity and occlusion problems of three-dimensional hand posture estimation and action recognition in RGB images are solved, and the time dimension utilization of the movement and posture is improved, and the realization of the realization of the action and posture is achieved. High-precision three-dimensional hand posture estimation and motion recognition.

CN119049117APending Publication Date: 2024-11-29GUILIN UNIV OF ELECTRONIC TECH
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202410842652.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-27
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Three-dimensional hand posture estimation and motion recognition based on RGB images face the complexity of hand structure, occlusion problems, labeling difficulties and real-time challenges. The prior art has not fully utilized time information, resulting in low dynamic gesture recognition accuracy.

Method used

Using the Transformer framework, a hierarchical mobile window strategy is constructed through layered ideas and multi-scale features, using inter-frame time information for pose estimation and action recognition, and an pose smoothing network is attached to capture semantic relationships at different time levels, and achieving end-to-end three-dimensional Hand posture estimation and motion recognition.

Benefits of technology

Improve the accuracy and robustness of three-dimensional hand posture estimation and motion recognition, effectively solve the problem of insufficient hand-object occlusion and single-frame judgment, realize a simple end-to-end training framework, and improve the time of movement and posture Dimension utilization enhances the stability and performance of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119049117A_ABST
    Figure CN119049117A_ABST
Patent Text Reader

Abstract

The invention provides a three-dimensional hand posture estimation and action recognition algorithm based on deep learning, and the algorithm comprises the steps: obtaining a public first person hand action data set FPHA, carrying out the preprocessing, taking a video sample according to an experiment condition and a set proportion, and dividing the video sample into a training set and a test set; a three-dimensional hand posture estimation and action recognition model which is based on deep learning and comprises a double-layer transformer and a lightweight posture smoothing network smopnet is established; inputting the processed first person hand action data set training set into a posture estimation and action recognition model according to a set batch by utilizing a hierarchical moving window thought to carry out supervised training, and sending the preprocessed data into a pre-trained ResNet model to carry out shallow feature extraction; the method comprises the following steps: firstly, carrying out two-dimensional hand gesture recognition on the two-dimensional hand gesture, then sending the two-dimensional hand gesture to a first-layer transformer, carrying out gesture estimation and smooth optimization by utilizing coded short-time inter-frame time sequence information, fusing the two-dimensional hand gesture, operation object classification information and a feature map after the first-layer coded time sequence information by a second-layer transformer in a longer time span, and modeling a semantic relationship between the gesture and the action so as to judge the type of the action. Through the method provided by the invention, three-dimensional hand posture estimation with higher precision and action type judgment with higher accuracy can be carried out on the first person hand action video.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Cited By

  • A digital human motion recognition method, device, medium and product

    CN119741759A

  • Time series data classification system based on mixed contrast learning network

    CN120105190A

  • Interactive gesture recognition method based on feature mining and adaptive shielding model

    CN120877392A

  • Robot grabbing detection method and system for disordered scene

    CN121083672A

  • Indoor personnel position and typical attitude monitoring method based on electrostatic detection technology

    CN121804451A