Data-driven real-time hand action evaluation method based on RGB video

Abstract

The invention discloses a data-driven real-time hand action evaluation method based on an RGB video, and belongs to the field of human behavior analysis based on video processing. The method comprisesa hand posture estimation unit and an action evaluation unit. The hand posture estimation unit is used for extracting hand key point coordinates from the frame of images; the action evaluation unit is used for predicting scores of hand action quality and giving suggestions on how to improve the scores; gesture estimation and organization are carried out through a method based on deep learning, and action quality evaluation is carried out. The matching accuracy of the extracted features and the details of the human hand in the real scene in the continuous changing process of the visual angle of the camera is improved. The calculation / operation efficiency of overall action recognition and evaluation can be improved, real-time virtual reconstruction of hand actions is achieved, real-time andaccurate evaluation can be conducted on the human hand actions, and the accuracy and robustness of overall action evaluation are improved. The invention can be widely applied to the fields of vision-based hand posture estimation, action quality evaluation methods and the like.

Description

technical field [0001] The invention belongs to the field of human behavior analysis based on video processing, in particular to a real-time evaluation method for hand movements based on RGB video. Background technique [0002] The rapid development of the field of computer vision in recent years has led to many reliable methods for object detection and action recognition from images and videos. To this end, the academic community has gradually begun to explore the field of video-based human motion quality assessment. [0003] At present, a lot of progress has been made in the macroscopic movement of the human body. [0004] In the paper "Assessing the Quality of Actions" (Hamed Pirsiavas, Carl von Derik and Antonio Toraba, European Conference on Computer Vision 2014. Springer International Press, 556-571. HamedPirsiavash, Carl Vondrick, and Antonio Torralba. 2014. Assessing the Quality of Actions. In 2014 European Conference on Computer Vision (ECCV). Springer Internation...

AI Technical Summary

Problems solved by technology

Aiming at the research gap of hand motion evaluation in the field of human behavior analysis based on video processing, the problem of corresponding error between features and real physical parts in human motion evaluation, and the real-time problem of motion evaluation system, a real-time hand motion based on RGB video is proposed. Recognition and evaluation methods to solve the matching accuracy between the extracted features and the details of the human hand in the real scene during the continuous change of the camera angle of view, and at the same time improve the calculation / operation efficiency of the overall action recognition and evaluation, and realize real-time hand movement Virtual reconstruction, capable of real-time and accurate assessment of human hand movements, improving the accuracy and robustness of the entire movement assessment

Images