The application provides a human
action recognition and quality evaluation method and
system based on multi-
modal data fusion, the application collects RGB visual data, 3D skeleton data, inertial IMU data and
electromyography sEMG data of human action, and carries out corresponding pretreatment; through
branch parallel
feature extraction, RGB visual feature vectors, 3D skeleton feature vectors, IMU inertial feature vectors and sEMG
electromyography feature vectors are obtained respectively; based on the
modal reliability values of each
branch, multi-
modal fusion feature vectors are obtained by dynamically distributing and fusing the weights of the cross-modal attention mechanism to the multi-path features; the
action recognition result is obtained by using a lightweight
action recognition network; and the action recognition result is quality evaluated through the comprehensive action
quality score. The application can not only accurately recognize the action category, but also quantize the action quality from three dimensions of form, dynamics and force, thereby improving the robustness and accuracy of action recognition in a complex scene.