Action recognition method, device, equipment and storage medium

Abstract

The invention discloses an action recognition method, device, equipment and storage medium. The method comprises: determining the shallow features of each video frame according to the key points of the human skeleton of each video frame in the video to be identified; obtaining the image features of each video frame; obtaining the action features of each video frame according to the shallow features and the image features; Input the action features of each video frame into the action recognition model to obtain the action recognition result of each video frame, the action recognition result includes the state and state probability of the action; determine the execution state of the target action of each video frame according to the action recognition result of each video frame . In the embodiment of the present invention, the action feature combines shallow features and image features, which improves the accuracy of action recognition. When determining the execution state of the target action of each video frame, not only based on the action recognition result of the video frame, but also combined with the action recognition results of other video frames, thereby further improving the accuracy of action recognition.

Description

technical field [0001] Embodiments of the present invention relate to computer vision technology, and in particular to an action recognition method, device, equipment and storage medium. Background technique [0002] Action recognition is one of the most challenging research directions in the field of computer vision. It is widely used in the field of mobile entertainment interaction, such as real-time short video production, live real-time interaction and somatosensory games. [0003] In the prior art, the following two methods are usually used to identify actions. Specifically: method 1, input the RGB image and optical flow information of the video into the convolutional neural network to obtain the type of action contained in the video; method 2, use the human body The bone key point recognition algorithm obtains the key points of human bones in each video frame, and inputs the key points of human bones into the convolutional neural network to obtain the action types cont...

AI Technical Summary

Problems solved by technology

[0004] In the process of realizing the present invention, the inventors found that there are at least the following problems in the prior art: for the first method, since only the action type contained in the video can be obtained, the position information and speed information of the key points of the human skeleton cannot be obtained. Therefore, the flexibility and scalability of the above methods are limited

For the second method, since the key point recognition algorithm of the human skeleton is sensitive to light, environment and motion speed, the key points of the human skeleton are often lost or the key points of the human skeleton deviate from the real position. high

Images