The invention discloses an efficient supervised action segmentation method based on frame-level and action-level double-layer time modeling, which relates to the technical field of
computer vision, and comprises the steps of learning a frame relationship by using a
convolutional neural network, learning an action dependency relationship by using a converter, and carrying out communication between the two relationships through a cross-self-attention mechanism. A frame
branch captures
temporal information between frames using multiple CNN networks, an action
branch initializes action flags using a multi-head cross-attention and self-attention
transformer, using cross-attention to allow frame features to utilize high-level action dependencies learned by the action
branch while allowing the action flags to access low-level information in the frame branch, and finally, carrying out final prediction through an average value of the features. According to the method, an action mark set with a fixed size can be efficiently learned to clearly represent an action segment, and actions and frame-level time modeling are executed in parallel, so that the method can be superior to a frame-based and two-stage method at lower calculation cost.