Weak supervision time sequence action detection method based on space-time correlation learning

Abstract

The invention relates to the technical field of computer vision, in particular to a weak supervision time sequence action detection method based on space-time correlation learning, which comprises the following steps: S1, extracting features from video frames through an I3D network; s2, constructing a dynamic space graph network structure for the video to obtain video space features; s3, constructing a one-dimensional time sequence convolutional network to obtain video time sequence features; s4, fusing the time sequence features and the spatial features; s5, using action-background attention mechanisms, namely action attention and background attention, which are respectively used for pooling original video features; s6, predicting a class activation sequence of space-time correlation of actions and backgrounds in the video, predicting an action activation sequence or a background activation sequence in the video, and respectively obtaining three classification losses; s7, calculating a total loss function; and S8, using the trained model for action detection. According to the method, the problem that the action example is incomplete and inaccurate in the existing weak supervision time sequence action detection method is solved.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to a weakly supervised time sequence action detection method based on spatiotemporal correlation learning. Background technique [0002] Video sequence action detection is one of the important research topics in the field of computer vision and multimedia, and has a very wide range of applications in the fields of autonomous driving, human-computer interaction, and patient monitoring. The task is to analyze and understand the video in the actual scene. The purpose is to detect the type of action performed by the characters in the video and the start and end time of the action, so that the computer can better detect the activities in the video instead of manual labor, reducing expensive manpower and material costs. There are three main types of video time-series action detection: fully supervised, unsupervised and weakly supervised. The fully supervised time-series action d...

AI Technical Summary

Problems solved by technology

[0005] Problems in the existing technology: the existing weakly supervised temporal action detection method has incomplete and inaccurate action examples, especially due to the insufficient and unreasonable description of the temporal-spatial correlation features and the distinction between actions and backgrounds, which affects Action classification and localization performance improvements

Images