Weak supervision time sequence behavior positioning method

Abstract

The invention provides a weak supervision time sequence behavior positioning method. The method comprises the following steps: acquiring an embedded feature of a video clip; classifying the embedded feature, introducing a non-action class, and obtaining an initial class activation sequence corresponding to the embedded feature; constructing an action attention module and a background attention module according to an attention mechanism, constructing a context attention module by using the difference between the action attention module and the background attention module, and weighting the initial class activation sequence based on the three attention values to obtain an action instance class activation sequence, a background class activation sequence and a context class activation sequence; executing a top-k policy on the action instance activation sequence to obtain a video level classification score, wherein the action class of which the score is greater than a threshold value is the action class involved in a to-be-positioned video; and for the involved class, applying a threshold policy to corresponding action class activation sequence and action instance attention, so that behavior positioning is realized. According to the method, the attention model is concisely and effectively defined, action context and background activation is effectively inhibited, and the positioning performance is improved.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to a weakly supervised temporal sequence behavior positioning method. Background technique [0002] In video applications such as video description, video summarization, video understanding, and intelligent surveillance, temporal behavior localization is one of the essential basic tasks. Its goal is to accurately locate the temporal boundaries of all action instances in an uncropped video and determine their action categories. However, an uncropped video usually has a large number of backgrounds and even multiple temporal actions, which poses a huge challenge for temporal behavior location. Due to the rapid development of deep learning, many methods have been proposed and achieved excellent performance. However, most of these methods are implemented under the definition of full supervision, which requires a large collection of videos with precise annotations during trainin...

AI Technical Summary

Problems solved by technology

But a large number of videos created in real life, in addition to the most recognizable video clips, there are also a large number of blurred video clips that cannot be correctly identified

Images