Video space-time action positioning method based on progressive attention hypergraph

Abstract

The invention discloses a video space-time action positioning method based on a progressive attention hypergraph. The method comprises the following steps: firstly, sampling a given original video to obtain a frame sequence, and obtaining a target context feature and a video spatio-temporal feature map by using a convolutional neural network; target context features and a time-space relation matrix are obtained through a time-space relation encoder; generating a long-term target first-order feature by using a progressive variable-length window method module; meanwhile, target short-term high-order features are obtained through a hypergraph module with shared attribute constraints and a diffusion mechanism; and finally, outputting spatial positions and action categories of all targets at different moments by using a target action regression module. According to the method, the window size can be adaptively adjusted according to the original duration of the action, the target first-order feature consistent with the original duration of the action can be obtained, the potential relation of the target can be captured through the hypergraph module, the target interaction relation can be effectively utilized, and the accuracy of video space-time action positioning is improved.

Description

technical field [0001] The invention belongs to the technical field of computer vision, in particular to the field of action localization in video processing, and relates to a video spatiotemporal action localization method based on a progressive attention hypergraph. Background technique [0002] The rapid rise of the self-media industry has produced massive amounts of video-based multimedia data. Compared with traditional graphic data, video has gradually become the mainstream media form with its rich visual content and intuitive expression. However, massive videos contain a lot of complex scene information, such as a large number of targets and complex actions. Therefore, how to quickly and accurately identify and locate the action categories of all objects from complex scenes has become an important research direction for researchers, the task of Spatio-temporal Action Localization. This task takes as input a long video that is unedited and may contain multiple targets...

AI Technical Summary

Problems solved by technology

[0004] The shortcomings of the above spatio-temporal action localization methods are mainly manifested in the following three aspects: (1) Although the long-term feature library with a fixed window size can capture the first-order relationship of long-term targets well, for short-term actions, the long-term feature Excessively large time ranges in the library will lead to the extraction of context-independent features by the model, thereby reducing the accuracy of short-duration action feature representation; (2) The impact of high-order relationships on the identification of action categories at the current moment decreases as the time interval increases, However, its computational cost increases with the increase of the time interval. Therefore, it is difficult to construct a long-term target high-order relationship to meet the high real-time requirements of the model; (3) the traditional graph structure can only represent the characteristics of the paired relationship, and it is difficult to describe the complex Diverse target high-order relations

Therefore, it is urgent to design a method that can adaptively adjust the window size according to the original action duration in order to solve the problem of low short-term action confidence caused by the inaccurate capture range of the target first-order relationship and the problem of high computational overhead caused by the unreasonable description of the target high-order relationship. , and can correctly reflect the high-order relationship of the target's spatio-temporal action localization method

Images