Weak supervision time sequence action detection method based on space-time correlation learning

A technology of spatio-temporal association and action detection, applied in the field of computer vision, can solve problems such as unreasonable, insufficient description of spatio-temporal association characteristic action and background distinction, affect the improvement of action classification and positioning performance, and achieve the effect of promoting accuracy

Inactive Publication Date: 2022-07-29
CHANGZHOU INST OF MECHATRONIC TECH
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Problems in the existing technology: the existing weakly supervised temporal action detection method has incomplete and inaccurate action examples, especially due to the insufficient and unr

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Weak supervision time sequence action detection method based on space-time correlation learning
  • Weak supervision time sequence action detection method based on space-time correlation learning
  • Weak supervision time sequence action detection method based on space-time correlation learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0071] The present invention will be further described below with reference to the accompanying drawings and embodiments. This figure is a simplified schematic diagram, and only illustrates the basic structure of the present invention in a schematic manner, so it only shows the structure related to the present invention.

[0072] like figure 1 , 2 As shown, a weakly supervised time-series action detection method based on spatiotemporal association learning includes the following steps:

[0073] S1. Input video frame sequence where t is the video frame number, T is the total number of frames in the video, v t is the t frame in the video frame sequence number;

[0074] Extract features from video frames through I3D network to generate RGB features and optical flow features Among them, D is the dimension of the feature, and the RGB feature and the optical flow feature are spliced, and finally the video feature is obtained as T is the sample length of the video.

[0075...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the technical field of computer vision, in particular to a weak supervision time sequence action detection method based on space-time correlation learning, which comprises the following steps: S1, extracting features from video frames through an I3D network; s2, constructing a dynamic space graph network structure for the video to obtain video space features; s3, constructing a one-dimensional time sequence convolutional network to obtain video time sequence features; s4, fusing the time sequence features and the spatial features; s5, using action-background attention mechanisms, namely action attention and background attention, which are respectively used for pooling original video features; s6, predicting a class activation sequence of space-time correlation of actions and backgrounds in the video, predicting an action activation sequence or a background activation sequence in the video, and respectively obtaining three classification losses; s7, calculating a total loss function; and S8, using the trained model for action detection. According to the method, the problem that the action example is incomplete and inaccurate in the existing weak supervision time sequence action detection method is solved.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to a weakly supervised time sequence action detection method based on spatiotemporal correlation learning. Background technique [0002] Video sequence action detection is one of the important research topics in the field of computer vision and multimedia, and has a very wide range of applications in the fields of autonomous driving, human-computer interaction, and patient monitoring. The task is to analyze and understand the video in the actual scene. The purpose is to detect the type of action performed by the characters in the video and the start and end time of the action, so that the computer can better detect the activities in the video instead of manual labor, reducing expensive manpower and material costs. There are three main types of video time-series action detection: fully supervised, unsupervised and weakly supervised. The fully supervised time-series action d...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06V40/20G06V20/40G06K9/62G06V10/764G06V10/82G06N3/04G06N3/08
CPCG06N3/08G06N3/047G06N3/045G06F18/2415
Inventor 夏惠芬詹永照朱斌
Owner CHANGZHOU INST OF MECHATRONIC TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap