Video Action Detection Using Contour Sequences and RNN
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision methods for action detection in videos are inefficient due to time-consuming computations and high memory requirements, making them impractical for real-world applications.
Innovation Solution
The use of a sequence of contour images computed from video frames to represent motion information, combined with a deep neural network with recurrent connections and a multi-stream recurrent neural network (RNN) that includes Long Short-Term Memory (LSTM) layers, to detect actions efficiently in terms of time and memory, while spatially localizing actions within video frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional approaches use hand-crafted features such as HOG, MBH, and HOF with dense trajectories or dense optical flow, then action detection accuracy can be achieved, but computational time and memory requirements become excessively high
Solution Approach 1:
The patent extracts only the essential motion information needed for action detection by using contour sequences from a subset of video frames, rather than processing complete dense trajectories or optical flow fields. This selective extraction maintains detection accuracy while dramatically reducing computational burden by focusing only on boundary contour evolution across frames.
Solution Approach 2:
The patent replaces expensive, computationally intensive optical flow calculations with simple contour extraction and matching operations. The contour sequences serve as lightweight, disposable representations of motion that can be quickly computed and discarded after feature extraction, eliminating the need for storing and processing large amounts of intermediate optical flow data.
2Measurement precision
If conventional approaches use hand-crafted features such as HOG, MBH, and HOF with dense trajectories or dense optical flow, then action detection accuracy can be achieved, but memory storage requirements become excessively high
Solution Approach 1:
The patent extracts only the essential motion information needed for action detection by using contour sequences from a subset of video frames, rather than processing complete dense trajectories or optical flow fields. This selective extraction maintains detection accuracy while dramatically reducing computational burden by focusing only on boundary contour evolution across frames.
Solution Approach 2:
The patent replaces expensive, computationally intensive optical flow calculations with simple contour extraction and matching operations. The contour sequences serve as lightweight, disposable representations of motion that can be quickly computed and discarded after feature extraction, eliminating the need for storing and processing large amounts of intermediate optical flow data.
3Measurement precision
If action detection methods process entire video sequences with detailed feature extraction, then detection accuracy improves, but processing speed decreases
Solution Approach 1:
The patent applies periodic sampling to video frame processing by selecting and analyzing contour sequences from a subset of frames rather than processing every frame. This periodic approach maintains detection accuracy by capturing sufficient motion information while significantly improving processing speed through selective frame analysis.
Solution Approach 2:
The patent uses partial action by processing only the essential contour information from selected frames rather than performing complete feature extraction on all frames. This partial processing approach provides sufficient motion data for accurate detection while reducing overall processing time and computational resources required.
Data Source
AI summary
A method and system for detecting actions of an object in a scene from a video of the scene. The video is a video sequence partitioned into chunks, and each chunk includes consecutive video frames. The method including the following elements. Acquiring the video of the scene, wherein the video includes a sequence of images. Tracking the object in the video, and for each object and each chunk of the video, further comprising: determining a sequence of contour images from video frames of the video sequence to represent motion data within a bounding box located around the object. Using the bounding box to produce cropped contour images and cropped images for one or more images in each chunk. Passing the cropped contour images and the cropped images to a recurrent neural network (RNN) that outputs a relative score for each action of interest.