Salient Event Detection Using Sensor-Guided Video Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated detection of salient events in video images is hindered by high computational complexity, particularly when analyzing content from multiple cameras, which can exceed the resources of mobile terminals and prolong processing time.
Innovation Solution
A method and apparatus that focus analysis on images captured at specific orientations where salient events are likely to occur, using sensor information to define these orientations and filter images within a predefined angular distance threshold, thereby reducing computational resources and false positives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If automated detection of salient events is performed on video images from multiple cameras, then the detection accuracy and event detection capability are improved, but the computational complexity increases significantly
Solution Approach 1:
The patent segments the video analysis process into two distinct phases: (1) a low-complexity pre-screening phase that uses simplified features to identify candidate segments containing potential salient events, and (2) a high-accuracy analysis phase that applies complex detection algorithms only to these candidate segments. This segmentation allows the system to maintain high event detection accuracy while significantly reducing overall computational complexity compared to analyzing all video content with full detection algorithms.
Solution Approach 2:
The patent applies partial action by performing comprehensive salient event detection only on a subset of video segments (candidates identified as potentially containing salient events) rather than analyzing all video content from multiple cameras with full detection complexity. This selective application of detection resources maintains reliability for actual events while reducing overall computational burden.
2Reliability
If comprehensive visual analysis is performed on all video content to detect salient events, then the detection accuracy is improved, but the processing time increases significantly
Solution Approach 1:
The patent performs preliminary action by executing a fast pre-screening analysis on video segments before applying the comprehensive salient event detection algorithm. This preliminary step identifies candidate segments that are likely to contain salient events based on simple temporal and motion features, allowing the system to skip detailed analysis of segments that clearly do not contain events, thereby significantly reducing total processing time while maintaining detection accuracy.
Solution Approach 2:
The system applies full detection accuracy only to candidate segments identified through preliminary screening, rather than performing comprehensive analysis on all video content. This partial application of detailed analysis reduces processing time significantly while maintaining high salient event detection accuracy for actual events.
3Reliability
If full computational resources are allocated to salient event detection on mobile terminals, then the detection accuracy is improved, but the energy consumption and hardware requirements exceed mobile terminal capabilities
Solution Approach 1:
The patent segments the detection workload into a lightweight pre-screening stage that can be executed on mobile terminals with limited resources, and a more intensive analysis stage applied only to candidate segments. This segmentation enables mobile terminals to perform meaningful salient event detection by concentrating computational energy only on segments that require detailed analysis, rather than attempting to process all video content with full detection algorithms.
Solution Approach 2:
The system applies comprehensive detection algorithms only to a subset of candidate segments identified through preliminary screening, rather than allocating full computational resources to analyze all video content. This selective approach enables mobile terminals to achieve improved event detection accuracy for actual salient events while keeping overall energy consumption within device capabilities.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method, apparatus and computer program product are provided to identify one or more salient events from an analysis of one or more images in an efficient and accurate manner. In this regard, the method, apparatus and computer program product may limit the visual analysis of the images to only a subset of the images that are determined to be potentially relevant based upon sensor information provided by one or more sensors carried by the image capturing device. In the context of a method, one or more images that are captured by an image capturing device are identified to be a salient video segment based upon sensor information provided by one or more sensors carried by the image capturing device. The method also includes identifying one or more salient events based upon an analysis of the one or more images of the salient video segment.