Event-Centric Timelapse Video Generation via Neural Network Frame Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional internet-connected/cloud-enabled cameras require significant CPU cycles on cloud servers to decode and transcode video streams for timelapse videos, and users face difficulty in finding important details due to fixed frame rates that do not utilize all captured video data effectively.
Innovation Solution
Implementing an event/object-of-interest centric timelapse video generation on camera devices using neural network input, which enables event detection and adaptive frame rate adjustment, allowing for selective encoding of video frames based on detected events, thereby prioritizing meaningful content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If cloud servers decode and transcode video streams for timelapse videos, then timelapse videos can be generated, but significant CPU cycles are consumed on cloud servers
Solution Approach 1:
The system performs preliminary actions by detecting events and selecting important video frames in real-time during video capture, rather than processing all frames afterward. This pre-selection of meaningful frames reduces the computational burden during timelapse generation, as only selected frames need to be processed and displayed.
2Ease of operation
If fixed frame rate is used for timelapse videos, then video playback is simplified, but important details are difficult to find and normal display speed frames are not available when needed
Solution Approach 1:
The system dynamically adjusts the frame rate based on detected events. When an event is detected, the system switches to normal display speed to capture and show all important details. When no event is present, it returns to timelapse mode for efficient playback. This dynamic adjustment ensures both ease of operation and preservation of important information.
Solution Approach 2:
The system applies different quality levels to different time segments of the video. During event periods, full-resolution normal-speed frames are preserved and displayed. During non-event periods, timelapse frames are used. This local differentiation ensures important details are not lost while maintaining efficient playback for routine periods.
3Loss of information
If all video data is captured at normal speed, then complete video information is available, but computational resources are required for processing and storage
Solution Approach 1:
The system extracts only the essential information by detecting events and selecting specific video frames that contain important details. Instead of processing and storing all video data, the neural network identifies and extracts only the meaningful moments, significantly reducing computational and storage requirements while maintaining video information completeness.
Data Source
AI summary
An apparatus including an interface and a processor. The interface may be configured to receive pixel data generated by a capture device. The processor may be configured to generate video frames in response to the pixel data, perform computer vision operations on the video frames to detect objects, perform a classification of the objects detected based on characteristics of the objects, determine whether the classification of the objects corresponds to a user-defined event and generate encoded video frames from the video frames. The encoded video frames may be communicated to a cloud storage service. The encoded video frames may comprise a first sample of the video frames selected at a first rate when the user-defined event is not detected and a second sample of the video frames selected at a second rate while the user-defined event is detected. The second rate may be greater than the first rate.