DVS Frame Generation With Speed-Adaptive Pixel Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Dynamic Vision Sensors (DVS) struggle to capture sufficient pixels for stationary or slowly moving objects and excessive redundant pixels for fast-moving objects, leading to unsatisfactory image or frame generation.
Innovation Solution
A method and system that adjust the timespan for integrating DVS pixels based on object movement speed, combining consecutive frames to enhance pixel density and clarity, using deep learning for evaluation and object detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional video coding tools are used for screen content coding, then general video compression is achieved, but text and graphic elements cannot be selectively manipulated and precise editing operations cannot be performed
Solution Approach 1:
The patent segments the screen content into distinct object types (text objects, graphic objects, and video objects) with different coding tools applied to each type. Text objects use text-specific coding tools that preserve editability, graphic objects use graphic-specific tools, and video objects use conventional video coding. This segmentation enables selective manipulation of different content types while maintaining overall system organization.
Solution Approach 2:
The patent creates a universal screen content coding system that incorporates multiple coding tools within a single framework. The system can selectively apply different coding strategies (text, graphic, video) to different portions of the same content, providing multi-functionality that handles various content types with appropriate specialized tools while maintaining a unified coding structure.
2Productivity
If all screen content is coded as video data, then uniform processing is achieved, but file size increases and editing efficiency decreases
Solution Approach 1:
The patent applies different coding qualities and compression strategies to different regions of the screen content based on their type. Text objects receive lossless or near-lossless coding to preserve editability, while video backgrounds may use more aggressive compression. This local differentiation optimizes both file size and editing efficiency for each content type according to its specific requirements.
Solution Approach 2:
The patent changes the coding parameters dynamically based on object type. Text objects use parameters optimized for text preservation (higher bitrates, lossless modes), graphic objects use parameters for vector or raster graphics, and video objects use conventional video parameters. This parameter adaptation reduces overall data volume while maintaining editing efficiency for each object type.
3Quantity of substance
If lossy compression is applied to reduce file size, then storage efficiency improves, but quality degradation occurs and precise editing becomes difficult
Solution Approach 1:
The patent implements dynamic quality control where the compression level is adjusted based on the object type and intended use. Text objects maintain high quality or lossless compression to preserve editing precision, while less critical video portions may use higher compression. The system dynamically selects appropriate compression strategies during encoding to balance file size and editing precision requirements.
Solution Approach 2:
The patent performs preliminary classification of screen content into different object types before applying compression. By identifying text, graphic, and video regions in advance, the system can apply appropriate compression strategies to each type, ensuring that objects requiring precise editing (text and graphics) receive minimal compression while maintaining overall file size efficiency.
4Ease of operation
If screen content is captured and coded as conventional video, then video playback is achieved, but selective access and manipulation of specific elements is not possible
Solution Approach 1:
The patent segments the coded bitstream into distinct object streams or syntax structures that identify different object types (text, graphic, video). This segmentation allows selective access to specific element types during playback or editing by parsing the structured syntax and extracting only the desired object type, improving ease of operation without requiring full video decoding.
Solution Approach 2:
The patent introduces an intermediary syntax structure that sits between the raw video data and the final playback or editing operations. This intermediary layer contains metadata and syntax elements that identify object types, boundaries, and properties, enabling selective manipulation of specific elements while maintaining the overall video coding structure. The intermediary syntax acts as a mediator that facilitates element accessibility.
Data Source
Figure 1A~1C
Figure 2A~2B
Figure 3A~3B
AI summary
A method and system for generating frames for a DVS is disclosed. The method comprises: receiving DVS pixels of an object from a DVS; initializing a timespan to a timespan value; generating a frame by integrating DVS pixels captured within a timespan; evaluating a moving speed of the object; combining a plurality of consecutive frames into one combined frame when the evaluated moving speed is lower than a first threshold value. The system comprises a DVS configured to capture DVS pixels of an object; and a processor configured to receive the DVS pixels from the DVS and performed the method.