Spherical Video Framing Using Human Gaze Direction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Videos with wide fields of view, such as spherical videos, make it difficult to determine which parts contain interesting views, and manually reviewing them for framing is time-consuming.
Innovation Solution
A system that determines the gaze direction of human subjects within a spherical video and automatically frames the video based on whether the gaze direction passes through the center, positioning a viewing window to include either the subject or the scene they are looking at, using a processor to facilitate this framing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If a wide field of view (spherical video) is used, then the coverage area is improved, but the difficulty of determining interesting views increases
Solution Approach 1:
The system uses gaze tracking technology to automatically determine interesting views by analyzing where human subjects are looking, eliminating the need for manual review. The gaze direction data serves as an objective metric to identify compelling moments and regions within the spherical video, allowing the system to self-determine framing without human intervention.
Solution Approach 2:
The patent replaces manual visual inspection with automated computational analysis. By substituting the mechanical process of manually reviewing video frames with electronic gaze direction detection and processing algorithms, the system achieves automatic identification of interesting views while maintaining the comprehensive coverage of spherical video.
2Measurement precision
If manual review of video is performed to determine framing, then the accuracy of framing selection is improved, but the time consumption increases
Solution Approach 1:
The system incorporates gaze direction data as feedback to automatically adjust and determine the appropriate framing. By continuously monitoring where subjects look and using this feedback information, the system real-time identifies interesting views and determines optimal framing parameters without requiring time-consuming manual analysis.
Solution Approach 2:
The gaze tracking system performs preliminary analysis of viewer attention patterns before final framing decisions are made. By pre-processing the gaze direction data to identify trends and patterns of interest, the system prepares framing recommendations in advance, reducing the overall time required for framing selection while maintaining high accuracy.
3Productivity
If automatic framing based on gaze direction is implemented, then the productivity is improved, but the device complexity increases
Solution Approach 1:
The system integrates multiple functions into a unified gaze tracking and framing solution. The same gaze direction detection mechanism serves multiple purposes: identifying interesting views, determining framing parameters, selecting key moments, and generating video summaries. This multi-functionality increases productivity while managing complexity by consolidating operations rather than adding separate systems.
Data Source
AI summary
A spherical video depicting a scene including one or more human subjects is obtained. Gaze direction(s) of the human subjects are used to determine how the spherical video will be framed for presentation. Based on the gaze direction(s) passing through a center of the spherical video, the spherical video is framed to include the spatial extent that depicts the human subject(s). Based on the gaze direction(s) not passing through the center of the spherical video, the spherical video is framed to include the spatial extent that depicts a portion of the scene looked at by the human subject(s).