3D Scene Rendering via Gaze-Driven Fixation Expansion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D computer graphics systems are not naturalistic, limiting the user's ability to explore and interact with 3D scenes due to their reliance on artificial behaviors and linear perspective, which results in narrow fields of view, distortions, and discomfort, as they fail to accurately represent the human visual experience.
Innovation Solution
A method that uses eye and head tracking sensors to modify the 3D scene geometry by computationally isolating a fixation region and expanding it while compressing the peripheral region, adjusting based on the user's distance from the display and head movement, to emulate a natural field of view and maintain visual stability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If linear perspective geometry is used to render 3D scenes, then the rendering method is simple and computationally efficient, but the field of view becomes narrow (40-60°) and distortions occur at edges
Solution Approach 1:
The patent changes the geometric parameters of the projection system from conventional linear perspective to a modified projection geometry that allows for wider field of view (up to 180° or more) while maintaining computational efficiency. This involves adjusting the projection equations and camera model parameters to achieve naturalistic rendering without excessive complexity
Solution Approach 2:
The patent introduces dynamic adjustment of the field of view and projection parameters based on user eye tracking data. The system dynamically modifies the rendering parameters to match the user's natural visual exploration patterns, allowing the field of view to expand and contract naturally as the user looks around
2Productivity
If linear perspective is used to render 3D scenes, then the rendering process is computationally efficient, but objects become excessively magnified or minified causing distorted appearance
Solution Approach 1:
The patent modifies the projection parameters and geometric transformation equations to preserve object proportions and scale relationships across the entire field of view. This involves changing the projection model to avoid the extreme magnification/minification effects of linear perspective while maintaining computational efficiency through optimized algorithms
Solution Approach 2:
The patent applies different local scaling and distortion corrections to different regions of the image based on their distance from the center of projection. This ensures that objects throughout the wide field of view maintain their natural appearance without excessive magnification or minification, while the center region maintains high visual accuracy
3Adaptability or versatility
If the virtual camera moves to emulate eye motion, then the viewer's perspective changes, but the motion appears unnatural and uncomfortable
Solution Approach 1:
The patent uses eye tracking sensors to automatically detect the user's gaze direction and fixation points, then uses this data to control the virtual camera position and orientation. This eliminates the need for manual camera control and makes the system adapt automatically to the user's natural viewing behavior, providing both perspective adaptability and viewing comfort
Solution Approach 2:
The patent implements a feedback loop where eye tracking data continuously informs adjustments to the virtual camera parameters. The system monitors user eye movements and automatically adjusts the camera position, field of view, and projection parameters to match the user's natural visual exploration patterns, creating a comfortable and adaptive viewing experience
4Device complexity
If conventional 3D graphics methods are used, then the system complexity is low, but the visual representation does not match human natural perception
Solution Approach 1:
The patent changes the fundamental geometric parameters of the rendering system from linear perspective to a modified projection model that better matches human visual perception. This includes adjusting the projection equations, camera model, and field of view parameters to achieve naturalistic rendering while keeping the overall system architecture simple and maintainable
Solution Approach 2:
The patent replaces conventional mechanical interaction devices (mouse, joystick) with eye tracking-based control. This substitution allows the system to automatically adapt to user viewing behavior without requiring complex manual control mechanisms, thereby improving perceptual accuracy while actually reducing system complexity
Data Source
AI summary
A method of modifying an image on a computational device is disclosed. The method comprises providing image data representative of at least a portion of a three-dimensional scene, the scene being visible to a human observer from a viewing point when fixating on a visual fixation point within the scene; displaying an image by rendering the image data on a display device; capturing user input by user input capturing means, wherein capturing comprises monitoring a point of gaze of a user so as to determine a spatial coordinate in the three dimensional scene, the coordinate representing a movable visual fixation point of the human observer; computationally processing the image data so as to enclose each object of the three dimensional scene in a three dimensional detection region which is configured to identify coincidence of the respective object with the visual fixation point; modifying the image by: computationally isolating a fixation region within the image, the fixation region being defined by a subset of image data representing an image object within the image, wherein the image object is associated with the visual fixation point; spatially reconstructing the subset of image data to computationally expand the fixation region; spatially reconstructing remaining image data relative to the subset of image data to computationally compress a peripheral region of the image relative to the fixation region in a progressive fashion as a function of a distance from the fixation region, the method further comprising determining a distance between a head of the user and the display device; computationally processing the image data so as to move the fixation region towards a centre of a display of the display device, wherein the fixation region represents the object enclosed by the respective detection region; wherein the computational expansion of the fixation region and the computational compression of the peripheral region are modulated by the distance between the head of the user and the display device.