Virtual Writing via Gaze and Pointer Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for electronic writing and drawing require a compatible writing surface and tool, limiting flexibility and collaboration without specialized equipment.
Innovation Solution
An electronic system that uses scene cameras and user-facing cameras to track the motion of a pointer relative to a surface and the user's gaze, generating virtual marks on a display based on proximity and gaze direction, enabling virtual writing and drawing without the need for a specialized surface or tool.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional writing tools and surfaces are used, then writing and drawing functions are achieved, but flexibility and adaptability are limited due to requirement for compatible equipment
Solution Approach 1:
The patent replaces mechanical writing systems (pens, pencils, styluses requiring physical contact with specialized surfaces) with an optical/electronic system using cameras to track pointer motion and gaze direction. This substitution eliminates the need for specialized writing surfaces and tools, enabling virtual writing anywhere in the visual field, thus dramatically improving adaptability and flexibility.
Solution Approach 2:
The patent introduces an intermediary system consisting of scene cameras and user-facing cameras that mediate between the user's writing intent and the display output. These cameras track pointer motion and gaze direction, translating physical gestures into virtual marks on the display without requiring direct contact with specialized surfaces, thereby eliminating equipment constraints.
2Measurement precision
If gaze tracking is added to pointer motion tracking, then accuracy of virtual mark generation is improved, but device complexity increases
Solution Approach 1:
The patent merges pointer motion tracking and gaze direction tracking into a unified virtual writing system. Both functions are achieved using the same camera infrastructure (scene cameras and user-facing cameras), combining multiple measurement functions into a single integrated system rather than using separate specialized devices for each function.
Solution Approach 2:
The camera system serves multiple functions: tracking pointer motion, detecting gaze direction, and generating virtual marks. This multi-functionality reduces the need for separate specialized devices, managing complexity by making existing components perform multiple roles rather than adding dedicated systems for each function.
Data Source
AI summary
An electronic system generates at a display virtual writing corresponding to tracked motion of the tip of a pointer with respect to a surface based on proximity of the tip of the pointer to the surface and the gaze of a user's eye. The electronic system determines the location and motion of the tip of the pointer with respect to the surface based on images captured by scene cameras, and determines the focus and gaze direction of the user's eye based on images captured by a user-facing camera. By generating virtual writing at the display corresponding to tracked motion of the tip of the pointer based on proximity of the tip of the pointer to the surface and based on the focus and gaze direction of the user's eye, the electronic system can enable virtual writing and associated collaboration services without the need for a specialized writing surface or pointer.