AR Interface Control Using Face Tracking and Virtual Overlays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing telecommunications applications lack effective methods for controlling graphical user interfaces using real-world objects captured in video streams, limiting user interaction and engagement.
Innovation Solution
An augmented reality system that captures real-world objects, such as a user's face or hand, within a video stream and depicts them in a graphical user interface, enabling control through manipulation of functions or operations within the interface using image capture devices and augmented reality elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional touchscreen control methods are used, then the device interface is simple to operate, but user engagement and interaction depth are limited
Solution Approach 1:
The patent creates a digital copy of the real-world object (face, hand, or other object) detected by the camera and renders it as a virtual representation within the graphical user interface. This virtual copy can then be manipulated to control interface elements, allowing users to interact with the device through natural gestures captured in the video stream without requiring complex traditional UI elements.
Solution Approach 2:
The patent introduces a video stream processing system as an intermediary between the user and the device interface. The system captures real-world objects via camera, processes them through object detection algorithms, and translates them into control signals for the graphical user interface, serving as a mediator that converts physical gestures into digital commands.
2Adaptability or versatility
If video stream processing and object detection are implemented, then user engagement is enhanced, but processing time and computational resources increase
Solution Approach 1:
The patent implements periodic frame sampling from the video stream rather than continuous processing of every frame. The system selectively processes frames at optimized intervals or only when specific objects are detected, reducing computational load and processing time while maintaining effective control responsiveness.
Solution Approach 2:
The patent pre-processes and pre-identifies objects of interest in the video stream before they reach the main processing pipeline. By detecting and tracking objects in advance, the system can prepare control mappings and reduce latency when actual interaction is needed, minimizing perceived processing delays.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems, devices, media, and methods are presented for controlling user interface with an object depicted within the user interface. The systems and methods initiates an augmented reality mode configured to present augmented reality elements within a graphical user interface. A face is detected within a field of view of an image capture component and presented within the graphical user interface. In response to detecting the face, the systems and methods sequentially present a set of augmented reality elements within the graphical user interface. A subset of the augmented reality elements and the face may be depicted contemporaneously. A movement is detected of at least a portion of the face relative to a first augmented reality element and presentation of the first augmented reality element is modified and at least one second augmented reality element is presented.