Behind-display interface for 3D object manipulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional user interfaces, such as those using a standard keyboard, video, and mouse, can impede users' ability to fully visualize and control three-dimensional graphical objects due to their two-dimensional nature, limiting intuitive interaction and manipulation.
Innovation Solution
A behind-display user interface system that uses a computing device with a display component and sensors to track the movement of a physical pointer, allowing users to interact with graphical objects in a three-dimensional manner by sensing and modifying their movement, even when the pointer is on the opposite side of the device, creating a pseudo-transparent interface for manipulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a standard keyboard, video, and mouse user interface is used, then the system structure remains simple and cost-effective, but the user's ability to visualize and control three-dimensional graphical objects is impeded
Solution Approach 1:
The patent introduces a camera as an intermediary device positioned behind the display to capture the user's hand movements and gestures. This intermediary captures physical movements in 3D space and translates them into digital commands, bridging the gap between the simple VUI structure and the need for intuitive 3D object manipulation without adding complex specialized hardware
Solution Approach 2:
The patent replaces the traditional mechanical mouse pointing system with an optical recognition system using a camera and image processing algorithms. Instead of mechanical movement translation, the system uses computer vision to detect hand gestures and translate them into cursor movements and commands, enabling more natural 3D interaction while maintaining system simplicity
2Ease of operation
If a touchscreen is used to combine input and output interface functions, then the manipulation and viewing operations are co-located improving the designer's experience, but the two-dimensional nature of the touchscreen limits the ability to create and manipulate three-dimensional graphical objects
Solution Approach 1:
The patent adds a third dimension of interaction by positioning the camera behind the display, allowing users to perform gestures in 3D space rather than being confined to the 2D touchscreen surface. This enables natural hand movements, pointing, and manipulating actions that correspond to spatial relationships in 3D graphical objects, while the display continues to provide 2D visual feedback
Solution Approach 2:
The camera acts as an intermediary that captures 3D hand gestures behind the display and translates them into 2D touchscreen commands. This intermediary layer enables the co-location benefit of touchscreen interaction while overcoming its 2D limitation by interpreting 3D spatial gestures through optical recognition and gesture mapping algorithms
3Ease of operation
If specialized cameras, sensors, eyewear, and displays are added to provide easier and more intuitive user access, then user interaction with graphical functions is improved, but the hardware costs of the system are significantly increased
Solution Approach 1:
The patent makes the display serve multiple functions: it continues to provide visual output while simultaneously acting as a transparent interface that allows the camera to capture gestures behind it. The camera, positioned behind the display, captures hand movements and gestures, translating them into digital commands. This multi-functional approach enables intuitive 3D interaction without adding specialized expensive hardware components
Solution Approach 2:
The system uses the existing display structure and a single rear-positioned camera to enable advanced 3D gestures. The camera captures the user's natural hand movements and gestures in 3D space, and the system's image processing algorithms automatically recognize and interpret these gestures, eliminating the need for specialized sensors, eyewear, or complex hardware additions
Data Source
AI summary
Example systems and methods of providing a user interface are presented. In one example, a graphical object is displayed on an opaque display component on a user-facing side of a computing device. Using a sensing component of the computing device, movement of a physical pointer controlled by a user is sensed. The physical pointer may be located opposite the user-facing side of the computer device. On the opaque display component, a representation of the physical pointer is displayed during the movement of the physical pointer. The graphical object, as displayed on the opaque display component, is modified based on the sensed movement of the physical pointer during the movement of the physical pointer.