In-Vehicle AR Gesture Control for Mobile Terminal Interfaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for operating mobile terminals in vehicles, such as smartphones or tablets, are outdated and limited to single-user operation, requiring direct touch input and not compatible with modern, multi-user scenarios like chauffeured services or autonomous driving.
Innovation Solution
A method that mirrors the user interface of a mobile terminal onto a wearable output apparatus, enabling spatial gesture recognition for control, allowing users to operate the terminal without touching the screen, compatible with various mobile terminals and wearable output devices, and functional in both augmented and virtual reality environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a mobile terminal is operated using a touchscreen interface in a vehicle, then the user can directly interact with the device, but the operation is limited to single-user access and requires physical touch contact
Solution Approach 1:
The patent introduces a gesture recognition device as an intermediary between the user and the mobile terminal. This device captures gestures in the air space around the terminal and translates them into touch commands, enabling remote operation without physical contact and allowing multiple users to interact with the device sequentially
Solution Approach 2:
The patent replaces the mechanical touchscreen interface with an optical sensing system. The gesture recognition device uses cameras or sensors to detect hand movements and gestures in three-dimensional space, converting physical gestures into digital commands without requiring contact with the terminal's physical surface
2Adaptability or versatility
If a tablet PC is used as a hardware operating part for vehicle functions, then vehicle controls can be adjusted, but the device can only be used by one passenger at a time
Solution Approach 1:
The gesture recognition device serves as a mediator that enables multiple users to access the vehicle control system through their respective gestures. Each user can perform gestures to control vehicle functions without needing to physically hold or touch the tablet, making the system accessible to multiple passengers including those in difficult positions
3Adaptability or versatility
If a smartphone is used for mobile terminal operation in a vehicle, then portability is improved, but the device cannot be operated by users in seated or recumbent positions who cannot reach the screen
Solution Approach 1:
The patent extends the interaction space from the two-dimensional touchscreen surface to three-dimensional air space around the device. Users can perform gestures at various distances and angles from the terminal, allowing operation from seated, reclined, or other positions where the user cannot physically reach or touch the screen
Data Source
AI summary
A gesture recognition and control device recognizes a mobile terminal and ascertains a current graphical user interface generated by a display device of the mobile terminal. The gesture recognition and control device provides an output signal describing, as display content, the graphical user interface generated by the display device of the mobile terminal, and transmits the output signal to an output apparatus that can be worn on the head for outputting the display content in a predefined output region in the interior of the motor vehicle as part of augmented reality or virtual reality output by the output apparatus. During the process of outputting the display content, the gesture recognition and control device recognizes a spatial gesture of the user, generates a remote control signal for triggering an operating function of the mobile terminal, and transmits the remote control signal to a control device of the recognized mobile terminal.