Mobile Face-Tracking Interaction for Touch-Free Game Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mobile game interaction methods require finger-based operations on touch screens, limiting immersion and accessibility for users, especially those with hand deficiencies.
Innovation Solution
Implement facial expression recognition using a camera to capture and analyze successive face images, generating control instructions for interactive applications without finger contact, enhancing immersion and accuracy through magnetic and damping effects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If finger-based operations on touch screen are used for mobile game interaction, then device compatibility and ease of operation are maintained, but user immersion and accessibility for users with hand deficiencies are limited
Solution Approach 1:
The patent replaces the mechanical touch screen interaction system with an optical recognition system. The camera captures facial expressions and the system processes these visual inputs to generate control instructions, substituting physical finger operations with optical-based facial expression recognition. This resolves the contradiction by providing a non-contact interaction method that enhances both accessibility for users with hand deficiencies and immersion through natural facial expression control.
2Measurement precision
If facial expression recognition is implemented using camera to capture and analyze successive face images, then user immersion and control accuracy are enhanced, but device complexity and processing requirements increase
Solution Approach 1:
The patent extracts only the necessary facial feature points from successive camera frames rather than processing entire images. By identifying and tracking specific key points (eyes, eyebrows, mouth) and their movements, the system achieves high control accuracy while reducing computational complexity. This selective extraction approach resolves the contradiction between precision and system complexity.
Solution Approach 2:
The patent performs preliminary processing by detecting face presence and establishing baseline facial configurations before detailed expression analysis. The system first determines whether a face is present, then establishes reference positions for facial features, and only then proceeds to analyze expression changes. This preliminary action reduces processing complexity by filtering out unnecessary frames and pre-processing data structures.
3Speed
If real-time image acquisition and comparison is performed to obtain face actions and amplitudes, then interaction responsiveness and control smoothness are improved, but processing time and computational load increase
Solution Approach 1:
The patent implements periodic action by acquiring images at fixed time intervals rather than continuously processing all available frames. The system captures successive frames at predetermined rates, compares them periodically, and generates control instructions based on these periodic comparisons. This approach maintains interaction responsiveness through regular updates while reducing overall processing energy consumption by skipping unnecessary continuous processing.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
Embodiments of this application disclose an interaction method of an application scene, a mobile terminal, and a storage medium, to implement immersive interaction on the mobile terminal. In the embodiments of this application, real-time image acquisition is performed on a target face by using a camera configured on the mobile terminal, to obtain a first frame of image and a second frame of image, the first frame of image and the second frame of image being two successive frames of images that are separately captured; the first frame of image is compared with the second frame of image, to obtain an action of the target face and a corresponding amplitude; a control instruction of a simulated object in an interactive application scene is generated according to the action of the target face and the corresponding amplitude, the simulated object and an interactive item being displayed in the interactive application scene; and the simulated object is controlled, according to the control instruction, to interact with the interactive item in the interactive application scene.