Wearable Extended-Screen Interaction Using Corner-Point Gesture Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Intelligent wearable devices currently cannot directly operate extended screen components within the display interface, leading to a poor user experience in interaction functionality with mobile terminals.
Innovation Solution
The method involves detecting the attitude of a mobile terminal using an image acquisition apparatus, acquiring corner point coordinates when a preset interaction operation is matched, determining the target extended screen component based on these coordinates, and executing operations based on gesture actions, thereby enhancing interaction efficiency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If extended screen mapping is implemented on intelligent wearable devices, then service display capability is improved, but direct operation capability of extended screen components deteriorates
Solution Approach 1:
The patent introduces an image acquisition apparatus as an intermediary to capture the mobile terminal's screen display, enabling the wearable device to display extended screen content without directly operating the original screen components. The captured image serves as a mediator between the mobile terminal's display and the wearable device's interaction interface.
Solution Approach 2:
The patent creates a copy of the mobile terminal's screen display by capturing the image and displaying it on the wearable device. This copied display enables the wearable device to show extended screen content while maintaining the ability to interact with it through gesture recognition rather than direct component operation.
2Ease of operation
If gesture recognition is added to enable direct operation, then ease of operation is improved, but device complexity increases
Solution Approach 1:
The patent makes the wearable device's processing unit perform multiple functions: it processes both the extended screen content display and the gesture recognition tasks. By integrating these functions into a single processing unit, the patent avoids adding separate dedicated hardware for gesture recognition, thereby limiting the increase in device complexity.
Solution Approach 2:
The wearable device uses its own image acquisition apparatus and processing unit to perform gesture recognition independently. The device serves itself by using its existing resources (camera, processor) to enable new interaction capabilities without requiring additional external systems or complex integrated circuits.
3Measurement precision
If corner point coordinate detection is implemented, then operation precision is improved, but measurement complexity increases
Solution Approach 1:
The patent segments the screen interaction by detecting corner points of the mobile terminal's display. Instead of analyzing the entire screen image, the system focuses on detecting specific corner points, which simplifies the measurement process while maintaining high precision for identifying extended screen components and their positions.
Solution Approach 2:
The patent replaces complex manual coordinate measurement and calculation with automated image processing algorithms. The processing unit automatically detects corner points and calculates coordinates from the captured image, substituting what would otherwise be a complex manual measurement process with an automated computational approach.
Data Source
AI summary
The present disclosure provides a method for interaction between an intelligent wearable device and a mobile terminal. The method includes: collecting corner point coordinates of a target corner point on a mobile terminal, determining a target extended screen component on which the current user wants to operate, collecting the current gesture action of the current user, generating a corresponding component operation instruction, and executing a target operation on the target extended screen component.