Gesture Recognition via Video Frame Analysis for Terminal Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computer operation methods heavily rely on peripheral devices like mice and keyboards, limiting user interaction and flexibility.
Innovation Solution
A method and device that utilize gesture recognition by analyzing video streams to identify finger movements and corresponding operations, allowing users to interact with terminal devices without direct physical contact, by obtaining gesture video segments, extracting gesture track information, and mapping it to predefined operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional operation methods using peripheral devices like mouse and keyboard are used, then the operation control is precise and reliable, but the user interaction flexibility and diversity are limited
Solution Approach 1:
The patent replaces the mechanical interaction system (mouse and keyboard) with an optical recognition system. The video device captures gestures in the air space, and the processor analyzes the video data to recognize gesture tracks and convert them into operation commands, eliminating the need for physical peripheral devices and enabling more flexible and diverse user interactions.
2Measurement precision
If gesture recognition is implemented by analyzing each frame of image in gesture video segment, then the gesture recognition accuracy is improved, but the processing time and computational complexity increase
Solution Approach 1:
The patent performs preliminary actions by extracting feature points (fingertip locations) from each frame of the gesture video segment and tracking their movement trajectories in advance. This preprocessing of gesture data allows for accurate gesture recognition while optimizing processing efficiency, as the essential gesture characteristics are captured and stored for subsequent operation matching without requiring intensive real-time analysis of every pixel in each frame.
Data Source
AI summary
A method for operation a terminal device with gesture and a device are provided. The method includes obtaining a gesture video segment comprising a preset number of frames of images, obtaining gesture track information of a user according to a location of a finger of the user in each frame of image of the gesture video segment, searching in a preset corresponding relationship between the gesture track information and an operation according to the gesture track information, obtaining the operation corresponding to the gesture track information of the user and performing the operation. With the technical scheme of the present disclosure, the gesture track information of the user is obtained via analyzing each frame of image in the obtained gesture video segment. The gesture track information of the user in a dimensional plane is concerned and the corresponding operation thereof is obtained via the gesture track information. This technical scheme may enrich the methods for triggering the operation, operate the terminal device without directly touching the terminal device and reduce the dependence on the peripheral devices, such as the mouse and the keyboard.