Gesture-Controlled Widget Interaction Region Mapping

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Gesture-controlled devices face challenges in accurately associating mid-air gestures with specific virtual widgets, leading to potential unintended control of devices or processes, especially when virtual widgets are not displayed or have no graphical representation.

Innovation Solution

A method and system that capture video frames of a user's body, detect body landmarks, generate widget interaction regions, recognize mid-air hand gestures, and map these gestures to virtual widgets, ensuring accurate manipulation by matching gesture types with corresponding virtual widget types, preventing accidental activation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If mid-air gestures are used to control virtual widgets without display, then ease of operation is improved, but reliability deteriorates due to inaccurate gesture-widget association

Engineering Contradiction:

Improvegesture controlVSAvoidgesture-widget association accuracy

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

The system segments the interaction space into multiple widget interaction regions, each corresponding to a specific virtual widget. By dividing the gesture recognition space into distinct zones rather than treating it as a uniform area, the system can more accurately associate gestures with their intended target widgets, thereby improving reliability while maintaining ease of operation.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces widget interaction regions as an intermediary layer between the gesture input and virtual widgets. These regions act as mediators that facilitate accurate mapping between gestures and widgets, solving the association accuracy problem while preserving the simplicity of gesture-based control.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If widget interaction regions are generated for each virtual widget, then measurement precision is improved, but device complexity increases

Engineering Contradiction:

Improvegesture location detectionVSAvoidsystem structure

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system employs a universal widget interaction region generation mechanism that can be applied to any virtual widget regardless of its type or function. This multi-functional approach allows the system to maintain high measurement precision across different widgets while avoiding the need for separate complex recognition systems for each widget type, thereby controlling overall system complexity.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11693482B2Systems and methods for controlling virtual widgets in a gesture-controlled device

Publication Date: 2023.07.04 HUAWEI TECH CO LTD

AI summary

A gesture-controlled device and a method thereon are provided. The method includes capturing a plurality of video frames of a user's body and processing the plurality of video frames to allow detecting a portion of the user's body and to allow recognizing hand gestures. In response to detecting the portion of the user's body, the method includes generating at least one widget interaction region corresponding to the portion of the user's body. The method further includes recognizing, a mid-air hand gesture in the at least one widget interaction region, mapping the mid-air hand gesture to at least one virtual widget on the gesture-controlled device, and manipulating the at least one virtual widget based on the mid-air hand gesture. The method and device allow manipulation of virtual widgets using mid-air gestures on a gesture-controlled device.