Unified Gesture Controller API for Gaming Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current gaming systems face a divide between traditional hand-held controllers and natural user interface (NUI) systems, with NUI systems often requiring full or partial skeletal tracking and recognition, which can be more input than needed for software development, and there is a need for a system that combines the advantages of both.
Innovation Solution
A system that receives inputs from both hand-held controllers and NUI systems, processing tactile and gestural data into Boolean or floating point formats for easy consumption by software applications, using a gesture and controller application programming interface (API) to unify input formats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional NUI systems use full or partial skeletal tracking and recognition, then gesture recognition capability is improved, but device complexity and data processing burden increase
Solution Approach 1:
The patent extracts only the essential gesture information needed for software applications from the complete skeletal tracking data. The NUI system captures full skeletal data, but the system selectively processes and transmits only relevant gesture parameters to applications, reducing data processing burden while maintaining gesture recognition capability.
Solution Approach 2:
The system creates a universal input interface that handles both traditional controller inputs and NUI gesture inputs through a unified API. This multi-functional approach allows software applications to receive standardized input data regardless of the input source, simplifying integration while supporting diverse input methods.
2Measurement precision
If NUI systems provide full skeletal tracking data, then gesture recognition accuracy is improved, but ease of operation for software developers deteriorates due to complex data formats
Solution Approach 1:
The patent introduces an intermediary processing layer between the NUI system and software applications. This layer receives comprehensive skeletal tracking data, processes it to extract relevant gesture information, and presents simplified standardized inputs to applications. The intermediary maintains measurement precision by preserving full data internally while easing operation by providing simplified interfaces to developers.
Solution Approach 2:
The system transforms complex skeletal tracking parameters into simplified gesture parameters that are more suitable for software application consumption. By changing the parameter representation from detailed joint coordinates to meaningful gesture states, the system maintains recognition accuracy while improving ease of operation for developers.
3Ease of operation
If a unified input format system is implemented, then ease of operation for developers is improved, but the quantity of processed data from NUI systems increases
Solution Approach 1:
The system extracts only the necessary gesture information from the complete NUI data stream before converting it to unified input format. By filtering and selecting only relevant data elements for gesture recognition, the system reduces the quantity of data that needs to be processed and transmitted, while still providing comprehensive unified input functionality to developers.
Data Source
AI summary
A system and method are disclosed for providing input to a software application such as a gaming application. The inputs may be received from one or both of a hand-held controller and a NUI system accepting gestural inputs. In an example, a gesture and controller API receives skeletal pose data from the NUI system and converts that data to Boolean or floating point data. In this way, data from both the controller and the NUI system may be exposed to the software application as easily consumable data in the same format.