Hybrid Game Controller Combining Manual Input and Camera Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video and computer game control systems face limitations in providing fine-grained, intuitive control, especially in games that require complex interactions beyond simple movement replication, as they often rely on either hand-held controllers or voice/gesture recognition, which may not fully leverage the potential of both modalities for enhanced player experience.
Innovation Solution
A hybrid control system integrating a hand-held controller with a camera-based control system using depth cameras and gesture recognition, allowing players to use both manual inputs and body movements to control game avatars, enabling more complex interactions and momentary actions by recognizing specific gestures and combining inputs from both systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If hand-held controllers with buttons and joysticks are used for fine-grained control, then control precision is improved, but device complexity and ease of operation deteriorate due to physical restrictions and learning curves
Solution Approach 1:
The patent combines hand-held controller input with camera-based gesture recognition to create a hybrid control system. The controller handles precise input actions while the camera captures natural gestures, merging both modalities to achieve fine-grained control without the limitations of either system alone.
Solution Approach 2:
The camera system acts as an intermediary that translates natural body gestures into control commands. This mediator layer allows players to use intuitive physical gestures rather than directly manipulating complex controller interfaces, improving ease of operation while maintaining control precision through the combined system.
2Ease of operation
If voice recognition and gesture recognition are used without hand-held controllers, then ease of operation is improved, but control precision and reliability deteriorate for complex game interactions
Solution Approach 1:
The system merges camera-based gesture recognition with hand-held controller input to compensate for the precision limitations of gesture-only control. The controller provides reliable, precise input for complex game actions while the camera handles natural movement and positioning, creating a complementary hybrid system.
Solution Approach 2:
The control system is segmented into different functional components: the camera handles natural gesture capture and positioning, while the hand-held controller handles precise action input. This segmentation allows each component to excel at its specialized function, with the camera providing ease of operation and the controller providing control precision.
3Ease of operation
If camera-based control systems are used alone, then ease of operation is improved through natural gesture recognition, but reliability and measurement precision worsen for complex interactions
Solution Approach 1:
The patent merges camera-based control with hand-held controller input to create a redundant hybrid system. The controller provides reliable backup and precise input verification for camera-based gestures, ensuring consistent and reliable control for complex game interactions while maintaining the natural ease of gesture recognition.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Manual and camera-based avatar control is described for example, in computer game systems. In one embodiment, an image capture system tracks motion of a player's arm holding a real world object and player also makes input to the game system at same time, using a manual controller. In an example, tracked motion of a player's arm is used to control aim of a virtual dart in a game display and player makes manual input at the controller to fire the dart. In various embodiments, position of a part of a player is detected and mapped to screen space using a mapping function which may vary. For example, a mapped position of a player's hand in screen space is used to control the position of an avatar's hand by extending the avatar's arm. In examples, conditions are met before camera-based control of an avatar.