Hand Pose Control Using Camera and Capacitive Finger Sensing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing VR devices struggle to accurately detect the shape and status of a user's fingers when they are invisible from the camera's position or outside its field of view.
Innovation Solution
The system combines data from hand-mounted sensors and a head-mounted camera to determine the pose of a user's hands, using capacitive sensors on the hand-mounted devices to detect finger proximity and a camera to detect hand shape, merging the data to ensure accurate hand pose recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a camera is used to detect the user's hands, then the hand detection can be performed visually, but the detection fails when fingers are invisible from the camera position or outside the field of view
Solution Approach 1:
The patent combines a camera-based visual detection system with a hand-mounted sensor system to create a hybrid detection apparatus. The camera captures visual information about hand poses when visible, while capacitive sensors mounted on the hand detect finger states through direct contact. These multiple detection sources are merged to provide comprehensive hand pose recognition that works both when hands are visible and when they are outside the camera field of view.
Solution Approach 2:
The hand-mounted operating device serves as an intermediary detection element that directly contacts the user's hand. The capacitive sensors on this device detect the electrical properties of the hand and fingers, providing information about hand pose and finger states without relying on visual line of sight. This intermediary device bridges the gap when the camera cannot directly observe the hand.
2Reliability
If only a camera is used for hand detection, then the device structure remains simple, but the detection reliability decreases when hands are outside the camera field of view
Solution Approach 1:
The system merges a camera-based detection component with a hand-mounted sensor component into an integrated detection apparatus. The camera provides reliable detection when hands are within field of view, while the hand-mounted capacitive sensors provide reliable detection when hands are outside the field of view. By combining these components, the system achieves consistent detection reliability across all hand positions while accepting the increased device complexity.
Solution Approach 2:
The hand-mounted operating device serves multiple functions: it acts as a control interface for the VR system, provides capacitive sensing for finger detection, and serves as a mounting platform for sensors. This multi-functionality increases detection reliability without proportionally increasing overall system complexity, as the same physical device performs multiple roles.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances the ability to detect hand poses with high accuracy, even when fingers are not visible to the camera, by leveraging multiple data sources for comprehensive hand pose estimation.
Implementation Method 1
the sensor section 15 includes a capacitive sensor that generates information indicative of whether the user's finger is in proximity
Data Source
Figure 1
Figure 2(a)~3
Figure 4(a)~4(b)
AI summary
Provided is a control apparatus that receives information regarding a motion of user's fingers and information regarding a shape of user's hands, generates first pose information indicative of a pose of user's hands in accordance with the received information regarding the motion of user's fingers, generates second pose information indicative of a pose of user's hands in accordance with the received information regarding the shape of user's hands, combines the generated first pose information with the generated second pose information in accordance with criteria based on a predetermined condition, and generates and outputs information indicative of the pose of user's hands.