Hand Shape Estimation via Sensor-to-Image Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional technologies for detecting hand movements using sensors on game controllers struggle with accuracy due to differences in information nature between sensor data and image-based hand shape recognition, leading to insufficient shape estimation.
Innovation Solution
An information processing device with sensors on a user-operated device generates an image with pixel values based on sensor outputs, which is used to estimate hand joint positions in a three-dimensional space through plane-unfolding and neural network processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sensors are arranged on the surface of a game controller to detect hand movements, then the device can acquire spatial positional information, but the accuracy of hand shape estimation is insufficient due to the fundamental difference in information nature between sensor data and image-based recognition data
Solution Approach 1:
The patent creates a virtual copy of the sensor array arrangement by generating an image where pixel values correspond to sensor outputs positioned at their respective physical locations on the controller surface. This copying approach allows image-based recognition algorithms to process sensor data, bridging the information nature gap between physical sensors and digital image processing while maintaining measurement accuracy.
Solution Approach 2:
The patent introduces an intermediate processing step that converts sensor output values into an image format with pixel values. This intermediary representation serves as a bridge between the raw sensor data and the image-based hand shape recognition algorithm, enabling accurate hand shape estimation without requiring fundamental changes to the recognition system.
2Adaptability or versatility
If conventional image-based hand recognition technology is applied to sensor data from game controllers, then the system can process the data, but sufficient accuracy cannot be acquired because the information nature is significantly different
Solution Approach 1:
The patent transforms sensor output parameters into image-compatible pixel value parameters. By mapping sensor data values to pixel intensity values and sensor positions to pixel coordinates, the system changes the parameter representation to match what image-based recognition algorithms expect, thereby improving both adaptability and measurement precision simultaneously.
Solution Approach 2:
The patent creates a virtual image copy of the sensor data arrangement, where each sensor's output is represented as a pixel at the corresponding position on the controller surface. This copying enables conventional image-based recognition technology to process sensor data effectively, achieving both high adaptability and accurate hand shape recognition.
3Shape
If sensors are arranged along the surface shape of the controller housing, then the device can capture spatial positional relations, but the acquired information differs significantly from single-point space image data
Solution Approach 1:
The patent transforms the spatial arrangement information from a three-dimensional surface distribution into a two-dimensional image plane representation. By mapping sensor positions on the controller surface to pixel positions in an image, the system preserves spatial relational information while converting it to a format compatible with image-based processing, preventing information loss.
Solution Approach 2:
The patent creates a two-dimensional image copy that represents the three-dimensional sensor arrangement on the controller surface. This copying process preserves the spatial positional relationships between sensors while converting the data into a format that maintains compatibility with image-based recognition systems, eliminating information incompatibility.
Data Source
AI summary
Provided is an information processing device coupled to a device which a user operates with a hand of the user and on the surface of which a plurality of sensors that each output information associated with a spatial positional relation with the surface of the hand of the user are arranged, the information processing device being configured to receive output from each of the sensors included in the device and generate an image in which an image element having been drawn with a pixel value determined on the basis of a value represented by the output from each of the sensors is arranged at a position corresponding to a position at which each of the sensors is arranged, on a mapping space obtained by plane-unfolding a portion that constitutes the surface of the device and on which the sensors are arranged, Further, the information processing device is configured to, upon receipt of input of the generated image, estimate information associated with the position of each of joints of the hand of the user who operates the device and output the estimated information as coordinate information in a three-dimensional space whose origin is placed on a point that moves with the device.