Floor Estimation for Depth-Based HCI Gesture Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Depth-based human-computer interaction systems face challenges in processing and interpreting data from dynamic and static environments, requiring integration of design conditions with mechanical constraints to achieve a successful user experience, especially when depth sensors are positioned at oblique angles relative to the floor or wall.
Innovation Solution
The implementation of a depth data processing pipeline that includes preprocessing via plane clipping and classification, using floor and wall planes as clipping planes to isolate relevant depth data, and employing a floor estimation procedure to improve accuracy, which involves determining the best candidate floor plane by iteratively generating and evaluating metrics based on depth point projections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If depth-based sensors are used for HCI, then lighting problems and shadows are eliminated, but dynamic and static environmental data processing complexity increases
Solution Approach 1:
The patent segments the depth data processing by introducing a floor estimation module that separates floor detection from general environmental processing. The floor plane is identified as a distinct reference frame, allowing the system to divide processing into: (1) floor plane detection, (2) gesture data isolation relative to floor, and (3) environmental background processing. This segmentation reduces overall processing complexity.
Solution Approach 2:
The system performs preliminary floor plane estimation before processing gesture recognition. By establishing the floor plane as a reference frame in advance, the system pre-processes environmental data by removing floor-related depth information, which simplifies subsequent gesture detection and reduces real-time processing requirements.
2Adaptability or versatility
If sensors are positioned at oblique angles relative to the floor, then installation flexibility increases, but floor and wall plane determination accuracy decreases
Solution Approach 1:
The patent implements a dynamic floor estimation procedure that adapts to different sensor orientations. Rather than assuming a fixed sensor-to-floor relationship, the system iteratively generates candidate floor planes and evaluates them based on depth point projections. This dynamic approach allows accurate plane determination regardless of whether the sensor is positioned at oblique angles or mounted on walls, ceilings, or floors.
Solution Approach 2:
The system changes the parameter space by evaluating multiple candidate floor planes with different orientations and positions. By iteratively adjusting floor plane parameters (normal vectors, distances from sensor) and selecting the best match based on depth data distribution, the system maintains accuracy across various installation configurations without requiring fixed sensor positioning.
3Measurement precision
If plane clipping is used to isolate gestures, then gesture recognition accuracy improves, but processing time increases
Solution Approach 1:
The system performs preliminary floor plane estimation and environmental background removal before gesture recognition. By pre-processing the depth data to establish reference planes and remove static environmental elements, the system reduces the complexity of subsequent gesture isolation operations, allowing plane clipping to be applied more efficiently to the already-simplified data.
Data Source
AI summary
Human Computer Interfaces (HCI) may allow a user to interact with a computer via a variety of mechanisms, such as hand, head, and body gestures. Various of the disclosed embodiments allow information captured from a depth camera on an HCI system to be used to recognize such gestures. Particularly, the HCI system's depth sensor may capture depth frames of the user's movements over time. To discern gestures from these movements, the system may group portions of the user's anatomy represented by the depth data into classes. This grouping may require that the relevant depth data be extracted from the depth frame. Such extraction may itself require that appropriate clipping planes be determined. Various of the disclosed embodiments better establish floor planes from which such clipping planes may be derived.