In-Cabin Gesture Recognition Using User Grids and Depth Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gesture recognition systems in vehicles face challenges in accurately recognizing non-contact gestures, particularly when the user is not in a fixed position relative to the sensor, leading to limited recognizable ranges and potential misinterpretation of gestures.
Innovation Solution
A vehicle control method that constructs a user grid based on in-cockpit images and user position information, incorporating depth data and cockpit physical parameters to enhance gesture recognition accuracy, allowing for multiple users and dynamic hand positioning, and includes permission-based control to ensure safety and flexibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If arrayed photoelectric sensor is used to collect gesture operation, then device complexity is reduced, but recognizable range is limited and gesture recognition accuracy deteriorates
Solution Approach 1:
The patent transitions from 2D planar sensor arrays to 3D spatial coordinate systems by introducing depth information through multiple cameras and time-of-flight sensors. This dimensional expansion enables accurate gesture recognition across varied distances and angles, resolving the limitation of fixed-position sensors while maintaining system feasibility.
Solution Approach 2:
The system employs multiple types of sensors (color cameras, time-of-flight cameras, depth sensors) that serve multiple functions: capturing color information, depth data, and gesture trajectories simultaneously. This multi-functional approach improves gesture recognition accuracy without proportionally increasing device complexity.
2Device complexity
If fixed-position sensor array is used, then device structure is simplified, but adaptability to dynamic user positions deteriorates
Solution Approach 1:
The system implements dynamic calibration that continuously adapts to changing user positions and orientations. The calibration parameters are updated in real-time based on detected user locations, enabling the fixed sensor array to effectively track and recognize gestures regardless of user movement within the vehicle cabin.
Solution Approach 2:
The system uses feedback from depth sensors and user position detection to continuously adjust calibration parameters. This closed-loop approach allows the fixed sensor array to adapt to dynamic user positions by incorporating real-time position information into gesture recognition calculations.
3Device complexity
If single-user gesture recognition is implemented, then system complexity is reduced, but user experience and versatility deteriorate
Solution Approach 1:
The system segments the gesture recognition process into distinct modules: user detection, individual calibration, gesture identification, and permission-based control. This segmentation enables efficient handling of multiple users by maintaining separate calibration data and permission settings for each user while sharing common gesture recognition algorithms.
Data Source
AI summary
Embodiments of this application provide a vehicle control method based on gesture recognition, an apparatus, and a vehicle. The method includes: obtaining an in-cockpit image of a vehicle and user in-position information, where the user in-position information indicates positions of M users in the vehicle that are presented in the in-cockpit image; constructing a user grid based on the in-cockpit image and the user in-position information, where the user grid includes depth data of the M users; for each of N target users performing a gesture operation, recognizing a gesture operation intention of the target user based on the user grid, where the M users include the N target users; and separately controlling the vehicle based on gesture operation intentions of the N target users, where both M and N are positive integers, and M is greater than or equal to N.