In-Cabin Gesture Recognition With User Grid and Multi-User Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vehicle control systems face challenges in accurately recognizing non-contact gesture operations, particularly when the driver or passenger is not in a position to interact with the screen, limiting the range of gesture recognition and potentially leading to inaccurate or unrecognized commands.
Innovation Solution
A vehicle control method that constructs a user grid based on in-cockpit images and user position information, using sensors like RGB cameras, IR cameras, and radar to enhance gesture recognition accuracy by providing depth data and identifying multiple users, and determines control permissions based on user positions and facial data to ensure safe and flexible vehicle operation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If an arrayed photoelectric sensor is used to collect gesture operations, then the device structure is simplified, but the recognizable range is limited and gesture operations may not be accurately recognized when users are in certain positions
Solution Approach 1:
The patent combines multiple sensing approaches (photoelectric sensors, depth cameras, radar) into a unified gesture recognition system. This merging of different sensing technologies expands the recognizable range while maintaining relatively simple device structure, resolving the contradiction between simplicity and accuracy.
Solution Approach 2:
The system is designed to support multiple users (driver and front passenger) with gesture recognition capabilities. By making the system universal for different users and positions, it overcomes the limitation of restricted recognizable range while keeping the overall device architecture manageable.
2Device complexity
If gesture recognition is limited to a single user, then the system complexity is reduced, but the adaptability and user experience are degraded
Solution Approach 1:
The patent segments the gesture recognition space into multiple user zones (driver zone and front passenger zone) with different control permissions. This segmentation allows multi-user support while managing system complexity through clear spatial division and permission assignment.
Solution Approach 2:
The system adds a spatial dimension to gesture recognition by determining user positions and assigning different control permissions based on location. This dimensional approach enables multi-user adaptability without proportionally increasing system complexity, as the permission structure follows clear spatial rules.
3Adaptability or versatility
If the gesture recognition range is expanded to multiple users, then the adaptability is improved, but the difficulty of detecting and measuring gestures increases
Solution Approach 1:
The patent applies different detection strategies and permission rules to different spatial zones (driver area vs. front passenger area). This local quality approach allows multi-user gesture recognition while managing detection complexity by tailoring the recognition criteria to each user's position and expected gesture patterns.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The method improves gesture recognition accuracy and flexibility, allowing multiple users to control vehicle functions safely and efficiently, expanding the range of gesture recognition beyond the display and ensuring authorized users can perform intended operations.
Implementation Method 1
obtaining an in-cockpit image of a vehicle and user in-position information
Implementation Method 2
using sensors like RGB cameras, IR cameras, and radar to enhance gesture recognition accuracy by providing depth data
Implementation Method 3
using sensors like RGB cameras, IR cameras, and radar to enhance gesture recognition accuracy
Implementation Method 4
using sensors like RGB cameras, IR cameras, and radar to enhance gesture recognition accuracy by providing depth data
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments of this application provide a vehicle control method based on gesture recognition, an apparatus, and a vehicle. The method includes: obtaining an in-cockpit image of a vehicle and user in-position information, where the user in-position information indicates positions of M users in the vehicle that are presented in the in-cockpit image; constructing a user grid based on the in-cockpit image and the user in-position information, where the user grid includes depth data of the M users; for each of N target users performing a gesture operation, recognizing a gesture operation intention of the target user based on the user grid, where the M users include the N target users; and separately controlling the vehicle based on gesture operation intentions of the N target users, where both M and N are positive integers, and M is greater than or equal to N. The user grid is constructed based on the user in-position information, to provide refined data of the gesture operation for gesture recognition, thereby improving accuracy of gesture recognition.