In-Cabin Gesture Tracking With Interval Detection and Hand Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gesture tracking technologies require significant computational power, making them unsuitable for devices with weaker computing capabilities, such as those in vehicle cabins, leading to limitations in gesture detection and tracking.
Innovation Solution
A method and apparatus that reduces computational power requirements by performing hand detection at preset intervals, determining hand position information for non-triggered frames based on triggered frames, and using a bipartite graph with a KM algorithm for hand matching, ensuring accurate gesture tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning model is applied to each frame for gesture detection, then detection accuracy is improved, but computational power consumption increases significantly
Solution Approach 1:
The patent applies periodic action by triggering hand detection only at preset time intervals rather than on every frame. The system identifies triggered frames (where hands are detected) and non-triggered frames (where hands are not detected), and only performs computationally intensive deep learning model inference on triggered frames. This periodic sampling approach significantly reduces computational power consumption while maintaining gesture detection accuracy through temporal interpolation of hand positions in non-triggered frames.
2Measurement precision
If hand detection is performed on every frame, then hand position accuracy is improved, but device complexity requirements increase
Solution Approach 1:
The patent applies preliminary action by performing hand detection at preset time intervals in advance, obtaining hand position information for triggered frames. For non-triggered frames, the system preliminarily determines hand position information through temporal interpolation based on the detected hand positions from triggered frames. This approach reduces device complexity by avoiding full hand detection on every frame while maintaining sufficient hand position accuracy for gesture recognition.
3Measurement precision
If deep learning model runs multiple times for gesture tracking, then tracking accuracy is improved, but processing time increases
Solution Approach 1:
The patent reduces processing time by applying periodic action - performing deep learning model inference only on triggered frames at preset time intervals rather than on every frame. The system processes triggered frames and non-triggered frames differently, using lightweight temporal interpolation for non-triggered frames. This periodic processing approach maintains gesture tracking accuracy while significantly reducing the number of model invocations and overall processing time.
Solution Approach 2:
The patent applies segmentation by dividing video frames into triggered frames and non-triggered frames, and processing them through different pipelines. Triggered frames undergo full hand detection using the deep learning model, while non-triggered frames use temporal interpolation based on neighboring triggered frames. This segmentation allows the system to maintain tracking accuracy for triggered frames while reducing processing time for non-triggered frames.
Data Source
Figure 1~2
Figure 3~5
Figure 6~7
AI summary
The present invention provides a gesture tracking method and apparatus, a computer device, and a readable storage medium. The gesture tracking method includes: acquiring a vehicle cabin video data stream, and capturing images from the vehicle cabin video data stream to obtain a plurality of image frames; triggering hand detection on image frames at preset time intervals to obtain hand position information for each triggered image frame; determining, for each triggered image frame, hand position information for each non-triggered image frame based on the hand position information for the triggered image frame; matching the hand position information for each hand based on the hand position information for every two consecutive image frames to obtain a matching result of the hand position information for each hand; and determining a dynamic gesture in the vehicle cabin video data stream based on the matching result of the hand position information for each hand in every two consecutive image frames. This method reduces the computational power required for gesture tracking, thereby lowering the requirements for device computing capability, minimizing limitations, and ensuring normal gesture tracking through various devices.