Hand Tracking via Gesture Classification and Matrix Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision technologies face challenges in accurately tracking targets in special scenes, such as tabletop game scenarios, due to misjudgment in single-frame image recognition and high system time consumption with large deep learning models when processing multiple frames of video.
Innovation Solution
A target tracking method that detects a hand pose in a bounding box within a to-be-processed image, determines if it conforms to a predefined target gesture, and tracks the target hand in a synchronously captured video stream using a gesture classification neural network and matrix prediction techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning models are used to recognize target hands in multiple frames of video, then recognition accuracy is improved, but system time consumption increases
Solution Approach 1:
The patent divides the hand detection and recognition task into two independent parts: a lightweight hand detection model (YOLACT) that processes each frame independently to provide bounding boxes, and a gesture classification neural network that processes only the cropped regions within those bounding boxes. This segmentation allows the system to maintain high recognition accuracy while reducing overall processing time by avoiding the need to analyze entire frames with complex models.
Solution Approach 2:
The patent extracts only the relevant portions (hand regions within bounding boxes) from each video frame for further processing. By using YOLACT to first identify and crop the hand regions, the system extracts only the necessary information for gesture classification, eliminating the need to process the entire frame with computationally intensive deep learning models, thus reducing time consumption while maintaining accuracy.
2Productivity
If hand detection is performed on single frame images, then processing speed is improved, but misjudgment in recognition occurs
Solution Approach 1:
The patent performs preliminary hand detection and bounding box extraction on each frame using the lightweight YOLACT model before applying the gesture classification neural network. This preliminary action quickly identifies candidate hand regions, allowing the system to process frames at high speed while the subsequent gesture classification step ensures accurate recognition by analyzing only the relevant cropped regions.
Solution Approach 2:
The patent introduces an intermediary cropping step that bridges the lightweight detection model and the gesture classification network. By cropping the image to only the bounding box regions identified by YOLACT, the system creates an intermediary representation that maintains processing speed advantages while providing sufficient detail for accurate gesture recognition, preventing misjudgment.
Data Source
AI summary
Disclosed are a target tracking method and apparatus, an electronic device, and a storage medium. The method includes: detecting a to-be-processed image to obtain a hand detection result; in response to the hand detection result including a bounding box of hand, determining a hand in the bounding box with a hand pose conforming to a hand pose in a target gesture as a target hand; and tracking the target hand in a video stream according to the target hand in the to-be-processed image, where images in the video stream and the to-be-processed image are obtained by capturing a same target area, and the images in the video stream are captured after the to-be-processed image is captured.