Multi-Lens Hand Pose Recognition via View Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hand pose recognition technologies face challenges in accuracy and processing time due to the need for target detection on multiple views, leading to data redundancy and inefficiency.
Innovation Solution
A method that utilizes a multi-lens camera to acquire a current frame of a multi-lens video, performing hand detection on one view and hand estimation on another, while filtering out redundant boxes and converting two-dimensional joint points to three-dimensional joint points in a world coordinate system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If target detection algorithm is used to obtain hand detection boxes on multiple views, then accuracy of hand pose recognition is improved, but processing time increases significantly
Solution Approach 1:
The patent segments the processing of multiple views by performing target detection only on a first view to obtain hand detection boxes, while using estimation algorithm on a second view to obtain hand estimation boxes. This segmentation reduces the computational burden while maintaining adequate accuracy through the combination of detected and estimated results.
Solution Approach 2:
The patent introduces an estimation algorithm as an intermediary method to generate hand estimation boxes on the second view, which then serves as a substitute for full target detection. This intermediary approach provides a time-efficient alternative that still contributes useful information to the final hand pose recognition.
2Reliability
If all hands appearing in the picture are processed, then completeness of hand pose recognition is improved, but data redundancy increases leading to higher processing time
Solution Approach 1:
The patent extracts and processes only the relevant hand detection boxes and estimation boxes from the multiple views, rather than processing all hands in the picture. By selectively taking out the necessary detection results and filtering out redundant information, the system maintains completeness for the target hand while reducing processing time.
3Loss of time
If estimation algorithm is used to obtain hand detection boxes, then processing time is reduced, but accuracy of hand joint point location deteriorates especially when wrist flips
Solution Approach 1:
The patent merges the results from target detection on the first view with the results from estimation on the second view. By combining the detected hand boxes (which are accurate) with the estimated hand boxes (which are time-efficient), the system achieves a balance between processing time and accuracy, compensating for the weaknesses of each individual method.
Solution Approach 2:
The patent applies partial target detection (only on the first view) and partial estimation (on the second view) rather than applying either method to all views. This partial application of each method allows the system to achieve adequate accuracy while significantly reducing processing time compared to full detection on all views.
Data Source
AI summary
A hand pose recognition method is performed by a computer device, including: acquiring a current frame of a multi-lens video of a target object; performing hand detection on a first view of the current frame to obtain a first lens detection result; performing hand estimation on a second view of the current frame to obtain a second lens estimation result; removing, from the hand detection boxes in the first view and the hand estimation boxes in the second view, redundant boxes corresponding to redundant hands, and then performing hand joint point recognition on remaining boxes to obtain two-dimensional joint points; converting the two-dimensional joint points into three-dimensional joint points in a three-dimensional hand coordinate system; and converting the three-dimensional joint points in the three-dimensional hand coordinate system into three-dimensional joint points of the current frame in a world coordinate system according to pose estimation parameters corresponding to the current frame.