Multi-View Human Pose Fusion for Occlusion-Robust Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional human motion estimation methods using RGB-D cameras are costly, require joint point markers, and suffer from reduced accuracy due to occlusions, especially when testers cross their arms or turn sideways.
Innovation Solution
A multi-view fusion human motion estimation method utilizing two Azure Kinect DK cameras with distributed progressive Gaussian filtering, involving data synchronization, preprocessing, and core computing to enhance accuracy and reduce costs without markers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If a single RGB-D camera is used for human motion estimation, then the cost is reduced and the system is simplified, but the accuracy of occluded joint points deteriorates when the tester crosses arms or turns sideways
Solution Approach 1:
The system segments the observation task across multiple cameras positioned at different locations. Each camera captures a specific view of the tester, and the data from multiple segmented views are fused to achieve complete coverage of all joint points, including those that would be occluded in any single view.
Solution Approach 2:
The system transitions from single-view (2D image plane) to multi-view (3D spatial arrangement of multiple cameras). By adding the spatial dimension of multiple camera positions, the system achieves comprehensive observation of the tester's pose without occlusion, resolving the accuracy problem while maintaining cost-effectiveness.
2Measurement precision
If professional human pose capture systems with more than a dozen cameras are used, then the accuracy of human pose data is improved, but the hardware cost and system complexity increase significantly
Solution Approach 1:
The system applies partial action by using only the minimum necessary number of cameras (two RGB-D cameras) to achieve the required measurement precision. This is excessive compared to a single camera but far less than a dozen cameras, optimizing the balance between accuracy and complexity.
Solution Approach 2:
The system merges the data from multiple cameras through multi-view fusion algorithms. By combining the partial observations from two cameras positioned at different angles, the system achieves complete and accurate pose estimation without requiring a large number of cameras.
3Measurement precision
If joint point markers are worn by testers for accurate motion capture, then the measurement accuracy is improved, but the convenience and comfort of testers deteriorate
Solution Approach 1:
The system replaces the mechanical marker-based approach with an optical/computer vision-based approach. RGB-D cameras capture the tester's pose through image processing and 3D reconstruction algorithms, eliminating the need for physical markers while maintaining measurement accuracy.
Solution Approach 2:
The tester's own body features (natural joints and contours) serve as the detection targets. The system processes the tester's natural appearance in the images to identify joint points, making the tester themselves the source of measurement information rather than requiring external markers.
Data Source
AI summary
Disclosed is a multi-view fusion human motion estimation method based on distributed progressive Gaussian filtering. Firstly, data acquisition is performed by means of two Azure Kinect DK cameras and initial data is determined; then, data filtering and fusion processing is performed, the initial data of the two Azure Kinect DK cameras is classified and processed by using Mahalanobis distance, the measurement information of the Azure Kinect DK cameras that are greatly affected by visual occlusion is filtered out and discarded, and the measurement information of one Azure Kinect DK camera that is determined to be less affected by visual occlusion is guided by the measurement information of the other Azure Kinect DK camera to undergo progressive filtering fusion, thereby achieving the effect of implicit compensation; finally, global fusion is performed, thereby improving the accuracy of human pose estimation.