Monocular 3D Pose Estimation for View-Invariant Ergonomic Risk
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current vision-based ergonomic risk assessment methods rely on 2D joint angle calculations, which are prone to perspective errors due to camera orientation, and are inadequate for real-world industrial settings with complex postures and occlusions, lacking essential keypoints for intricate angle analysis.
Innovation Solution
A vision-based 3D pose estimation system that utilizes a monocular video to perform whole-body 2D and 3D pose estimation, calculating joint angles using a deep learning model trained on a material-handling task dataset, with additional keypoints for neck, wrist, and hand movements, and a custom tracking algorithm to handle complex scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If 2D joint angle calculations are used for ergonomic risk assessment, then the system is simpler and requires fewer computational resources, but the measurement accuracy deteriorates due to perspective errors from camera orientation
Solution Approach 1:
The patent transitions from 2D pose estimation to 3D pose estimation by introducing depth information through a depth map generated by a convolutional neural network. This dimensional transformation allows the system to calculate accurate 3D joint angles that are invariant to camera orientation, resolving the accuracy issue while maintaining practical system complexity through efficient deep learning-based depth estimation.
2Measurement precision
If 3D pose estimation with depth maps is implemented, then joint angle measurement accuracy improves and becomes view-invariant, but device complexity and computational requirements increase
Solution Approach 1:
The patent replaces traditional mechanical or manual depth sensing equipment with a computational approach using a convolutional neural network to generate depth maps from monocular video. This substitution achieves accurate 3D pose estimation without requiring complex multi-camera setups or specialized depth-sensing hardware, thereby improving measurement precision while controlling system complexity through software-based solutions.
3Measurement precision
If manual observation-based ergonomic assessment is performed, then detailed ergonomic analysis can be conducted, but the assessment process becomes time-consuming and reliant on limited ergonomist availability
Solution Approach 1:
The patent implements an automated system that performs ergonomic risk assessment without requiring manual observation by ergonomists. The system automatically captures video, estimates 3D pose, calculates joint angles, and determines ergonomic risk levels through algorithmic processing. This self-service approach maintains detailed ergonomic analysis capability while eliminating time consumption and dependency on limited ergonomist availability.
4Productivity
If automated video analysis is used, then the assessment process becomes faster and less dependent on ergonomist availability, but perspective errors from camera orientation reduce measurement accuracy
Solution Approach 1:
The patent resolves the accuracy issue in automated video analysis by transitioning from 2D to 3D pose estimation. The depth map generated by the convolutional neural network provides the necessary depth information to calculate view-invariant 3D joint angles, eliminating perspective errors while maintaining the efficiency benefits of automated video analysis.
Data Source
AI summary
Disclosed herein are vision-based three-dimensional (3D) pose estimation system and method for ergonomic risk assessment. An example system may comprise a computing device configured to obtain a monocular video capturing motions of a subject performing at least one working activity for a selected duration of time, perform a whole-body two dimensional (2D) pose estimation based at least on extracted frames of the monocular video, perform a whole-body 3D pose estimation based at least on the whole-body 2D pose estimation, calculate joint angles based at least on the whole-body 3D pose estimation, determine a posture score for each identified joint in each frame of the monocular video, and determine an ergonomic risk level of each identified joint based at least upon the posture score.