Pose Estimation Using Keypoint Constraints for Occluded Vehicle Cabins
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Pose estimation in vehicles is challenging due to occluding objects like steering wheels and chairs, leading to lower accuracy in existing deep learning-based methods, which are typically trained in free space and fail to generalize to different vehicle configurations.
Innovation Solution
A computer-implemented method that determines candidate positions for human keypoints, generates combinations based on these positions, and incorporates spatial and temporal constraints such as limb length, angle, and movement to improve accuracy, using heatmaps and vector fields for precise pose estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If deep learning-based pose estimation algorithms are used, then the system can process images automatically, but the accuracy decreases due to occluding objects in vehicle environments
Solution Approach 1:
The patent introduces an intermediary refinement process that takes the initial keypoint detections from deep learning algorithms and improves them by incorporating spatial consistency checks, physical constraints (limb length, joint angles), and temporal smoothness constraints. This intermediary step mediates between the automated detection and accurate pose estimation, resolving the contradiction by adding a refinement layer that corrects errors introduced by occlusions.
Solution Approach 2:
The patent changes multiple parameters simultaneously: spatial parameters (keypoint positions are adjusted to satisfy limb length and angle constraints), temporal parameters (pose estimates are smoothed over time to reduce jitter), and confidence parameters (weighting different constraints based on their reliability). These parameter changes enable the system to maintain automation while significantly improving accuracy in occluded scenarios.
2Ease of manufacture
If algorithms are trained in free space, then training data is easier to obtain, but the results fail to generalize to vehicle configurations with occluding objects
Solution Approach 1:
The patent performs preliminary action by pre-computing and storing spatial relationships, physical constraints, and temporal patterns during the training phase. Instead of training on specific vehicle configurations, the system pre-learns general human body mechanics and pose constraints that are invariant across different environments. This preliminary preparation enables the algorithm to adapt to various vehicle configurations without requiring retraining on each specific environment.
Solution Approach 2:
The patent creates a universal pose estimation framework that works across multiple environments (free space, vehicles, different lighting conditions) by incorporating constraints that are universally applicable to human anatomy. The physical constraints (limb lengths, joint ranges) and temporal smoothness constraints are universal properties that hold regardless of the environment, enabling the system to generalize from free-space training data to vehicle environments with occluding objects.
3Device complexity
If individual keypoint positions are determined in isolation, then the processing is simpler, but the overall pose accuracy decreases
Solution Approach 1:
The patent merges individual keypoint determinations by introducing spatial consistency constraints that link keypoints through肢体 relationships (limb lengths, joint angles). Instead of treating each keypoint independently, the system combines them into a coherent pose structure where the position of each keypoint is constrained by its relationship to neighboring keypoints. This merging approach increases accuracy while maintaining computational efficiency through gradient-based optimization that handles all constraints simultaneously.
Solution Approach 2:
The patent segments the pose estimation problem into independent constraint modules (spatial constraints, temporal constraints, physical constraints) that can be applied separately and then combined. Each module handles a specific aspect of pose validation, and the overall solution is obtained by optimizing all segments together. This segmentation allows the system to manage complexity while achieving high accuracy through the cumulative effect of multiple specialized constraints.
Data Source
Figure 1A~1B
Figure 2~4
Figure 3
AI summary
The invention relates to a method and system for pose estimation, the method comprising receiving at least one image frame captured by an imaging device, wherein the imaging device is arranged to image at least one subject; determining one or more candidate positions for each of a plurality of human keypoints, wherein each candidate position is associated with a likelihood that a human keypoint is located at such position; generating one or more combinations of human keypoints based on the determined one or more candidate positions; and determining a pose of each of the at least one subject based at least on the one or more generated combinations of human keypoints. In some embodiments, at least one constraint may be taken into account. The invention further relates to an in-cabin method of subject monitoring, computer program, machine-readable storage medium or data carrier signal.