Multi-Camera 2D-to-3D Joint Pose Estimation for Mobile Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing joint estimation techniques, particularly on mobile devices, fail to provide accurate 3D position estimation of bodily joints with acceptable resource requirements, such as computation and power consumption.
Innovation Solution
A method utilizing combinative position-estimation transformations and generative joint data estimation techniques, including generating two-dimensional joint positions from multiple camera views and calculating a three-dimensional position based on rotational and translational transformations, using a machine learning model with components like convolution blocks and inverted residual blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing joint estimation techniques are used, then device complexity is reduced, but 3D position estimation accuracy deteriorates
Solution Approach 1:
The patent segments the joint estimation task into two independent stages: (1) a lightweight 2D joint position estimation model that processes each camera view separately, and (2) a simple geometric transformation module that combines results from multiple cameras. This segmentation allows each component to be computationally efficient while achieving accurate 3D estimation through their coordinated operation.
Solution Approach 2:
The patent transforms the estimation problem from direct 3D estimation to a two-stage process involving 2D estimation followed by geometric transformation to 3D space. By first estimating 2D positions from multiple camera views and then applying rigid body transformations, the system achieves accurate 3D joint positions without requiring computationally intensive direct 3D estimation methods.
2Measurement precision
If existing joint estimation techniques are used, then ease of operation is improved, but 3D position estimation accuracy deteriorates
Solution Approach 1:
The patent divides the computation into separate functional modules: a 2D estimation model that operates on individual camera images and a geometric transformation module that processes the results. This modular segmentation enables the system to maintain low resource consumption while improving accuracy, as each module can be optimized independently for mobile device constraints.
Solution Approach 2:
The patent changes the estimation parameters from direct 3D coordinates to 2D image coordinates followed by geometric transformations. This parameter transformation allows the system to use simpler, more resource-efficient 2D estimation algorithms while achieving accurate 3D results through mathematical transformations that are computationally inexpensive on mobile devices.
3Measurement precision
If direct 3D joint estimation is performed, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent avoids direct 3D estimation by first performing 2D estimation from multiple camera views and then transforming these 2D results into 3D space using geometric transformations. This dimensional approach reduces computational complexity because 2D estimation is less resource-intensive than direct 3D estimation, while still achieving accurate 3D joint positions through the transformation step.
Solution Approach 2:
The patent introduces 2D joint position estimates from multiple camera views as an intermediary representation between the input images and the final 3D joint positions. This intermediary 2D estimation stage serves as a computationally efficient bridge that leads to accurate 3D results without requiring direct complex 3D estimation algorithms.
Data Source
AI summary
A system and a method are disclosed for joint pose estimation. In some embodiments, a method includes: generating a first two-dimensional joint position estimate relative to a first camera in a first camera position; generating a second two-dimensional joint position estimate relative to a second camera in a second camera position; generating an estimated three-dimensional joint position, and transmitting the generated three-dimensional joint position estimate. The estimated three-dimensional joint position may be based at least on: a rotation transformation between the first camera position and the second camera position, a generated translational transformation between the first camera position and the second camera position, the first two-dimensional joint position estimate, and the second two-dimensional joint position estimate.