Depth Estimation Network Training for Moving-Object Pose Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The PoseNet algorithm model inaccurately estimates posture transformation matrices for moving objects due to inconsistencies between the camera's RT and the object's RT, leading to errors in posture estimation.
Innovation Solution
A method for training a depth estimation network that includes an image acquiring module, depth acquiring module, posture acquiring module, generating module, determining module, and adjusting module, which adjusts the network parameters to improve accuracy by learning and adjusting pixel posture transformations for moving objects using a depth estimation sub-network and posture estimation sub-network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the PoseNet algorithm model is used for posture estimation, then the posture transformation matrix is correct for static objects, but there is an error in the posture transformation matrix for moving objects
Solution Approach 1:
The patent applies dynamics by transitioning from a static posture estimation model to a dynamic one that can handle moving objects. The depth estimation sub-network and posture estimation sub-network are trained together with motion-aware loss functions that account for object movement between frames, enabling the model to adapt to dynamic scenes while maintaining accuracy.
Solution Approach 2:
The patent segments the posture estimation problem into two independent sub-networks: a depth estimation sub-network that estimates depth information and a posture estimation sub-network that estimates posture transformation matrices. This segmentation allows each sub-network to specialize in its specific task, improving overall accuracy for moving objects while maintaining the benefits of the original PoseNet architecture.
2Ease of manufacture
If the camera's RT is used directly for moving objects, then the processing is simple, but there is an inconsistency between the camera's RT and the object's RT
Solution Approach 1:
The patent introduces an intermediary depth estimation sub-network that processes the camera's RT to generate depth information, which then serves as input to the posture estimation sub-network. This intermediary step reconciles the camera's RT with the object's RT by incorporating depth-aware processing, ensuring consistency while maintaining computational efficiency.
Solution Approach 2:
The patent implements a feedback mechanism where the depth estimation results are fed back into the posture estimation process. The motion-aware loss function uses ground truth depth information to guide the depth estimation sub-network, creating a feedback loop that improves the consistency between camera RT and object RT iteratively during training.
Data Source
AI summary
A method acquires a first image and a second image of a target object being inputted into the depth estimation network for outputting a depth image. A pixel posture conversion relationship between the first image and the second image is obtained. The pixel posture conversion relationship includes a position relationship between each first pixel in the first image and a second pixel in the second image, which correspond to a same part of the target object. A restored image is generated based on the depth image, the pixel posture conversion relationship, and pre-obtained camera parameters. A loss of the depth estimation network is determined based on a difference between the first image, the depth image, the restored image, and the second image for adjusting the parameters of the depth estimation network. A training apparatus, and an electronic device applying the method are also disclosed.