Neural Motion Estimation for Moving Cameras and Articulated Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods struggle to accurately decouple the motion of articulated objects from the motion of a moving camera in video, as the entangled motions complicate estimation and current solutions fail to provide effective joint estimation.
Innovation Solution
A method using a neural network to jointly estimate the motion of a camera and articulated objects in video, employing supervised gradient descent and global optimizations to refine the initial estimation, incorporating human motion priors and background scene features to disentangle the motions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current motion estimation methods are used on video captured by a moving camera, then the estimation process becomes computationally complex and inaccurate, but the entangled camera and object motions make it difficult to separate and estimate each independently
Solution Approach 1:
The patent segments the entangled motion estimation problem into two separate estimation tasks: camera motion estimation and articulated object motion estimation. This is achieved by independently estimating camera pose from video frames and separately estimating object pose, thereby reducing the overall complexity while improving accuracy of each individual estimation
Solution Approach 2:
The patent introduces an intermediary coordinate system transformation approach, where motions are estimated in different coordinate frames (camera coordinate system vs. world coordinate system) and then transformed to a common reference frame. This intermediary transformation enables accurate separation and recombination of camera and object motions
2Reliability
If human motion estimation is performed from videos captured by a static camera, then the estimation is highly under-constrained and ambiguous, but adding camera motion introduces additional spurious correspondences that further complicate the estimation
Solution Approach 1:
The patent inverts the traditional approach by first estimating camera motion from the video frames, then using this camera motion estimation to compensate for camera movement before estimating object motion. This inversion of the estimation sequence (camera first, then object) provides constraints that reduce ambiguity and improve reliability
Solution Approach 2:
The patent changes the parameter representation by estimating motions in different coordinate systems and using transformation parameters to relate them. By parameterizing camera and object motions separately with respect to different reference frames, the patent reduces the coupling between the two estimation problems
Data Source
AI summary
Estimating motion of a human or other object in video is a common computer task with applications in robotics, sports, mixed reality, etc. However, motion estimation becomes difficult when the camera capturing the video is moving, because the observed object and camera motions are entangled. The present disclosure provides for joint estimation of the motion of a camera and the motion of articulated objects captured in video by the camera.