Markerless Motion Capture Using Video Keypoint Propagation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional motion capture is a time-consuming and expensive process that requires specialized equipment and controlled studio environments, limiting its accessibility and practicality for capturing motion in various locations.
Innovation Solution
A markerless motion capture system that uses an image capture apparatus to analyze video frames, estimate keypoint locations, and convert them into digital media files in formats like BVH and FBX, enabling the transformation of physical motion into digital media without the need for specialized suits or highly controlled environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional motion capture is used with specialized equipment and controlled studio environments, then measurement precision and reliability are improved, but device complexity and cost increase significantly
Solution Approach 1:
The patent uses video recordings as copies of physical motion to create digital motion capture data. Instead of requiring specialized motion capture equipment, the system captures motion through standard video cameras and then extracts motion information from the video frames through image processing and machine learning algorithms, effectively copying the motion data from visual records.
Solution Approach 2:
The patent replaces the mechanical motion capture system (with physical markers and specialized cameras) with a computational approach using standard video cameras and software-based keypoint detection. The mechanical tracking system is substituted with image processing and neural network-based pose estimation that runs on conventional computing hardware.
2Measurement precision
If conventional motion capture is used with specialized suits and markers, then measurement precision is improved, but ease of operation and accessibility worsen
Solution Approach 1:
The system captures motion by copying visual information from video recordings rather than requiring physical markers or specialized suits. The motion data is extracted computationally from standard video footage, making the process accessible to anyone with a video camera and eliminating the need for specialized equipment on the subject.
Solution Approach 2:
The system performs self-service by automatically detecting keypoints and generating motion capture data from video frames without requiring manual marker placement or specialized preparation. The machine learning models automatically identify and track body parts, eliminating the need for operators to manually annotate or prepare the subject with markers.
3Measurement precision
If conventional motion capture is used in controlled studio environments, then measurement precision and reliability are improved, but adaptability and versatility worsen
Solution Approach 1:
The system achieves universality by using standard video cameras that can function in any lighting and environmental condition rather than requiring specialized motion capture cameras. The same video recording device used for ordinary purposes can also capture motion data, making the system adaptable to diverse locations including outdoor settings, studios, and field environments without specialized infrastructure.
Solution Approach 2:
The system adapts to different environments by adjusting processing parameters and using machine learning models that can handle varying lighting conditions, backgrounds, and camera angles. The keypoint detection algorithms are designed to work across different environmental parameters, allowing motion capture in diverse settings without requiring controlled studio conditions.
4Measurement precision
If conventional motion capture is used with multiple engineers and support staff, then measurement precision and data quality are improved, but productivity and time efficiency worsen
Solution Approach 1:
The system performs self-service by automatically processing video frames and extracting motion data without requiring manual intervention from multiple engineers. The machine learning models automatically detect keypoints, track motion across frames, and generate motion capture output, eliminating the need for teams of specialists to manually process the data while maintaining high quality results.
Solution Approach 2:
The patent replaces the human-based processing system with automated computational algorithms. Instead of engineers manually analyzing video footage and extracting motion data, neural networks and image processing algorithms automatically perform the analysis, significantly increasing processing speed and efficiency while maintaining or improving data quality through consistent algorithmic application.
Data Source
AI summary
A system for markerless motion capture may include an image capture apparatus that obtains video including live movement of one or more subjects. An estimation module analyzes frames of the video and estimates the location of a plurality of keypoints on each of the subjects. A propagation module applies the keypoints corresponding to a first subject to the first subject additional frames of the video. The propagation module does the same for additional subjects in the video. A conversion module converts the plurality of keypoints from each frame of the video into a digital media file having a first file format and then converts the digital media file in the first file format into a digital media file having second file format. The system also includes a data storage apparatus configured to save the digital media files in the first and second file formats.