Markerless Motion Capture Using Neural Networks for Biomechanical Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional markerless motion capture systems lack the accuracy achieved by marker-based systems, requiring extensive setup and multiple systems for biomechanical analysis, and are limited by the need for external markers and specialized equipment.
Innovation Solution
A markerless 3D motion capture system utilizing deep neural networks and sensor fusion techniques, combining convolutional neural networks (CNN) and recurrent neural networks (RNN) to transform 2D video data into 3D kinematic data without external markers, using commercial video components and minimal cameras, and integrating biomechanical sensor data for enhanced accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If markerless motion capture systems are used, then setup time and equipment requirements are reduced, but measurement precision deteriorates
Solution Approach 1:
The patent replaces the mechanical marker-based system with a machine learning-based visual system. Convolutional neural networks process video images to detect body keypoints and infer 3D pose, eliminating the need for physical markers while achieving comparable measurement precision through learned biomechanical models
Solution Approach 2:
The system transforms 2D image parameters into 3D kinematic parameters through neural network processing. By changing the parameter space from 2D visual features to 3D biomechanical states, the system achieves accurate motion capture without markers, resolving the contradiction between setup simplicity and measurement precision
2Measurement precision
If marker-based systems are used, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent extracts and removes the markers from the motion capture system, keeping only the essential function of tracking body motion. The neural network learns to infer motion from visual data alone, eliminating the need for specialized marker equipment while maintaining measurement precision through learned biomechanical relationships
Solution Approach 2:
The system uses universal video cameras that can capture motion without requiring specialized marker-based equipment. The machine learning model provides multi-functionality by handling both 2D pose estimation and 3D reconstruction, replacing multiple specialized systems with a single versatile platform
3Ease of operation
If conventional markerless systems are used, then ease of operation is improved, but measurement precision deteriorates
Solution Approach 1:
The patent replaces simple visual tracking with a sophisticated machine learning system that infers 3D pose from 2D images. The neural network learns complex biomechanical relationships during training, enabling accurate measurement while maintaining ease of operation through automatic keypoint detection and pose estimation
4Measurement precision
If multiple systems are used for biomechanical analysis, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent merges video capture, pose estimation, 3D reconstruction, and biomechanical analysis into a single integrated system. The neural network performs multiple functions simultaneously - detecting keypoints, estimating pose, and inferring 3D motion - replacing multiple separate systems while maintaining comprehensive biomechanical analysis capability
Data Source
AI summary
A method of using a learning machine to provide a biomechanical data representation of a subject based on markerless video motion capture. The learning machine is trained with both markerless video and marker-based (or other worn body sensor) data, with the marker-based or body worn sensor data being used to generate a full biomechanical model, which is the “ground truth” data. This ground truth data is combined with the markerless video data to generate a training dataset.