A method and system for automatic processing of motion tracking data

Abstract

The invention comprises a machine learning-based optical system and methods for performing motion capture. The system comprises an optical sensor setup that provides raw, unstructured landmark inputs and relies on a neural network, trained for balanced regression, to denoise, label and complete them, estimating a richer set of refined landmark positions. The system and methods automatically calibrates the captured subject, and automatically refines the captured motions, improving the naturalism, smoothness and correcting capturing artifacts. The system and methods benefit from a balanced regression training scheme to improve said neural network's performance on long-tailed motion datasets.