Markerless Performer Capture for Interactive Virtual Characters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Live performances often lack audience participation, limiting the value of attending such events.
Innovation Solution
Systems and methods that integrate real-world performers into interactive content by capturing their movements using markerless motion capture techniques, generating three-dimensional models, and rendering them into virtual characters within interactive content, allowing audience interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional one-way performance delivery is used, then performer presentation is simple and direct, but audience participation and engagement are limited
Solution Approach 1:
The patent creates virtual copies of real-world performers using motion capture technology. Cameras track performer movements and generate three-dimensional virtual characters that replicate the performer's actions in real-time within the interactive content, enabling audience interaction without requiring the actual performer to be physically present in the digital environment
Solution Approach 2:
The system introduces a motion capture and rendering intermediary layer between the real performer and the audience. This intermediary captures performer movements, transforms them into virtual character animations, and delivers them to audience devices, enabling two-way interaction while maintaining the performer's physical safety and comfort
2Adaptability or versatility
If performers are required to directly interact with audience devices or virtual environments, then audience engagement increases, but performer comfort and safety deteriorate due to exposure to crowds and technology
Solution Approach 1:
The patent creates virtual copies of real-world performers using motion capture technology. Cameras track performer movements and generate three-dimensional virtual characters that replicate the performer's actions in real-time within the interactive content, enabling audience interaction without requiring the actual performer to be physically present in the digital environment
Solution Approach 2:
The patent extracts the performer's motion data from the physical environment and separates it from the digital interaction environment. By capturing only the movement information through cameras and sensors while leaving the performer in the safe physical venue, the system eliminates exposure to crowds and technological hazards in the digital space
3Ease of operation
If markerless motion capture is used to capture performer movements, then ease of operation improves, but measurement precision may be affected compared to marker-based systems
Solution Approach 1:
The patent replaces the mechanical marker-based motion capture system with an optical computer vision system. Instead of requiring physical markers attached to the performer's body, the system uses cameras and image processing algorithms to automatically detect and track key body points, eliminating the need for markers while achieving sufficient precision for virtual character animation
Data Source
AI summary
Systems, methods, and apparatuses disclosed herein can incorporate one or more real-world performers that are performing within a venue into interactive content. These systems, methods, and apparatuses can identify one or more joints or ligaments, for example, left shoulders, right knees, among others, of the one or more real-world performers from the image, or the series of images, of the one or more real-world performers. These joints or ligaments can be represented as one or more performer markers. These systems, methods, and apparatuses can generate one or more three-dimensional models of the one or more real-world performers in a three-dimensional space from the one or more performer markers. These systems, methods, and apparatuses can apply the one or more three-dimensional models of the one or more real-world performers to the one or more virtual characters in the three-dimensional space. These systems, methods, and apparatuses can render the one or more virtual characters from the three-dimensional space into a two-dimensional space of the interactive content to integrate the one or more real-world performers into the interactive content.