AR Virtual Crowd Rendering with LiDAR Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems fail to effectively populate a virtual crowd in real-time using augmented reality (AR) and virtual reality (VR) for live events, where user actions are based on motion capture, providing a meaningful and immersive audience experience.
Innovation Solution
A system comprising a computing device and an augmented reality system that maps a venue in 3D, receives user tracking data, processes it, and populates digital crowd members into the 3D view, allowing for real-time customization of their appearance and behavior, with geographically accurate sunlight simulation and dynamic lighting, using LiDar remote sensing and a TCP network for real-time propagation of commands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a virtual crowd is populated in real-time using AR and VR with motion capture, then the immersive audience experience is improved, but the system complexity increases
Solution Approach 1:
The system segments the virtual crowd into multiple digital twin instances, each representing an individual audience member. These segmented virtual entities can be independently controlled and rendered, allowing the complex task of creating a realistic crowd to be divided into manageable units that can be processed in parallel
Solution Approach 2:
The system creates digital twin copies of real audience members by capturing their movements and appearances through motion capture technology. These digital copies replicate the behavior and visual characteristics of physical attendees, enabling a photorealistic virtual crowd experience without requiring physical presence
2Adaptability or versatility
If digital crowd members are placed into 3D view with real-time customization, then the fan engagement is improved, but the processing time increases
Solution Approach 1:
The system performs preliminary actions by pre-processing and storing audience member data, motion capture templates, and customization parameters before the event begins. This allows rapid instantiation and customization of digital twins during the event without requiring extensive real-time computation
Solution Approach 2:
The system implements dynamic customization of digital crowd members based on real-time event data and user preferences. Virtual audience members can be dynamically adjusted to reflect different demographics, team preferences, and behavioral patterns, enabling adaptive fan engagement without static preconfiguration
3Manufacturing precision
If motion capture data is processed and combined with additional data to form digital crowd members, then the realism of virtual audience is improved, but the data processing complexity increases
Solution Approach 1:
The system merges multiple data sources including motion capture data, appearance data, demographic information, and behavioral patterns into unified digital twin models. This integration combines heterogeneous data types to create comprehensive virtual representations that capture both visual fidelity and behavioral realism
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables the creation of a realistic and immersive virtual audience that enhances fan engagement and loyalty, providing an engaging experience by simulating a dynamic and interactive virtual crowd that reacts to events in real-time, boosting fan interaction and loyalty.
Implementation Method 1
mapping, via a processor of a computing device and via a light detection and ranging (LiDar) remote sensing method, a venue of a first location hosting an event in a three-dimensional (3D) view
Data Source
AI summary
The present disclosure describes a software-based solution for rendering digital crowds in real-time. The system at large is a network of machines that processes and ingests broadcast camera feeds and tracking data and then leverages an augmented reality system for compositing and tracking platforms to render tens of thousands of crowd members on top of live footage.