Mixed Reality Simulation With Dynamic Agent Trajectory Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training autonomous systems in real-world environments is unsafe due to the risk of accidents caused by untrained virtual drivers, as they cannot effectively simulate the interactions and decisions that impact real-world scenarios.
Innovation Solution
A mixed reality simulation system that generates realistic scenarios by encoding sensor data and map information to create updated agent trajectories, allowing for the testing and training of virtual drivers in a virtual environment that reacts realistically to the autonomous system's actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the virtual driver is trained in the real world, then the training data is authentic and representative of real scenarios, but it causes safety risks and potential accidents
Solution Approach 1:
The patent creates a virtual copy of the real-world environment by replaying sensor data to reconstruct realistic scenarios. This copying approach allows the virtual driver to be trained in a simulated environment that authentically represents real-world conditions without exposing actual physical systems to safety risks. The sensor data replay technology generates virtual scenarios that preserve the statistical properties and interactions of real environments.
Solution Approach 2:
The patent introduces sensor data replay as an intermediary between the real world and the virtual driver training process. Instead of directly training in the real world, the system uses recorded sensor data to create a mediated virtual environment that bridges the gap between real-world authenticity and safe simulation. This intermediary layer allows realistic training while eliminating direct safety hazards.
2Object-affected harmful factors
If sensor data is merely replayed to create a virtual world, then safety risks are eliminated, but the virtual world fails to capture the effects of decisions made by the virtual driver on the real world
Solution Approach 1:
The patent transforms the static sensor data replay into a dynamic simulation system where the virtual driver's decisions actively influence the scenario evolution. Instead of simply replaying fixed recordings, the system allows the virtual driver to make decisions that dynamically alter the virtual environment, creating realistic cause-and-effect relationships. This dynamic approach ensures that the virtual world responds to agent actions in ways that accurately reflect real-world physics and interactions.
Solution Approach 2:
The patent implements feedback mechanisms where the virtual driver's decisions generate consequences that are fed back into the simulation. The sensor data replay framework is enhanced to capture how virtual agent actions affect the environment and other agents, creating closed-loop scenarios where decisions have visible impacts. This feedback loop ensures the virtual world realistically reflects the effects of the virtual driver's choices while maintaining safety.
3Reliability
If a realistic virtual world is created using sensor data, then the training environment is authentic, but it does not capture the interactive decisions and their effects on other agents
Solution Approach 1:
The patent segments the simulation system into multiple independent virtual agents, each capable of making decisions and reacting to the environment. By dividing the complex interactive system into discrete agent components, the patent enables realistic multi-agent interactions where each agent can independently process sensor data, make decisions, and affect the virtual world. This segmentation allows the virtual environment to authentically model diverse agent behaviors and their interactions.
Solution Approach 2:
The patent creates a universal simulation framework that can handle multiple types of agents (vehicles, pedestrians, cyclists) and various interaction scenarios within a single system. The sensor data replay technology is designed to be multi-functional, supporting different agent types, decision-making models, and interaction patterns. This universality allows the virtual environment to authentically represent diverse real-world scenarios while maintaining a cohesive simulation architecture.
Data Source
AI summary
A method includes obtaining, from sensor data, map data of a geographic region and multiple trajectories of multiple agents located in the geographic region. The agents and the map data have a corresponding physical location in the geographic region. The method further includes determining, for an agent, an agent route from a trajectory that corresponds to the agent, generating, by an encoder model, an interaction encoding that encodes the trajectories and the map data, and generating, from the interaction encoding, an agent attribute encoding of the agent and the agent route. The method further includes processing the agent attribute encoding to generate positional information for the agent, and updating the trajectory of the agent using the positional information to obtain an updated trajectory.