Sensor-Based Digital Twin Synchronization for Mixed Reality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mechanisms for incorporating real-world physical properties into virtual overlays, such as those used in mixed reality environments, are dependent on optical detectors that can be obscured by environmental conditions, leading to errors in data accuracy.
Innovation Solution
An apparatus utilizing a microcontroller and VR controller to map sensor data from a physical entity, create a digital twin, and transmit real-time updates via unique letter codes to ensure accurate synchronization between the physical and virtual environments, eliminating the need for optical trackers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If optical detectors are used to track physical properties for mixed reality environments, then real-time tracking capability is achieved, but data accuracy deteriorates when environmental conditions obscure the optical data
Solution Approach 1:
The patent introduces sensors as intermediary devices to collect physical property data independently of optical conditions. These sensors act as mediators between the physical entity and the mixed reality system, providing reliable data through alternative means (electromagnetic, acoustic, or other physical fields) that are not affected by optical obscuration, thereby resolving the contradiction between real-time tracking and data accuracy under poor optical conditions
Solution Approach 2:
The patent replaces the optical detection system with a multi-sensor system that uses non-optical physical principles (electromagnetic radiation, acoustic waves, or other physical fields) to measure physical properties. This substitution eliminates the dependency on optical clarity while maintaining real-time tracking capability, thus resolving the contradiction between speed and measurement precision
2Measurement precision
If multiple sensors are integrated into the physical entity to improve data reliability, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent segments the sensor system into distinct functional modules, each responsible for detecting specific physical properties. This segmentation allows for independent optimization of each sensor type, simplified data processing for each sensor category, and easier maintenance, thereby reducing the overall complexity despite having multiple sensors
Solution Approach 2:
The patent employs a universal data processing framework that handles multiple sensor types through a common architecture. The system uses unified protocols for data collection, processing, and synchronization across different sensor modalities, reducing the complexity that would otherwise arise from managing diverse sensor systems separately
3Ease of operation
If optical trackers are used for synchronization, then ease of operation is maintained, but reliability deteriorates in obscured environmental conditions
Solution Approach 1:
The patent introduces non-optical sensors as intermediary devices that maintain system operation under conditions where optical trackers fail. These sensors provide continuous, reliable data about the physical entity's state, ensuring synchronization remains reliable even when optical visibility is poor, thus resolving the contradiction between ease of operation and reliability
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Provides accurate and reliable real-time synchronization between physical and virtual environments, enhancing user experiences in mixed reality applications by ensuring seamless alignment and control of physical entities.
Implementation Method 1
A microcontroller is integrated with the physical entity and includes a first processor and a first tangible, non-transitory memory. The microcontroller is configured to receive sensor data from the plurality of sensors, the sensor data being converted into a respective position value.
Implementation Method 2
Converting the sensor data into a respective position value includes transmitting the sensor data to an analog-to-digital converter for conversion into a respective bit-value.
Data Source
AI summary
An apparatus includes a physical entity operatively connected to a plurality of sensors. A microcontroller is configured to map a respective baseline position of the sensors. A virtual reality (VR) controller is adapted to create a digital twin of the physical entity. The microcontroller receives sensor data from the plurality of sensors and converts the sensor data into a respective position value. The microcontroller is adapted to determine, in a predefined priority order, if the respective position value has deviated from the respective baseline position. The VR controller generates an updated digital twin by updating a respective virtual position value in the digital twin based on a unique letter code. The updated digital twin is transmitted to a virtual reality medium or a mixed reality medium, operation of the physical entity being controlled in real-time based in part on the updated digital twin.