Mixed Reality Scene Updates for Low-Latency Object Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional mixed reality content generation systems suffer from high latency due to long analysis times using deep neural networks and cyclic processing, leading to inconsistencies between real and virtual worlds.
Innovation Solution
An apparatus and method that perform operations for analyzing frame images and generating virtual objects on a per-virtual object basis, allowing immediate updates in the virtual scene without waiting for all objects to be generated, and rendering the scene in accordance with a display cycle.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural network model is used to analyze images, then analysis accuracy is improved, but analysis time increases significantly
Solution Approach 1:
The patent segments the image analysis process into multiple worker threads that process different regions or objects in parallel. Each worker thread independently analyzes specific portions of the image using the deep neural network model, allowing simultaneous execution of multiple analysis operations without sequential delays.
Solution Approach 2:
The patent performs preliminary actions by pre-processing images and preparing analysis data before the main analysis cycle. It also pre-loads model parameters and maintains ready-state worker threads that can immediately begin analysis when new images are captured, eliminating cold-start delays.
2Stability of the object's composition
If virtual scene generation follows a specific cycle, then system stability is improved, but latency increases due to waiting for cycle completion
Solution Approach 1:
The patent implements dynamic virtual scene generation where the system adapts its processing cycle based on real-time conditions. When objects are detected and analyzed, their corresponding virtual scenes are generated immediately rather than waiting for the next fixed cycle, creating a dynamic response system that balances stability with low latency.
Solution Approach 2:
The patent ensures continuous useful action by maintaining worker threads that continuously analyze images and generate virtual scenes without idle periods. The system processes multiple images concurrently through parallel workers, ensuring that analysis and generation operations continue uninterrupted, eliminating waiting time while maintaining systematic operation.
3Stability of the object's composition
If all virtual objects are generated before updating the scene, then consistency is improved, but productivity decreases due to sequential processing
Solution Approach 1:
The patent segments the virtual scene into multiple independent objects or regions that can be processed and updated separately. Each detected object triggers the generation of its corresponding virtual scene portion independently, allowing parallel updates without requiring complete scene generation before any updates occur.
Solution Approach 2:
The patent maintains continuous useful action by updating virtual scenes as soon as object analysis is complete, rather than waiting for all objects. Multiple worker threads continuously generate and update virtual scene portions in parallel, maximizing processing throughput while maintaining overall scene coherence through synchronized rendering.
Data Source
AI summary
Described herein are an apparatus and method for providing mixed reality content. The apparatus for providing mixed reality content includes: memory configured to store a program and data required for providing mixed reality content; and a controller provided with at least one processor, and configured to operate by executing the program stored in the memory, to receive a captured frame image, and to display a virtual scene, including a plurality of virtual objects generated based on the results of analyzing the frame image, on a display. The controller performs operations required for analyzing the frame image and generating a virtual object on a per-virtual object basis, updates a generated virtual object in the virtual scene regardless of whether generation of another virtual object is completed, and renders the virtual scene and displays it on the display in accordance with a display cycle.