Industrial Scene Rendering Using Metadata Instead of Video Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for rendering scenes in industrial metaverse environments face challenges with high resource intensity, network lags, and delays, especially in scenarios with multiple participants and intermittent connectivity, leading to degraded user experience.
Innovation Solution
A method and system that utilizes machine learning models to identify real-world scenes, convert frames into metadata, and transmit this metadata for efficient rendering in a computer simulated environment, fetching avatars from databases based on metadata descriptions, and generating animations using generative neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-configuration GPU servers and high-speed networking are used for photorealistic rendering, then rendering quality and realism are improved, but resource consumption and energy usage increase significantly
Solution Approach 1:
The patent uses pre-rendered photorealistic 3D models as copies of real-world objects. Instead of rendering scenes in real-time from scratch, the system captures real-world environments, creates detailed 3D models with photorealistic textures and lighting, and stores them in a library. During metaverse simulations, these pre-rendered models are reused and combined, significantly reducing computational energy requirements while maintaining high visual quality.
Solution Approach 2:
The system performs scene capture, 3D modeling, and photorealistic rendering in advance during an offline phase. All complex computational work including lighting calculations, shadow generation, and texture mapping is completed beforehand. The captured scenes are stored as reusable assets that can be quickly assembled and displayed during real-time simulations without requiring heavy GPU resources.
2Reliability
If real-time photorealistic rendering is performed with multiple participants, then immersion and collaboration quality are improved, but network latency and delays increase
Solution Approach 1:
The patent divides the virtual environment into discrete, pre-rendered scene segments or modules. Each scene is captured and modeled as an independent unit that can be individually loaded and displayed. This segmentation allows the system to transmit only the necessary scene components over the network rather than entire environments, reducing data transmission time and latency while maintaining high visual quality for collaborative interactions.
Solution Approach 2:
The system uses lightweight, pre-processed scene data structures that are optimized for rapid network transmission. Instead of transmitting complex real-time rendering data, the system sends compressed references to pre-rendered assets and lightweight scene graph information. These data structures are designed for efficient serialization and deserialization, enabling fast loading and switching between scenes with minimal network overhead.
3Measurement precision
If high bandwidth is allocated for transmitting visual data, then scene transmission quality is improved, but network resource consumption increases
Solution Approach 1:
The patent extracts and separates critical scene information from complete visual data during the offline capture phase. The system identifies and extracts key elements such as object positions, lighting conditions, camera viewpoints, and spatial relationships, storing them as compact metadata. During real-time operation, only this extracted essential information needs to be transmitted over the network, dramatically reducing bandwidth requirements while preserving the ability to reconstruct high-quality scenes locally using the pre-rendered asset library.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention provides a system (100), apparatus (110) and method (400) for efficiently rendering one or more scenes in a computer simulated environment from first site to second site in an industrial environment. The method comprises receiving, by a processing unit (202), visual data from data acquisition devices (102) arranged at first site. The method comprises identifying real-world scene from industrial environment in real-time frame-by-frame. The real-world scene is identified using machine learning models on visual data. The method comprises converting each frame identified from scene into metadata with predefined schema and transmitting the metadata to the second site. The method comprises extracting relevant description from metadata at second site in order to render scene frame-by-frame in computer simulated environment. The method comprises fetching avatars from database related to extracted entities in frame based on extracted metadata description. The method comprises rendering scenes in computer simulated environment at second site based on fetched avatars from database.