Digital twin real-time synchronous rendering method and system based on WebGPU

Abstract

The invention discloses a digital twinborn real-time synchronization and rendering method and system based on a WebGPU, and belongs to the technical field of digital twinborn and high-performance Web graphic computing. Aiming at the problems of performance bottleneck and synchronization delay in the existing scheme, the invention constructs a layered decoupling architecture: establishing a heterogeneous multi-granularity space-time state descriptor to realize twin unified abstraction; an asynchronous differential synchronization protocol based on a calculation shader is designed, and low-overhead increment synchronization is achieved; a viewport-driven progressive detail rendering strategy is provided, and model resources are dynamically scheduled through rendering and calculation pipeline collaboration. The system comprises a descriptor generation module, a synchronization agent module, a resource scheduler and a collaborative rendering engine. According to the method, the parallel capability of the WebGPU is fully utilized, and the synchronization efficiency and the rendering performance of the digital twin system in the Web environment are improved.

Description

Technical Field

[0001] This invention relates to the fields of digital twin technology, high-performance graphics computing, and web front-end engineering technology, specifically to a method and system for real-time state synchronization and high-performance rendering of high-concurrency, heterogeneous, multi-granular digital twins based on a new generation of Web Graphics API (WebGPU). Background Technology

[0002] Digital twins, as a core technology connecting the physical and information worlds, have been widely applied in smart cities, industrial IoT, autonomous driving simulation, and other fields. The core challenge lies in how to efficiently, in real-time, and with high fidelity, digitally map and visualize large-scale, heterogeneous physical entities (such as buildings, equipment, sensors, and vehicles) with different spatiotemporal granularities (such as city-level macro-level overviews and equipment-level micro-level details).

[0003] Currently, digital twin solutions based on web browser environments primarily rely on WebGL APIs (such as frameworks like Three.js and Babylon.js). While WebGL provides basic hardware-accelerated graphics capabilities, its architecture has inherent limitations:

[0004] The separation between rendering and computing power:

[0005] The WebGL shader language was originally designed for graphics rendering, but it lacks efficient support for general computing. As a result, complex computational tasks such as state synchronization logic, physics simulation, and large-scale culling can only be executed serially or pseudo-parallel by the CPU, which becomes a performance bottleneck.

[0006] Resource scheduling and rendering calls incur significant overhead.

[0007] For a large number of heterogeneous twins, frequent JavaScript driver calls, rendering state switching, and texture binding result in huge driver overhead between the CPU and GPU.

[0008] Low data transmission efficiency:

[0009] Real-time data stream and twin state synchronization usually rely on the CPU to perform full or simple incremental comparisons, which results in high latency in data serialization / deserialization and bus transmission, making it difficult to handle high-concurrency data updates.

[0010] As the next-generation standard for WebGL, WebGPU provides modern GPU features such as compute shaders, indirect drawing, and storage buffers, supporting explicit control and low-overhead rendering, thus laying the foundation for high-performance parallel computing and rendering collaboration on the web. However, current technologies have not yet fully integrated the computing and rendering capabilities of WebGPU to build an efficient end-to-end digital twin architecture.

[0011] Therefore, there is an urgent need to develop an integrated method and system that deeply integrates the general computing and rendering capabilities of WebGPU, targeting high-concurrency, heterogeneous, and multi-granular digital twin scenarios, to achieve ultra-low latency state synchronization and ultra-high throughput rendering. Summary of the Invention

[0012] To overcome the shortcomings of existing WebGL-based digital twin technologies in handling high-concurrency, large-scale, heterogeneous, and multi-granularity scenarios, such as high synchronization latency, low rendering performance, and excessive CPU load, this invention provides a high-performance digital twin solution based on WebGPU that optimizes the entire chain from data synchronization to visual presentation.

[0013] The core of this invention lies in constructing a layered and decoupled system architecture. This architecture uses heterogeneous multi-granular spatiotemporal state descriptors as a unified data abstraction, WebGPU computing and rendering pipeline collaboration as the execution engine, and asynchronous differential state synchronization and viewport-driven multi-granularity rendering as key technical paths to achieve efficient flow from data to pixels.

[0014] Unified Data Abstraction: Heterogeneous Multi-Granularity Spatiotemporal State Descriptors

[0015] This invention proposes for the first time a machine-friendly and standardized digital representation of twins: a heterogeneous, multi-granular spatiotemporal state descriptor. This descriptor is not only a data container but also a hub connecting physical entities, business logic, and GPU rendering resources. Its mathematical expression is as follows:

[0016]

[0017] in:

[0018] It serves as a unique identifier for the entity;

[0019] It is a directed acyclic graph that describes the composition and relationships of entities.

[0020] It is a spatiotemporal state matrix that supports sparse storage;

[0021] This is an array of resource handles, which enables indirect mapping between logic and video memory resources.

[0022] High-efficiency state synchronization: Asynchronous differential state synchronization protocol

[0023] To address the real-time synchronization challenge of high-concurrency data updates, this invention designs an asynchronous differential state synchronization protocol based on the WebGPU computing pipeline. This protocol innovatively offloads state comparison, differential compression, and reassembly tasks from the CPU to the GPU for parallel execution. The specific process includes:

[0024] Parallel differential computation:

[0025] At the sending end, the current state matrix is ​​computed using a shader. With the target state matrix Perform element-wise parallel comparison:

[0026]

[0027]

[0028] in This is the tolerance threshold. This is an indicator function.

[0029] Parallel entropy coding compression:

[0030] Using a parallel prefix sum algorithm based on a storage buffer for incremental data Adaptive Huffman coding is performed to achieve a high compression ratio. satisfy: This significantly reduces the amount of data transmitted over the network.

[0031] Network transmission and parallel reassembly:

[0032] The compressed differential data packets are transmitted to the receiving end via WebSocket / WebRTC, where the compute shader decompresses them in parallel and atomically updates the local state matrix, achieving low-latency, high-throughput state synchronization.

[0033] High-performance rendering: Viewport-driven multi-granular progressive rendering and culling

[0034] To improve the rendering efficiency of large-scale scenes, this invention proposes a viewport-driven multi-granularity progressive detail rendering and culling strategy. This strategy, centered on the computational pipeline, executes in parallel before rendering each frame:

[0035] Parallel frustum culling:

[0036] Perform parallel intersection tests on the axis-aligned bounding boxes of all twins in the scene:

[0037]

[0038] Quickly filter out visible entities.

[0039] Parallel LOD selection:

[0040] Based on the projected area of ​​the entity in the screen space Dynamically select level of detail:

[0041]

[0042] in This is a preset threshold.

[0043] Indirect drawing instruction generation:

[0044] Visible entities and their LOD information are encoded into an indirect drawing parameter buffer, driving the rendering pipeline to complete the full scene drawing with a single drawIndexedIndirect call, completely eliminating the overhead of traditional drawing calls.

[0045] To implement the above method, the present invention provides a system comprising:

[0046] Descriptor generation module: Constructs and manages heterogeneous multi-granularity spatiotemporal state descriptors;

[0047] Synchronous proxy module: Executes asynchronous differential state synchronization protocol;

[0048] Resource scheduler: Implements dynamic loading of multi-granularity resources based on viewport prediction and feedback adjustment;

[0049] Collaborative rendering engine: Integrates computation and rendering pipelines to achieve a collaborative working mechanism of "computation first, rendering follow-up". Attached Figure Description

[0050] Figure 1 System overall architecture diagram;

[0051] Figure 2 Schematic diagram of heterogeneous multi-granularity spatiotemporal state descriptor structure;

[0052] Figure 3 Asynchronous differential state synchronization protocol process and data flow diagram;

[0053] Figure 4 Flowchart of viewport-driven multi-granularity progressive detail rendering and culling strategy execution;

[0054] Figure 5 : A schematic diagram illustrating the collaborative operation of the rendering pipeline and the compute pipeline; Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0056] like Figure 1 As shown, the overall architecture of the system of this invention includes four core modules: a descriptor generation module, a synchronization proxy module, a resource scheduler, and a collaborative rendering engine. These modules run in a modern browser environment that supports WebGPU and interact with the backend digital twin data service platform.

[0057] Descriptor generation module:

[0058] It is responsible for interfacing with physical entity data sources (such as BIM / CAD models, IoT time-series databases, and GIS systems). For each entity, it extracts its static attributes (such as ID, type, and geometric relationships) and dynamic attribute metadata, constructing a structure like... Figure 2 The heterogeneous multi-granularity spatiotemporal state descriptor is shown. The descriptor uses a JSON-Binary hybrid encoding format, and its top-level structure includes:

[0059] entityUUID: A globally unique identifier conforming to RFC 4122.

[0060] topologyGraph: A directed acyclic graph where nodes represent entity components, edges represent spatial or logical relationships between components, and node attributes contain granular-level markers.

[0061] stateMatrix: A two-dimensional tensor where rows represent time series or attribute groups and columns represent specific state variables, supporting sparse storage;

[0062] resourceHandles: An array of resource indices pointing to geometry, material, and texture resources stored in the GPUBuffer.

[0063] Synchronous proxy module:

[0064] This is key to achieving low-latency synchronization. For example... Figure 3 As shown, its workflow is as follows:

[0065] The sender periodically or based on event triggers initiates the state comparison computation shader function in the computation pipeline, which uses the current state matrix... With the target state matrix As input, element-wise or rule-wise comparisons are performed in parallel using workgroups as units, and the output is a differential mask. With incremental data ;

[0066] The differential mask and incremental data are input into the entropy coding compression kernel function, and adaptive Huffman coding is performed using a parallel prefix sum algorithm based on WebGPU storage buffer;

[0067] The compressed differential data packets are transmitted over the network via WebSocket or WebRTC data channels;

[0068] The receiver's computation pipeline calls the state reconstruction kernel function to decompress the data packets in parallel and atomically update the corresponding positions of the local state matrix according to the differential mask.

[0069] Resource scheduler:

[0070] A hybrid resource scheduling algorithm combining a greedy algorithm based on viewport prediction and a feedback adjustment mechanism is adopted, which includes:

[0071] Prediction unit: based on camera motion vectors Predicting the future using historical frame data The set of twins that may enter the viewport within the frame;

[0072] Priority calculation unit: based on the granularity level of the twin. Distance to camera and business logic weight Calculate the priority score for loading each resource. :

[0073]

[0074] Flow control unit: Monitors WebGPU device memory usage With network bandwidth Dynamically adjust the concurrency and detail-level degradation strategies of the resource loading queue.

[0075] Collaborative rendering engine:

[0076] It is the core of the system's execution. For example... Figure 4 and Figure 5 As shown, the workflow for each frame is a tightly coordinated process of "computation first, rendering follow-up":

[0077] Phase 1: Parallel Computation Elimination and LOD Selection

[0078] Upload the view matrix and projection matrix of the current frame to the constant buffer;

[0079] Distribute a compute shader cullAndLOD, with each thread processing one twin;

[0080] The thread reads the world transformation matrix and pre-computed bounding box of the twin, transforms it to clip space, and performs a fast intersection test with the six planes of the view frustum;

[0081] For a visible twin, calculate the pixel radius of its bounding sphere projected in screen space, and determine the LOD level index to be used based on the preset LOD switching threshold table;

[0082] The index of the visible twin, its corresponding LOD level, and the starting position and instantiation data of the LOD resource in the global vertex / index buffer are compactly written into the indirect drawing parameter buffer and the instance data buffer.

[0083] Phase Two: Indirect Drawing and Rendering

[0084] Configure the rendering pipeline state, including vertex shader, fragment shader, blending state, depth stencil state, etc.

[0085] Bind the global vertex buffer, index buffer, and instance data buffer;

[0086] Bind the indirect drawing parameter buffer to the indirect drawing parameter source;

[0087] Calling the renderPassEncoder.drawIndexedIndirect method triggers the GPU to execute the instantiation and drawing of all visible twins;

[0088] In the fragment shader, the correct material parameters and textures are dynamically indexed through resource handles for shading calculations.

[0089] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A high-concurrency heterogeneous multi-granularity digital twin real-time synchronization and rendering system based on WebGPU, characterized in that, include: The descriptor generation module is used to construct and manage heterogeneous multi-granular spatiotemporal state descriptors based on multi-dimensional source data of physical entities. The descriptor is a unified interface between twin data and rendering scheduling. The synchronous proxy module integrates an asynchronous differential state synchronization protocol and deploys a WebGPU computing shader program for parallel state comparison, differential computation and compression at the data transmission layer, as well as state reorganization at the receiving end. The resource scheduler, connected to the descriptor generation module and the synchronization agent module, is used to manage geometric and texture assets at different levels of detail and to perform streaming loading and unloading of resources based on viewport information and load prediction. The collaborative rendering engine is the core module, built on the WebGPU API, and includes a computation pipeline subsystem and a rendering pipeline subsystem. The computation pipeline subsystem performs state synchronization, culling calculations, and instruction list generation, while the rendering pipeline subsystem performs the final geometry drawing and pixel shading based on the generated instruction list.

2. The system according to claim 1, characterized in that, The heterogeneous multi-granularity spatiotemporal state descriptor adopts a layered JSON-Binary hybrid encoding format, and its top-level structure includes: entityUUID: A globally unique identifier conforming to RFC 4122. topologyGraph: A directed acyclic graph where nodes represent entity components, edges represent spatial or logical relationships between components, and node attributes contain granular-level markers. stateMatrix: A two-dimensional tensor where rows represent time series or attribute groups and columns represent specific state variables, supporting sparse storage; resourceHandles: An array of resource indices pointing to geometry, material, and texture resources stored in the GPUBuffer.

3. The system according to claim 2, characterized in that, The asynchronous differential state synchronization protocol implemented by the synchronization proxy module includes the following workflow: The sender periodically or based on event triggers initiates the state comparison computation shader function in the computation pipeline, which uses the current state matrix... With the target state matrix As input, element-wise or rule-wise comparisons are performed in parallel using workgroups as the unit, and the output is a difference mask. With incremental data ; The differential mask and incremental data are input into the entropy coding compression kernel function, and adaptive Huffman coding is performed using a parallel prefix sum algorithm based on WebGPU storage buffer; The compressed differential data packets are transmitted over the network via WebSocket or WebRTC data channels; The receiver's computation pipeline calls the state reconstruction kernel function to decompress the data packets in parallel and atomically update the corresponding positions of the local state matrix according to the differential mask.

4. The system according to claim 1, characterized in that, The resource scheduler employs a hybrid resource scheduling algorithm that combines a greedy algorithm based on viewport prediction with a feedback adjustment mechanism, comprising: Prediction unit: based on camera motion vectors Predicting the future using historical frame data The set of twins that may enter the viewport within the frame; Priority calculation unit: based on the granularity level of the twin. Distance to camera and business logic weight Calculate the priority score for loading each resource. : in These are adjustable weighting coefficients; Flow control unit: Monitors WebGPU device memory usage With network bandwidth Dynamically adjust the concurrency and detail-level degradation strategies of the resource loading queue.

5. The system according to claim 1, characterized in that, The computation pipeline subsystem in the collaborative rendering engine executes a viewport-driven, multi-granularity progressive detail rendering and culling strategy, specifically: At the start of each rendering frame, the compute pipeline initiates a culling and LOD selection kernel function, which reads the bounding boxes of all active twins and the current view-projection matrix. Conservative frustum culling is performed in parallel. For twins that were not removed, their screen space coverage area is considered. Select the rendering resource handle corresponding to the level of detail from the resource scheduler; Based on the selection result, an indirect drawing parameter buffer is dynamically assembled. This buffer contains parameters such as the vertex count, instance count, and first vertex index for each drawing instance, and a corresponding drawing instruction buffer is generated.

6. The system according to claim 5, characterized in that, The rendering pipeline subsystem performs the following operations through the rendering pass encoder: Bind the indirect drawing parameter buffer and the drawing instruction buffer to the rendering pipeline; Set up unified vertex buffers and index buffers, which store geometry data at multiple LOD levels in a data-driven manner; Calling the drawIndexedIndirect method triggers the GPU to perform batch, multi-instance drawing based on indirect parameters. During the fragment shading stage, the corresponding material and texture data are dynamically sampled through the resource handle.

7. A method for real-time synchronization and rendering of high-concurrency heterogeneous multi-granularity digital twins based on WebGPU, characterized in that, The system applied to any one of claims 1-6 includes the following steps: S1: Construct a heterogeneous multi-granularity spatiotemporal state descriptor to digitally abstract physical entities. The descriptor includes an entity identifier for unique identification, a multi-granularity topology graph for describing structural composition and granularity, a spatiotemporal state matrix for encapsulating dynamic attribute sets, and a rendering resource handle for associating rendering resources. S2: Real-time data streams are processed through an asynchronous differential state synchronization protocol. The state comparison kernel function deployed in the computing pipeline is used to calculate the differential increment between the current state of the twin and the target state in parallel. The differential increment is then transmitted after entropy encoding compression. S3: At the receiving end, the compressed differential increment is decoded and reassembled in parallel through the computing pipeline, and the spatiotemporal state matrix is ​​updated based on the reassembled state to achieve high-concurrency, low-latency state synchronization. S4: Based on the updated spatiotemporal state descriptor, a viewport-driven multi-granularity progressive detail rendering and culling strategy is used to perform view frustum culling and detail level selection in the rendering pipeline, generating a list of drawing instructions containing indirect drawing parameters. S5: Submit the drawing instruction list to the rendering pipeline, and combine it with instantiation rendering technology to perform high-performance final image compositing and output.

8. The method according to claim 7, characterized in that, In step S2, the differential increment calculation employs a rule-based parallel comparison method, specifically including: Define a set of state comparison rules, which include absolute tolerance comparison, relative percentage comparison, and logical state transition detection; The state matrix is ​​divided into several comparison blocks of equal size, and each comparison block is assigned a GPU thread group for processing; Each thread group loads the corresponding current state block and target state block into shared memory, performs comparisons in parallel according to predefined rules, and generates a binary differential mask bitmap indicating whether a change has occurred and an increment array storing the changed values.

9. The method according to claim 7, characterized in that, In step S4, the parallel execution process of view frustum culling and LOD (Level of Detail) selection includes: Pre-store the axis-aligned bounding boxes (AABB) and sphere bounding boxes of all active twins in the scene in the GPUBuffer; In the compute shader, each thread processes one or more twins, using the SIMD (Single Instruction Multiple Data) instruction set to perform the intersection test between the AABB and the six planes of the view frustum in parallel; Based on the test results, the index of the visible twin is written into a compact buffer of the list of visible instances; Meanwhile, based on the estimated projected area of ​​the twin in the normalized device coordinates, the appropriate LOD level is determined by looking up a table, and the level index is written into the LOD selection list buffer.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method as described in any one of claims 7-9.

Images