A WebGPU-based incremental big data visualization rendering method and system
By using an incremental big data visualization rendering method based on WebGPU, the problems of high CPU and GPU data transfer overhead, frequent reconstruction, and high memory consumption in massive data processing are solved. This enables efficient real-time rendering of massive data and smooth browser interaction, supporting applications in multiple industries.
Patent Information
- Application Number
- CN202611259234.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-19
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies suffer from problems when processing massive amounts of data, such as huge overhead for CPU and GPU data transfer, frequent GPU buffer rebuilding, inability to achieve true incremental updates, high memory consumption, lack of dynamic viewport-level scheduling capabilities, and lack of a unified incremental scheduling mechanism for WebGPU, leading to problems such as decreased frame rate, browser lag, and resource waste.
An incremental big data visualization rendering method based on WebGPU is adopted. Through incremental data detection, dynamic viewport region detection, GPU chunk management and WebGPU Compute Shader parallel computing, data difference comparison and GPU parallel computing are achieved, rendering accuracy and task priority are dynamically adjusted, and invalid rendering areas are reduced.
Significantly reduces GPU data transfer overhead, enhances real-time rendering capabilities for massive amounts of data, reduces browser memory usage, improves interactive smoothness, and supports multiple industry application scenarios such as smart cities, industrial internet, and financial risk control.
Smart Images

Figure CN123067979A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of visualization rendering technology, and in particular relates to a WebGPU-based incremental big data visualization rendering method and system. Background Technology
[0002] With the development of smart cities, industrial internet, financial risk control, digital twins, IoT monitoring and other scenarios, front-end visualization systems need to process massive amounts of data in real time, such as millions of data points, real-time streaming logs, dynamic topology relationships, trajectory data, heat distribution data and so on.
[0003] The mainstream front-end visualization solutions in the industry today mainly include the following categories: 1. Canvas-based 2D rendering solutions; 2. SVG-based vector rendering solutions; 3. WebGL-based GPU acceleration solutions; 4. Graphics rendering solutions based on encapsulation frameworks such as ECharts and Three.js.
[0004] Existing technologies typically employ the following processing method: transmitting the entire data to the front end; the front end completes data parsing in one go; the CPU is used to complete layout and calculation; the complete data is resubmitted to the GPU; and a complete redraw is performed.
[0005] However, when the data scale reaches millions or more, existing technologies generally suffer from the following problems: (1) The data transfer overhead between the CPU and GPU is huge. In traditional WebGL rendering solutions, when data changes, it is usually necessary to re-upload the complete buffer data to the GPU, resulting in frequent data copying and GPU buffer rebuilding.
[0006] In high-frequency dynamic scenarios, such as real-time monitoring, log stream display, and IoT data stream scenarios, there will be a significant drop in frame rate.
[0007] (2) Unable to achieve true incremental updates Most existing solutions adopt a global redraw mode, which requires recalculating all nodes, all layers, and all rendered objects even if only local data changes.
[0008] This approach results in: wasted GPU resources; excessively large invalid rendering areas; browser main thread blocking; and page lag.
[0009] (3) High memory usage in large-scale data scenarios Existing visualization solutions typically store all data in the browser's memory.
[0010] In scenarios with massive amounts of data: JavaScript heap memory grows rapidly; GPU memory usage continues to rise; and browsers are prone to crashing.
[0011] (4) Lack of dynamic viewport-level scheduling capability Existing solutions typically cannot dynamically adjust rendering precision based on the user's current viewport area. Even if the user is only viewing a local area, the entire dataset is rendered. This results in: increased unnecessary GPU computation; decreased frame rate; and increased power consumption.
[0012] (5) Lack of a unified incremental scheduling mechanism based on WebGPU Most current front-end visualization frameworks are still based on WebGL. WebGL has limitations in the following aspects: insufficient computing power; inability to perform parallel computing efficiently; lack of Compute Shader; inability to perform GPU-side data preprocessing; and inability to flexibly control the lifecycle of video memory.
[0013] With the development of WebGPU technology, browsers have gradually acquired modern GPU parallel computing capabilities, but the industry currently lacks a WebGPU incremental rendering method for massive data scenarios.
[0014] Therefore, a front-end incremental big data visualization rendering method based on WebGPU is needed to solve the problems existing in the current technology, such as: low overall redraw efficiency; frequent GPU buffer reconstruction; browser lag; insufficient real-time rendering capability for massive data; and unreasonable GPU resource scheduling. Summary of the Invention
[0015] The purpose of this application is to overcome the problems of the prior art by disclosing a WebGPU-based incremental big data visualization rendering method and system, which realizes high-performance real-time rendering in massive data scenarios.
[0016] On the one hand, the objective of this application is achieved through the following technical solution: An incremental big data visualization rendering method based on WebGPU, the big data visualization rendering method includes: S1: Data incremental detection. It receives real-time data streams and compares the differences between newly received data and historical data. It records the unique identifier of each data item through a hash index structure. When a data change is detected, it generates only the corresponding data difference queue. S2: Dynamic viewport region detection, which obtains the current browser viewport information in real time, calculates the data blocks involved in the current viewport through the spatial index structure, and performs rendering on the currently visible area; S3: GPU Chunk Management. The GPU Chunk management adopts the GPU Buffer management mechanism to split massive amounts of data into several GPU Chunks and update the data blocks that have changed. S4: GPU parallel computing processing, using WebGPU Compute Shader to implement GPU parallel computing and output rendering calculation results.
[0017] According to a preferred embodiment, in step S1, the difference comparison content includes: newly added data, deleted data, modified data, data with changes in location, and data with changes in attributes.
[0018] According to a preferred embodiment, the difference queue structure includes: fields and corresponding descriptions; The fields include: id, type, offset, size, and priority; each field is described as: unique data identifier, data change type, GPU buffer offset, data length, and update priority.
[0019] According to a preferred embodiment, the current browser viewport information in step S2 includes: zoom level, scroll position, current display area, camera position, and user interaction behavior.
[0020] According to a preferred implementation, in step S3, each GPU Chunk independently maintains and processes: vertex data, index data, color data, and dynamic attribute data.
[0021] According to a preferred embodiment, in step S4, the Compute Shader is used to perform: vertex coordinate calculation, dynamic particle position update, thermal value calculation, data aggregation, color mapping, and collision detection.
[0022] According to a preferred embodiment, during the execution of the big data visualization rendering method, dynamic LOD rendering adjustment is performed, and the data precision is dynamically switched according to the current scaling level.
[0023] According to a preferred embodiment, the dynamic LOD rendering adjustment is switched based on the following rules: current scaling factor, GPU utilization, current frame rate, browser memory status, and user operation frequency.
[0024] According to a preferred embodiment, during the execution of the big data visualization rendering method, incremental rendering scheduling is performed, and the task execution frequency is dynamically adjusted according to the GPU load.
[0025] According to a preferred implementation, the scheduling rules for incremental rendering dynamically calculate task priority based on the following parameters: user interaction frequency (Fuser); data change frequency (Fdata); current viewport distance (Dview); and current GPU load (Gload). The priority calculation formula is as follows: Priority = α×Fuser + β×Fdata + γ×(1 / Dview) - δ×Gload Where α, β, γ, and δ are weighting coefficients; the higher the Priority, the higher the priority for execution, that is, the task queue is sorted according to Priority.
[0026] On the other hand, this application also discloses: A WebGPU-based incremental big data visualization rendering system, wherein the big data visualization rendering system uses the aforementioned method for visualization rendering; The big data visualization rendering system includes: a data incremental analysis module, a viewport detection module, a GPU Chunk management module, a LOD dynamic control module, a WebGPU rendering scheduling module, and a Compute Shader parallel computing module.
[0027] The aforementioned main solution and its various further alternative solutions can be freely combined to form multiple solutions, all of which are solutions that can be adopted and are claimed in this application. Those skilled in the art, after understanding the solution of this application, will realize that there are many combinations based on the prior art and common general knowledge, all of which are technical solutions to be protected in this application, and will not be exhaustively listed here.
[0028] The beneficial effects of this application are: (1) Significantly reduces GPU data transfer overhead By using the GPU Chunk incremental update mechanism, only the changed areas are updated. Compared to the overall buffer reconstruction method, this can reduce GPU data transfer by approximately 60% to 90%.
[0029] (2) Enhance the real-time rendering capability of massive data This invention implements GPU parallel computing based on WebGPU Compute Shader. Compared with traditional CPU computing solutions: the frame rate is increased by about 2 to 5 times in scenarios with millions of points; GPU utilization is more stable; and browser interaction latency is significantly reduced.
[0030] (3) Reduce browser memory usage Through dynamic viewport management and GPU buffer pool reuse mechanisms, the system only maintains currently active data blocks, reducing browser memory and GPU memory usage.
[0031] (4) Improve the smoothness of interaction This invention utilizes an asynchronous scheduling mechanism and dynamic task priority management to effectively prevent browser main thread lag, thereby improving smoothness in scenarios involving zooming, dragging, and real-time interaction.
[0032] (5) Improve system scalability This invention supports scenarios such as digital twins, smart cities, IoT monitoring, large-screen visualization, financial risk control, real-time trajectory analysis, and network topology monitoring. It has strong industry applicability. Attached Figure Description
[0033] Figure 1 This is a schematic diagram of the system structure of this application; Figure 2 This is a flowchart of the rendering method in this application; Figure 3 This is a flowchart of the dynamic LOD switching process for this application; Figure 4 This is a schematic diagram of the scheduling process for the task in this application. Detailed Implementation
[0034] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.
[0035] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, this application should indicate that unless otherwise specified, the structures, connections, positions, power sources, etc., involved in this application are all things that a person skilled in the art could discover without inventive effort based on existing technology.
[0036] refer to Figures 1 to 4 As shown in the figure, an incremental big data visualization rendering method based on WebGPU is illustrated. The big data visualization rendering method includes the following steps.
[0037] Step S1: Data incremental detection. Receive real-time data streams and compare the differences between newly received data and historical data. Record the unique identifier of each data entry using a hash index structure. When a data change is detected, only the corresponding data difference queue is generated, avoiding the need to re-upload all data.
[0038] Preferably, in step S1, the difference comparison content includes: newly added data, deleted data, modified data, data with changes in location, and data with changes in attributes.
[0039] Furthermore, the difference queue structure includes: fields and corresponding descriptions; The fields include: id, type, offset, size, and priority; each field is described as: unique data identifier, data change type, GPU buffer offset, data length, and update priority.
[0040] Specifically, in step S1, the data difference detection uses a hash mapping structure for incremental detection, as follows.
[0041] Let the current dataset be Dn; the previous dataset be Dn-1; and the unique identifier of the data be id.
[0042] Construction: HashMap<id, DataObject> ; Determine data changes using the following methods: ΔD = Dn - Dn-1 Among them: New data: id Dn-1, Delete data: id For Dn, modifying data: hash(data_new) ≠ hash(data_old). The system only writes ΔD to the GPU update queue, avoiding a full update.
[0043] Step S2: Dynamic viewport region detection. The current browser viewport information is obtained in real time. The data blocks involved in the current viewport are calculated through the spatial index structure (QuadTree or R-Tree). Rendering is performed on the currently visible area, and the non-viewport area enters a dormant state.
[0044] Preferably, the current browser viewport information in step S2 includes: zoom level, scroll position, current display area, camera position, and user interaction behavior.
[0045] In step S2, the viewport visibility calculation process can be performed as follows.
[0046] Let V be the current viewport region; C be the data block set; and B be the data block bounding box.
[0047] If the following is satisfied: B ∩ V ≠ ; then the Chunk is considered visible, and the system only renders the visible Chunk.
[0048] For a zoom scenario, let the current zoom level be z.
[0049] The formula for calculating the LOD level is as follows: LOD = floor(log2(z + 1)) When z is small, aggregated rendering is adopted; when z is large, fine rendering is adopted.
[0050] Step S3: GPU Chunk block management, where the GPU Chunk block management adopts a GPU Buffer block management mechanism, splits massive data into several GPU Chunks, updates the changed data blocks, and avoids full reconstruction of the GPU Buffer. In addition, the system maintains a GPU Buffer pool to reuse released GPU Buffers, so as to reduce the video memory fragmentation problem.
[0051] Preferably, in step S3, each GPU Chunk independently maintains and processes: vertex data, index data, color data, and dynamic attribute data.
[0052] The GPU Chunk data structure is shown in the following table.
[0053] Table 1
[0054] Specifically, the GPU Chunk is dynamically updated in the following manner.
[0055] Let: the number of GPU Chunks be N; the current set of changed Chunks be U; the data size of a single Chunk be S.
[0056] The data transmission amount of the conventional solution is: T_old = N × S The data transmission amount of the present invention is: T_new = |U| × S Where: |U|<<N.
[0057] Therefore: T_new<<T_old, which significantly reduces the GPU data transmission overhead.
[0058] Step S4: GPU parallel computing processing, using WebGPU Compute Shader to implement GPU parallel computing, and outputting rendering calculation results.
[0059] Prior to step S4, the Compute Shader is used to perform: vertex coordinate calculation, dynamic particle position update, thermal value calculation, data aggregation, color mapping, and collision detection.
[0060] In traditional solutions, the aforementioned calculations are typically performed by the CPU. This invention migrates them to the GPU. This achieves: reduced CPU usage; increased parallel computing capabilities; and improved frame rate stability.
[0061] Preferably, during the execution of the big data visualization rendering method, dynamic LOD rendering adjustment is performed, dynamically switching the data precision according to the current scaling level. For example: displaying aggregation points at a long distance; displaying regional nodes at a medium distance; and displaying complete data at a close distance.
[0062] Furthermore, the dynamic LOD rendering adjustment is switched based on the following rules: current scaling factor, GPU utilization, current frame rate, browser memory status, and user operation frequency. This dynamically reduces unnecessary rendering overhead.
[0063] For example, when the system detects that the frame rate is below a threshold, GPU usage is too high, or browser memory is insufficient, it automatically reduces the number of particles, rendering precision, and refresh rate to ensure stable system operation.
[0064] Specifically, the dynamic frame rate degradation strategy can be: Let: the current frame rate be FPS; the minimum target frame rate be Fmin.
[0065] When: FPS <Fmin Automatic execution: Reduce particle count; reduce refresh rate; reduce LOD accuracy; delay low-priority tasks.
[0066] The dynamic adjustment formula is as follows: RenderScale = FPS / Fmin The lower the FPS, the smaller the RenderScale, thus automatically reducing the rendering pressure.
[0067] Prior to this, during the execution of the big data visualization rendering method, incremental rendering scheduling is performed, a rendering task priority queue is maintained, and the task execution frequency is dynamically adjusted according to the GPU load to avoid blocking the browser's main thread.
[0068] Furthermore, the incremental rendering scheduling rules dynamically calculate task priorities based on the following parameters: user interaction frequency (Fuser); data change frequency (Fdata); current viewport distance (Dview); and current GPU load (Gload).
[0069] The priority calculation formula is as follows: Priority = α×Fuser + β×Fdata + γ×(1 / Dview) - δ×Gload Where α, β, γ, and δ are weighting coefficients; the higher the Priority, the higher the priority for execution, that is, the task queue is sorted according to Priority.
[0070] According to a preferred embodiment, this application also discloses: a WebGPU-based incremental big data visualization rendering system, wherein the big data visualization rendering system performs visualization rendering using the aforementioned method; the big data visualization rendering system includes: a data incremental analysis module, a viewport detection module, a GPU Chunk segmentation management module, an LOD dynamic control module, a WebGPU rendering scheduling module, and a Compute Shader parallel computing module.
[0071] Example Case 1 Real-time traffic monitoring system for smart cities Scenario Description: The system needs to display the real-time trajectories of taxis throughout the city. Data Scale: Real-time vehicle count: 1.2 million; Data update rate per second: 80,000 records; Map zoom levels: 20.
[0072] Implementation process Step 1 The backend pushes vehicle trajectory data in real time. After receiving the data, the frontend uses a data hash structure to detect changes. Only newly added and moved vehicles are recorded.
[0073] Step 2 The system calculates vehicle data blocks for the currently visible area based on the current map viewport. Rendering stops in non-visible areas.
[0074] Step 3 The system uses a GPU Chunk management mechanism. It only updates the GPU buffer corresponding to the changed vehicle, avoiding the upload of all data.
[0075] Step 4 Performed via Compute Shader: trajectory smoothing; color heat mapping; dynamic aggregation.
[0076] Implementation effect Compared to the traditional WebGL full-scale rendering solution, the specifics are shown in Table 2.
[0077] Table 2
[0078] Example Case 2 For industrial IoT real-time monitoring systems Scenario Description: The system monitors the real-time status of factory equipment. There are over 3 million pieces of equipment. The system needs to display the following in real-time: temperature; pressure; energy consumption; and fault alarms.
[0079] The system employs a dynamic LOD (Level of Detail) strategy: displaying the aggregated status of areas at long distances; displaying device groups at medium distances; and displaying detailed device information at close distances.
[0080] Simultaneously, background data parsing is performed via Web Worker. Real-time thermal calculations are completed using WebGPU Compute Shader.
[0081] Implementation Results: The system maintains a stable frame rate of over 45 FPS even in scenarios with millions of devices. Compared to traditional Canvas rendering solutions, CPU utilization is reduced by approximately 65%.
[0082] Example Case 3 For real-time financial risk topology systems Scenario Description: The system is used to display real-time transaction relationships between financial institutions. Data Scale: Number of nodes: 5 million; Number of connections: 20 million.
[0083] The technical solution involves: GPU chunking; dynamic viewport; Compute Shader aggregation calculation; incremental update mechanism; and high-concurrency real-time topology rendering.
[0084] Technical effect: In complex relationship graph scenarios, the system can stably maintain real-time interaction, avoiding problems such as page freezing, GPU crashes, and browser unresponsiveness that occur in traditional topology solutions.
[0085] This application proposes a front-end incremental big data visualization rendering method based on WebGPU, which achieves this through: incremental updates of GPU Chunks; dynamic viewport scheduling; parallel computation of Compute Shaders; dynamic LOD control; GPU buffer pool reuse; and rendering task priority management. It effectively solves the problems existing in current technologies, such as low efficiency in real-time rendering of massive data; high GPU data transfer overhead; browser lag; and high memory consumption. This invention has strong innovation, engineering feasibility, and industrial application value.
[0086] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A WebGPU-based incremental big data visualization rendering method, characterized in that, The big data visualization rendering method includes: S1: Data incremental detection. It receives real-time data streams and compares the differences between newly received data and historical data. It records the unique identifier of each data item through a hash index structure. When a data change is detected, it generates only the corresponding data difference queue. S2: Dynamic viewport region detection, which obtains the current browser viewport information in real time, calculates the data blocks involved in the current viewport through the spatial index structure, and performs rendering on the currently visible area; S3: GPU Chunk Management. The GPU Chunk management adopts the GPU Buffer management mechanism to split massive amounts of data into several GPU Chunks and update the data blocks that have changed. S4: GPU parallel computing processing, using WebGPU Compute Shader to implement GPU parallel computing and output rendering calculation results.
2. The WebGPU-based incremental big data visualization rendering method as described in claim 1, characterized in that, In step S1, the difference comparison includes: newly added data, deleted data, modified data, data with changes in location, and data with changes in attributes.
3. The WebGPU-based incremental big data visualization rendering method as described in claim 2, characterized in that, The difference queue structure includes: fields and their corresponding descriptions; The fields include: id, type, offset, size, and priority; each field is described as: unique data identifier, data change type, GPU buffer offset, data length, and update priority.
4. The WebGPU-based incremental big data visualization rendering method as described in claim 1, characterized in that, The current browser viewport information in step S2 includes: zoom level, scroll position, current display area, camera position, and user interaction behavior.
5. The WebGPU-based incremental big data visualization rendering method as described in claim 1, characterized in that, In step S3, each GPU Chunk independently maintains and processes: vertex data, index data, color data, and dynamic attribute data.
6. The WebGPU-based incremental big data visualization rendering method as described in claim 1, characterized in that, In step S4, the Compute Shader performs: vertex coordinate calculation, dynamic particle position update, thermal value calculation, data aggregation, color mapping, and collision detection.
7. The WebGPU-based incremental big data visualization rendering method as described in claim 1, characterized in that, During the execution of the big data visualization rendering method, dynamic LOD rendering adjustment is performed, and the data precision is dynamically switched according to the current scaling level; The dynamic LOD rendering adjustment switches data precision based on the following rules: current scaling factor, GPU utilization, current frame rate, browser memory status, and user operation frequency.
8. The WebGPU-based incremental big data visualization rendering method as described in claim 7, characterized in that, During the execution of the big data visualization rendering method, incremental rendering scheduling is performed, and the task execution frequency is dynamically adjusted according to the GPU load.
9. The WebGPU-based incremental big data visualization rendering method as described in claim 8, characterized in that, The incremental rendering scheduling rules dynamically calculate task priority based on the following parameters: user interaction frequency (Fuser); data change frequency (Fdata); current viewport distance (Dview); and current GPU load (Gload). The priority calculation formula is as follows: Priority = α×Fuser + β×Fdata + γ×(1 / Dview) - δ×Gload Where α, β, γ, and δ are weighting coefficients; the higher the Priority, the higher the priority for execution, that is, the task queue is sorted according to Priority.
10. A WebGPU-based incremental big data visualization rendering system, characterized in that, The big data visualization rendering system uses the method described in any one of claims 1 to 9 for visualization rendering; The big data visualization rendering system includes: a data incremental analysis module, a viewport detection module, a GPU Chunk management module, a LOD dynamic control module, a WebGPU rendering scheduling module, and a Compute Shader parallel computing module.
