Interactive large-screen visualization system and method based on multidimensional data real-time rendering

Abstract

The invention discloses an interactive large-screen visualization system and method based on multi-dimensional data real-time rendering, relates to the technical field of image rendering, and aims to solve the bottleneck problems of a traditional system in the aspects of data throughput efficiency, rendering delay and interactive response. The method comprises the following steps: dividing space blocks by adopting a non-uniform octree, dynamically adjusting the size of the blocks according to data density, and combining a ZSTD + Hoffman hybrid compression algorithm; a CPU-GPU three-level asynchronous processing assembly line is constructed, and task priorities are dynamically adjusted through a fixation point weight algorithm; the master node dynamically allocates LOD parameters, and the slave nodes achieve special-shaped large-screen sub-pixel-level splicing based on a spherical coordinate system model; a Q-learning algorithm is adopted to predict a user operation sequence, and dual-channel bandwidth adaptive allocation is realized in combination with an ASTP protocol. The system comprises a dynamic partitioning module, a pipeline scheduling module, a multi-node cooperation module, an intelligent preloading module and an exception handling module. Real-time interaction visualization of a large amount of data can be realized, the rendering efficiency is improved, and hardware resource occupation is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image rendering, and particularly to an interactive large-screen visualization system and method based on real-time rendering of multi-dimensional data. Background Art

[0002] With the growing demand for real-time visualization in scenarios such as smart cities, traditional rendering engines face three major technical bottlenecks when dealing with multi-dimensional data: First, the data throughput efficiency is insufficient, and existing compression algorithms are difficult to balance the integrity and transmission efficiency of million-level data streams; second, the dynamic LOD switching mechanism is imperfect, and the low GPU-CPU cooperation efficiency leads to screen tearing and rendering delays; third, there is an operation synchronization delay in the interaction response, which seriously affects real-time decision-making.

[0003] Existing improvement solutions are mostly one-sided: either using inefficient compression algorithms to increase the computing load, or relying on pre-rendering techniques to sacrifice dynamic interactivity, and distributed rendering is also restricted by the node synchronization problem. These limitations highlight the systematic defects of existing rendering engines in data processing pipeline optimization, hardware resource cooperation, and real-time interaction architecture. Summary of the Invention

[0004] The present invention provides an interactive large-screen visualization system and method based on real-time rendering of multi-dimensional data to solve the data throughput bottleneck, rendering delay bottleneck, and interaction response bottleneck problems existing in traditional large-screen visualization systems in the prior art.

[0005] The present invention provides an interactive large-screen visualization method based on real-time rendering of multi-dimensional data, and the method includes:

[0006] Step S1: Perform dynamic block LOD encoding on the input multi-dimensional data stream, divide the space into blocks using a non-uniform octree, independently bind LOD parameters to each block, and combine a streaming compression engine to perform ZSTD+Huffman hybrid compression on the incremental data, and the compression ratio is dynamically adjusted according to the real-time network bandwidth;

[0007] Step S2: Process the compressed block data through a GPU-CPU asynchronous pipeline scheduling mechanism, and the pipeline includes three levels of asynchronous execution units: a data decoding unit CPU, an LOD pre-computation unit, and a rendering unit, and dynamically adjust the processing priorities of each block based on user interaction behaviors;

[0008] Step S3: Adopt a master-slave rendering architecture, the master node calculates the global LOD distribution and distributes it to the slave nodes, the slave nodes render local pictures according to the blocks, and combine the dynamic projection mapping table to splice the multi-node output pictures onto a special-shaped large screen, and eliminate visible splicing defects through sub-pixel level projection compensation technology;

[0009] Step S4: Based on reinforcement learning, predict the user operation sequence, dynamically preload the block data with high probability of access into the local cache, and achieve dual-channel bandwidth allocation for the control channel and the data channel through the Adaptive Streaming Transport Protocol (ASTP) to achieve near-real-time interaction response.

[0010] Furthermore, the present application also proposes that the specific implementation of the dynamic block LOD encoding in step S1 includes:

[0011] Construct a non-uniform octree according to the multi-dimensional data density, and divide the block size into d×d×d, where d = 2 n and n ∈ [3, 5]. When the data update frequency > 60Hz, n = 3 is adopted: the divided block size is 8×8×8, and when the update frequency < 30Hz, n = 4 is adopted: the divided block size is 16×16×16;

[0012] For the incremental data of each block, perform the first-round compression using the ZSTD algorithm, and the compression rate threshold T is calculated through the historical bandwidth mean:

[0013]

[0014] Further perform Huffman coding on the data after ZSTD compression, and the compression level is adjusted inversely according to the block LOD level, and the lowest compression rate β ∈ [0.15, 0.25] is adaptively adjusted.

[0015] Furthermore, the present application also proposes that the asynchronous pipeline scheduling mechanism in step S2 includes:

[0016] Allocate an independent task queue for each block, and the task priority P is calculated by the formula:

[0017]

[0018] where d is the viewing angle distance from the block center to the user's fixation point;

[0019] When a change in the user interaction behavior is detected, insert the high-priority block task into the head of the pipeline pending queue through the cut-in algorithm;

[0020] When the LOD pre-computation unit performs LOD pre-computation, adopt the tiling calculation strategy, and divide the calculation task into 16×16 thread blocks for parallel processing.

[0021] Furthermore, the present application also proposes that the method for generating the dynamic projection mapping table in step S3 includes:

[0022] Establish a spherical coordinate system model for the special-shaped large screen, and map the output picture of each rendering node to the spherical coordinates (u, v);

[0023] The bicubic interpolation algorithm is used to perform smooth transition on the pixels in the overlapping area of adjacent node images, and the interpolation weight W is determined by the formula:

[0024]

[0025] Generate a pixel offset correction matrix, and combine with the sub-pixel projection technology to perform alignment compensation on the spliced image to eliminate visible splicing defects.

[0026] Furthermore, the present application also proposes that the implementation of predicting the user operation sequence based on reinforcement learning and dynamically preloading the block data with high probability of access to the local cache in step S4 includes:

[0027] Construct a Markov state transition model of the user operation sequence, and the state space S = {current viewpoint coordinates, operation type, time interval};

[0028] Use the Q-learning algorithm to update the preloading policy, and the reward function R is defined as: R = 0.7×cache hit rate + 0.3×(1 - cache space occupancy rate) - [network packet loss rate > 5%? 0.2:0.1]×network bandwidth consumption,

[0029] When the confidence level of predicting the future k-step operation > 85%, trigger the preloading request for the corresponding block.

[0030] Furthermore, the present application also proposes that the implementation of the adaptive streaming transport protocol ASTP includes:

[0031] The control channel uses UDP broadcast to transmit LOD metadata, and the data channel uses multiple TCP connections to transmit block content;

[0032] Dynamically allocate bandwidth B for each TCP connection i , and the allocation formula is:

[0033]

[0034] where P j is the priority of all active connections;

[0035] When it is detected that the data packet loss rate > 5%, automatically switch to the FEC forward error correction mode and reduce the bandwidth allocation weight of non-critical data streams.

[0036] Furthermore, the present application also proposes that the method further includes an exception recovery mechanism:

[0037] Real-time monitor the GPU video memory occupancy rate of the rendering node. When the occupancy rate > 90%, automatically reduce the LOD level of low-priority blocks;

[0038] If the network latency between nodes is > 50ms and ≤ 100ms, enable the predictive rendering mode; if the latency > 100ms, switch to the key frame I-Frame only transmission mode.

[0039] Furthermore, the present application also proposes an interactive large-screen visualization system based on real-time rendering of multi-dimensional data for performing the above method. The system includes:

[0040] Dynamic chunking module: Configure a non-uniform octree encoder and a streaming compression engine to divide the spatial blocks according to data density and perform hybrid compression;

[0041] Pipeline scheduling module: Integrate a CPU decoding unit, a GPU computing unit, and a rendering unit to implement a three-level asynchronous pipeline and a dynamic priority queueing algorithm;

[0042] Multi-node collaboration module: Include a master node LOD allocator and a slave node renderer, and drive the splicing of a special-shaped large screen through a dynamic projection mapping table;

[0043] Intelligent preloading module: Deploy a reinforcement learning prediction model and an ASTP protocol stack to achieve cache management and bandwidth adaptive allocation;

[0044] Exception handling module: Monitor the GPU video memory and network status in real time, and trigger a degraded rendering or error correction transmission mechanism.

[0045] An interactive large-screen visualization method and system based on real-time rendering of multi-dimensional data provided by the present application realizes efficient data processing, rendering, and interaction through dynamic chunking LOD encoding, GPU-CPU asynchronous pipeline scheduling, a master-slave rendering architecture, and a preloading strategy based on reinforcement learning. This solution effectively solves the problems of real-time processing and visualization of large-scale multi-dimensional data, and has the advantages of improving data throughput efficiency, optimizing rendering performance, and improving interaction response speed. Brief Description of the Drawings

[0046] Figure 1 It is a flowchart of the interactive large-screen visualization method based on real-time rendering of multi-dimensional data of the present invention;

[0047] Figure 2 It is an architecture diagram of the interactive large-screen visualization system based on real-time rendering of multi-dimensional data of the present invention. Detailed Embodiments

[0048] The present invention relates to, aiming to solve the data throughput bottleneck, rendering latency bottleneck, and interaction response bottleneck existing in traditional large-screen visualization systems. Traditional systems have obvious deficiencies in aspects such as real-time parsing and compression of multi-dimensional data, dynamic level of detail switching, and synchronization of user operations and rendering pipelines, and cannot meet the requirements of real-time interaction of large-scale data in scenarios such as smart cities and industrial Internet of Things.

[0049] The above technical solution will be described in detail below in combination with the accompanying drawings of the specification and specific implementation manners to better understand the above technical solution. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments of the present invention. It should be understood that the present invention is not limited to the example embodiments only for explaining the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts belong to the scope of protection of the present invention. In addition, it should be noted that, for the sake of convenience of description, only the parts related to the present invention are shown in the drawings rather than all of them.

[0050] Embodiment 1

[0051] As Figure 1 shown in the flowchart of the interactive large-screen visualization method based on real-time rendering of multi-dimensional data: The present application realizes the full-link optimization from data stream processing to rendering interaction through dynamic block LOD encoding, GPU-CPU asynchronous pipeline scheduling, master-slave rendering architecture, and reinforcement learning to predict the user operation sequence. Specifically, the present application uses a non-uniform octree to divide the spatial blocks, and combines a streaming compression engine to perform ZSTD + Huffman hybrid compression on the incremental data, and the compression ratio is dynamically adjusted according to the real-time network bandwidth. The compressed block data is processed through the GPU-CPU asynchronous pipeline scheduling mechanism. The pipeline includes a data decoding unit, an LOD pre-computation unit, and a rendering unit, and the processing priorities of each block are dynamically adjusted based on the user interaction behavior. The master-slave rendering architecture is adopted. The master node calculates the global LOD distribution and allocates it to the slave nodes. The slave nodes render the local pictures according to the blocks, and combine the dynamic projection mapping table to splice the output pictures of multiple nodes to the special-shaped large screen to ensure that the pixel alignment error ≤ 0.1px. Based on reinforcement learning to predict the user operation sequence, the block data with high probability of access is dynamically pre-loaded into the local cache, and the dual-channel bandwidth allocation of the control channel and the data channel is realized through the adaptive streaming transmission protocol, and the end-to-end delay ≤ 20ms.

[0052] When dealing with multi-dimensional data, traditional large-screen visualization systems often face the problem of insufficient data throughput. To solve this problem, this application proposes dynamic block LOD coding and hybrid compression technology. The spatial blocks are divided by a non-uniform octree, and the incremental data is subjected to ZSTD+Huffman hybrid compression in combination with a streaming compression engine. The compression ratio is dynamically adjusted according to the real-time network bandwidth. This not only improves the data parsing and compression efficiency but also ensures that the data throughput meets the real-time requirements. In terms of data processing, the GPU-CPU cooperation efficiency of traditional methods is relatively low, resulting in a large rendering delay. Therefore, this application adopts a GPU-CPU asynchronous pipeline scheduling mechanism. Through three-level asynchronous execution units, namely the data decoding unit, the LOD pre-computation unit, and the rendering unit, the data processing efficiency is improved and the rendering delay is reduced. In terms of user interaction, it is difficult for traditional systems to achieve millisecond-level synchronization. This application predicts the user operation sequence through reinforcement learning, dynamically preloads the block data with a high probability of access into the local cache, and realizes the dual-channel bandwidth allocation of the control channel and the data channel through an adaptive streaming transmission protocol, thus achieving fast response of user interaction and efficient data transmission, and ensuring that the end-to-end delay ≤ 20ms.

[0053] Dynamic block LOD coding is one of the key technologies of this application. The spatial blocks are divided by a non-uniform octree, each block is independently bound with LOD parameters, and the incremental data is subjected to ZSTD+Huffman hybrid compression in combination with a streaming compression engine. The compression ratio is dynamically adjusted according to the real-time network bandwidth. The GPU-CPU asynchronous pipeline scheduling mechanism includes three-level asynchronous execution units: the data decoding unit, the LOD pre-computation unit, and the rendering unit. The data decoding unit is responsible for decoding the compressed block data. The LOD pre-computation unit dynamically adjusts the processing priority of each block based on user interaction behavior. The rendering unit is responsible for the final screen rendering. The master-slave rendering architecture calculates the global LOD distribution by the master node and distributes it to the slave nodes. The slave nodes render the local screen according to the blocks, and combine the dynamic projection mapping table to splice the multi-node output screens onto the special-shaped large screen, ensuring that the pixel alignment error ≤ 0.1px. Predicting the user operation sequence through reinforcement learning constructs a Markov state transition model of the user operation sequence, uses the Q-learning algorithm to update the preloading strategy, dynamically preloads the block data with a high probability of access into the local cache, and realizes the dual-channel bandwidth allocation of the control channel and the data channel through an adaptive streaming transmission protocol, ensuring that the end-to-end delay ≤ 20ms.

[0054] Compared with the prior art, the present application has obvious advantages in terms of data throughput, rendering latency, and interaction response. Through dynamic block LOD encoding and hybrid compression technology, the parsing and compression efficiency of multi-dimensional data is improved, ensuring that the data throughput meets real-time requirements. The GPU-CPU asynchronous pipeline scheduling mechanism improves the data processing efficiency and reduces the rendering latency. The master-slave rendering architecture and the dynamic projection mapping table achieve multi-node collaborative rendering, ensuring the accuracy of the picture stitching. The reinforcement learning predicts the user operation sequence and the adaptive streaming protocol realizes the fast response of user interaction and the efficient data transmission, ensuring that the end-to-end latency ≤ 20ms.

[0055] Through dynamic block LOD encoding and hybrid compression technology, the problem of efficient parsing and compression of multi-dimensional data is solved, ensuring that the data throughput meets real-time requirements. The GPU-CPU asynchronous pipeline scheduling mechanism improves the data processing efficiency and reduces the rendering latency through three-level asynchronous execution units: the data decoding unit, the LOD pre-computation unit, and the rendering unit. The master-slave rendering architecture calculates the global LOD distribution by the master node and distributes it to the slave nodes. The slave nodes render the local pictures in blocks and combine with the dynamic projection mapping table to stitch the output pictures of multiple nodes onto the special-shaped large screen, ensuring that the pixel alignment error ≤ 0.1px. The reinforcement learning predicts the user operation sequence by constructing a Markov state transition model of the user operation sequence, using the Q-learning algorithm to update the preloading strategy, dynamically preloading the block data with high probability of access to the local cache, and realizing the dual-channel bandwidth allocation of the control channel and the data channel through the adaptive streaming protocol, ensuring that the end-to-end latency ≤ 20ms. Through the mutual cooperation of the above steps, the problems of data throughput, rendering latency, and interaction response in the real-time rendering of multi-dimensional data and the interactive large-screen visualization process are solved, and an efficient and real-time visualization effect is achieved.

[0056] Furthermore, the present application also proposes to construct a non-uniform octree according to the multi-dimensional data density, and divide the block size into d×d×d, where d = 2 n and n ∈ [3, 5]. When the data update frequency > 60Hz, n = 3 (8×8×8) is adopted, and when the update frequency < 30Hz, n = 4 (16×16×16) is adopted; for the incremental data of each block, the ZSTD algorithm is used for the first-round compression, and the compression ratio threshold T is calculated by the historical bandwidth mean:

[0057] T = 0.6×(current bandwidth / maximum bandwidth) + 0.2 + 0.05×bandwidth change rate;

[0058] The data after ZSTD compression is further Huffman encoded, and the compression level is adjusted inversely according to the block LOD level, and the lowest compression ratio β ∈ [0.15, 0.25] is adaptively adjusted.

[0059] Construct a non-uniform octree according to the multi-dimensional data density, with the divided block size being d×d×d, where d = 2 n And n ∈ [3, 5] is dynamically adjusted. This technical feature lies in flexibly adjusting the block size according to different data densities, thereby improving the encoding efficiency. For the incremental data of each block, the ZSTD algorithm is used for the first-round compression. The compression ratio threshold T is calculated through the historical bandwidth mean. This technical feature adapts to different bandwidth conditions by dynamically adjusting the compression ratio, thereby improving the data transmission efficiency. The data after ZSTD compression is further subjected to Huffman coding, and the compression level is adjusted inversely according to the LOD level of the block. This technical feature further compresses the data and adjusts the compression level according to the LOD level to ensure the data transmission efficiency at different levels of detail. Through the comprehensive application of these technical means, this application proposes a coding method that dynamically adjusts according to data density and bandwidth conditions, thus effectively solving the coding efficiency problem caused by uneven multi-dimensional data density.

[0060] The construction of the non-uniform octree is carried out according to the density of multi-dimensional data. Specifically, the density of each block can be determined by analyzing the distribution of data points, and then the size of the block can be divided according to the density. In practical applications, an adaptive algorithm can be adopted to dynamically adjust the size of the block according to the data update frequency to meet different data update requirements. The ZSTD algorithm is an efficient lossless compression algorithm suitable for the compression processing of large-scale data. By setting the compression ratio threshold T, the compression ratio can be dynamically adjusted according to the current bandwidth situation to ensure the high efficiency of data transmission. Huffman coding is a classic entropy coding method. By further compressing the data after ZSTD compression, the data volume can be further reduced and the transmission efficiency can be improved. The inverse adjustment of the compression level is carried out according to the LOD level of the block. Specifically, the compression level of Huffman coding can be dynamically adjusted according to the LOD level of each block to ensure the data transmission efficiency at different levels of detail.

[0061] This application realizes a coding method that dynamically adjusts according to data density and bandwidth conditions by introducing technical means such as non-uniform octrees, ZSTD algorithms, and Huffman coding, effectively solving the coding efficiency problem caused by uneven multi-dimensional data density. Compared with the prior art, the technical solution of this application has significant advantages in terms of data compression and transmission efficiency, and can significantly improve the data transmission efficiency while ensuring the data quality, meeting the requirements of different bandwidth conditions and data densities.

[0062] Furthermore, this application also proposes to allocate an independent task queue for each block, and the task priority P is calculated by the formula:

[0063]

[0064] where d is the viewing distance from the center of the block to the user's fixation point;

[0065] When a change in the user interaction behavior is detected, the high-priority block tasks are inserted into the head of the pipeline pending queue through the cut-in algorithm.

[0066] When the GPUComputeShader performs LOD pre-computation, it adopts a tiling calculation strategy, dividing the calculation tasks into 16×16 thread blocks for parallel processing.

[0067] The asynchronous pipeline scheduling mechanism of the present application assigns an independent task queue to each block and uses a specific priority calculation formula to dynamically adjust the task priority according to the user's fixation point weight, current transmission delay, and data volume. When a change in the user interaction behavior is detected, the cut-in algorithm can insert the high-priority block tasks into the head of the pending queue to ensure that important tasks are processed first. In addition, the GPUComputeShader adopts a tiling calculation strategy, dividing the calculation tasks into 16×16 thread blocks for parallel processing, which improves the efficiency of LOD pre-computation. These technical means together solve the problems of task priority scheduling and rendering efficiency.

[0068] In the present application, assigning an independent task queue to each block is a method to ensure task independence and efficient management. The calculation formula for task priority

[0069] can adjust the task priority according to the real-time situation to ensure that critical tasks can be processed in time. The application of the cut-in algorithm enables high-priority tasks to be quickly inserted into the head of the pending queue when the user interaction behavior changes, thus improving the system response speed. When the GPUComputeShader performs LOD pre-computation, it adopts a tiling calculation strategy, dividing the calculation tasks into 16×16 thread blocks for parallel processing. This strategy makes full use of the parallel computing power of the GPU and improves the calculation efficiency.

[0070] Thus, the present application effectively solves the problems of task priority scheduling and rendering efficiency in the data processing process through the asynchronous pipeline scheduling mechanism, using independent task queues, a priority calculation formula, a cut-in algorithm, and a tiling calculation strategy. Compared with the prior art, the present application can dynamically adjust the task priority to ensure that important tasks are processed first, and improves the calculation efficiency through the tiling calculation strategy, thereby achieving more efficient rendering and faster system response.

[0071] Furthermore, this application also proposes that the method for generating the dynamic projection mapping table in step S3 includes establishing a spherical coordinate system model for the special-shaped large screen and mapping the output images of each rendering node to the spherical coordinates (u, v). In this way, the images of different nodes can be mapped into a unified coordinate system, facilitating subsequent splicing and alignment operations. The bicubic interpolation algorithm is used to perform smooth transitions on the pixels in the overlapping areas of adjacent node images, and the interpolation weight W is determined by the formula:

[0072]

[0073] This interpolation method can effectively reduce the edge effects in the overlapping areas of different node images and ensure smooth transitions of the images. A pixel offset correction matrix is generated, and combined with the sub-pixel projection technology of the NVIDIA Warp API, sub-pixel-level alignment compensation is performed on the spliced image to ensure that the pixel alignment error of the final spliced image ≤ 0.1px.

[0074] When establishing the spherical coordinate system model for the special-shaped large screen, it is first necessary to accurately measure the physical size and shape of the special-shaped large screen, and construct a spherical coordinate system model based on these measurement data. The output images of each rendering node will be mapped according to the (u, v) coordinates of the spherical coordinate system, so as to ensure that the images can be processed in a unified coordinate system. The bicubic interpolation algorithm is used to perform smooth transitions on the pixels in the overlapping areas of adjacent node images, and the interpolation weight W is determined by calculating the ratio of the edge distance to the width of the overlapping area. This interpolation method can effectively reduce the edge effects in the overlapping areas of the images and ensure smooth transitions of the images. When generating the pixel offset correction matrix, the grid deformation function of the NVIDIA Warp API is used to perform sub-pixel-level adjustment on the spliced image to correct the pixel offset problem caused by splicing.

[0075] As a preferred implementation, high-precision cameras and laser rangefinders can be used to measure the physical size and shape of the special-shaped large screen to ensure the accuracy of the spherical coordinate system model. At the same time, the GPU can be used to accelerate the calculation of the bicubic interpolation algorithm to improve the calculation efficiency. When generating the pixel offset correction matrix, image processing algorithms can be used to perform detailed pixel-level adjustment on the spliced image to further improve the alignment accuracy of the image.

[0076] In this application, a spherical coordinate system model of a special-shaped large screen is established, and the output image of each rendering node is mapped to spherical coordinates (u, v). The bicubic interpolation algorithm is used to smoothly transition the pixels in the overlapping area of adjacent node images, and a pixel offset correction matrix is generated. Combined with the sub-pixel projection technology of NVIDIA Warp API, the spliced image is aligned and compensated. Thus, it can effectively solve the technical problems of pixel alignment error and smooth transition of the overlapping area of the special-shaped large screen during multi-node splicing, ensuring the accuracy and consistency of the final spliced image. Compared with the prior art, this application provides a more accurate and efficient method for splicing special-shaped large screens, significantly improving the display effect of the large screen visualization system.

[0077] Furthermore, this application also proposes to construct a Markov state transition model of the user operation sequence, and the state space S = {current viewpoint coordinates, operation type, time interval}. The Q-learning algorithm is used to update the preloading policy, and the reward function R is defined as:

[0078] R = 0.7 × cache hit rate + 0.3 × (1 - cache space occupancy rate) - [network packet loss rate > 5%? 0.2 : 0.1] × network bandwidth consumption.

[0079] When the confidence level of predicting the future k-step operation > 85%, trigger the preloading request for the corresponding block.

[0080] By constructing a Markov state transition model of the user operation sequence, the rules and patterns of user operations can be effectively captured. During the continuous update of the preloading policy by the Q-learning algorithm, through the definition of the reward function R, the cache hit rate, cache space occupancy rate, and network bandwidth consumption are optimized. When the confidence level of predicting future operations is higher than 85%, the system will preload the block data with a high probability of access in advance, thereby improving the accuracy and efficiency of data preloading.

[0081] Furthermore, the Markov state transition model of the user operation sequence can be constructed by collecting the operation data of the user in the large screen visualization system. Specifically, the state space S includes the current viewpoint coordinates, operation type, and time interval, where the current viewpoint coordinates represent the area currently concerned by the user, the operation type includes operations such as zooming and panning, and the time interval is used to record the time point when the operation occurs. The Q-learning algorithm updates the preloading policy by continuously interacting with the environment, enabling the system to make optimal preloading decisions when facing different user operations. The definition of the reward function R comprehensively considers the cache hit rate, cache space occupancy rate, and network bandwidth consumption. When the network packet loss rate > 5%, the bandwidth consumption penalty coefficient is increased to 0.2 to achieve the optimization of the overall performance.

[0082] As a preferred implementation, when the confidence level of the system's prediction of the user's future k-step operations exceeds 85%, a preloading request for the corresponding block will be triggered. Specifically, the system can predict the highly probable blocks that may be accessed in the future based on the user's current operation mode, and preload the data of these blocks into the local cache in advance. Thus, when the user actually operates, the required data can be quickly accessed, reducing the waiting time and improving the system's response speed and user experience.

[0083] By introducing the Markov state transition model and Q-learning algorithm, this application can achieve remarkable results in user operation sequence prediction and data preloading. Compared with traditional methods, this application can predict user operations more accurately, improve the accuracy and efficiency of data preloading, reduce cache space occupation and network bandwidth consumption, thus realizing a more efficient large-screen visualization system.

[0084] Furthermore, this application also proposes that the control channel uses UDP broadcast to transmit LOD metadata, and the data channel uses multiple TCP connections to transmit block content. Dynamically allocate bandwidth B for each TCP connection i , and the allocation formula is:

[0085]

[0086] When the detected data packet loss rate > 5%, automatically switch to the FEC forward error correction mode and reduce the bandwidth allocation weight of non-critical data streams.

[0087] This application adopts the method of separating the control channel and the data channel, and uses UDP and TCP protocols respectively for data transmission. The control channel transmits LOD metadata through UDP broadcast, ensuring the rapid transmission of control information, while the data channel transmits block content through multiple TCP connections, ensuring the reliable transmission of data content. In terms of bandwidth allocation, by dynamically allocating bandwidth B i , the utilization of network resources is optimized. Specifically, the bandwidth allocation formula is:

[0088]

[0089] where P j is the priority of all active connections. This bandwidth allocation mechanism can dynamically adjust the allocation of bandwidth resources according to the priority of each connection. When the detected data packet loss rate exceeds 5%, the system automatically switches to the FEC forward error correction mode, and preferentially reduces the bandwidth weight of non-critical data streams, and improves the reliability of data transmission by enabling the forward error correction mechanism.

[0090] Furthermore, the control channel uses UDP broadcast to transmit LOD metadata, which can deliver control information to all relevant nodes in a short time and is suitable for scenarios where rapid transmission of control information is required. The data channel uses multiple TCP connections to transmit block content. By using multiple TCP connections, the parallelism and reliability of data transmission can be improved. The dynamic bandwidth allocation mechanism ensures the reasonable allocation and efficient utilization of bandwidth resources by monitoring the priorities and bandwidth usage of each TCP connection in real time. When the packet loss rate exceeds 5%, it automatically switches to the FEC forward error correction mode and preferentially guarantees critical data streams through weight adjustment. This mechanism can still ensure the accuracy and integrity of data transmission in a network environment with a high packet loss rate.

[0091] Thus, by separating the control channel and the data channel and using UDP and TCP protocols for data transmission respectively, this application ensures the rapid transmission of control information and the reliable transmission of data content. By dynamically allocating bandwidth, it optimizes the utilization of network resources and improves the efficiency of data transmission. When the packet loss rate exceeds 5%, it automatically switches to the FEC forward error correction mode, improving the reliability of data transmission. Compared with the prior art, this application has significant advantages in terms of network resource utilization and data transmission reliability.

[0092] Furthermore, this application also proposes that the method further includes an exception recovery mechanism:

[0093] Real-time monitor the GPU video memory occupancy rate of the rendering node. When the occupancy rate > 90%, automatically reduce the LOD level of low-priority blocks;

[0094] If the network latency between nodes > 50ms and ≤ 100ms, enable the predictive rendering mode; if the latency > 100ms, switch to the key frame (I-Frame) only transmission mode.

[0095] The technical solution includes two parts: real-time monitoring and local degraded rendering. Real-time monitor the GPU video memory occupancy rate of the rendering node. When the video memory occupancy rate exceeds 90%, reduce the use of video memory by lowering the LOD level of low-priority blocks to ensure the stability of the system. When the network latency between nodes is detected to exceed 50ms, it is processed in two levels: if the latency is between 50ms and 100ms, enable the predictive rendering mode; if the latency > 100ms, switch to the key frame only transmission mode and use historical cache data to generate approximate images to ensure the continuity of the images. Through the above technical means, the problem of ensuring the stability and continuity of the large-screen visualization system when an exception occurs in the rendering node is solved.

[0096] In the exception recovery mechanism, real-time monitoring of the GPU video memory occupancy rate of the rendering node is a crucial step. When the video memory occupancy rate exceeds 90%, the system will automatically reduce the LOD level of the low-priority blocks. The LOD (Level of Detail) level refers to reducing the detail level of the model by decreasing the number of polygons in 3D computer graphics. By reducing the LOD level of the low-priority blocks, the use of video memory can be reduced, thus avoiding system instability problems caused by excessive video memory occupancy.

[0097] When the network latency between nodes exceeds 50 ms, the system will activate the local downgrade rendering mode. At this time, the system will use historical cache data to generate approximate images. Historical cache data refers to the previously rendered image data, which is stored for later use. By using this cached data, the system can still generate relatively smooth images under high network latency, thus ensuring the continuity of the user experience. If the latency > 100 ms, switch to the key frame (I-Frame) only transmission mode.

[0098] Furthermore, the exception recovery mechanism of this application effectively solves the system stability and continuity problems when the rendering node encounters an exception through real-time monitoring and local downgrade rendering. Compared with the prior art, the technical solution of this application can improve the system stability and user experience without increasing the hardware cost by automatically adjusting the LOD level and using historical cache data.

[0099] Embodiment 2

[0100] As Figure 2 shown in the architecture diagram of the interactive large-screen visualization system based on real-time rendering of multi-dimensional data, it includes:

[0101] Dynamic chunking module: Configure a non-uniform octree encoder and a streaming compression engine, divide the spatial blocks according to data density, and perform hybrid compression;

[0102] Pipeline scheduling module: Integrate a CPU decoding unit, a GPU computing unit, and a rendering unit to implement a three-level asynchronous pipeline and a dynamic priority cut-in algorithm;

[0103] Multi-node cooperation module: Include a master node LOD allocator and a slave node renderer, and drive the special-shaped large-screen splicing through a dynamic projection mapping table;

[0104] Intelligent preloading module: Deploy a reinforcement learning prediction model and an ASTP protocol stack to achieve cache management and bandwidth adaptive allocation;

[0105] Exception handling module: Real-time monitor the GPU video memory and network status, and trigger the downgrade rendering or error correction transmission mechanism.

[0106] The system includes five main modules, each of which plays a different role in solving technical problems. The dynamic chunking module divides the spatial blocks according to data density and performs hybrid compression through a non-uniform octree encoder and a streaming compression engine, solving the problems of insufficient real-time parsing and compression efficiency of multi-dimensional data. The pipeline scheduling module integrates a CPU decoding unit, a GPU computing unit, and a rendering unit, and solves the problems of rendering latency and low GPU-CPU cooperation efficiency through a three-level asynchronous pipeline and a dynamic priority queueing algorithm. The multi-node cooperation module ensures high-precision and low-error large-screen splicing through a master node LOD allocator and a slave node renderer, and uses a dynamic projection mapping table to drive the splicing of special-shaped large screens. The intelligent preloading module deploys a reinforcement learning prediction model and an ASTP protocol stack to achieve cache management and adaptive bandwidth allocation, solving the real-time problem of user interaction response. The exception handling module monitors the GPU video memory and network status in real time, triggering a degraded rendering or error-correcting transmission mechanism to ensure the stable operation of the system under abnormal conditions. Through the above technical means, the system can effectively achieve full-link optimization from data stream processing to rendering interaction, ensuring that the end-to-end latency ≤ 20ms, and solving the bottlenecks of data throughput, rendering latency, and interaction response existing in traditional large-screen visualization systems.

[0107] The non-uniform octree encoder of the dynamic chunking module can automatically adjust the block size according to data density, and the streaming compression engine dynamically adjusts the compression ratio according to the real-time network bandwidth. The pipeline scheduling module effectively reduces the rendering latency and improves the GPU-CPU cooperation efficiency through a three-level asynchronous pipeline and a dynamic priority queueing algorithm. The multi-node cooperation module uses a dynamic projection mapping table to achieve high-precision splicing of special-shaped large screens. The reinforcement learning prediction model deployed by the intelligent preloading module can accurately predict user operations and preload data in advance, and the ASTP protocol stack ensures the adaptive allocation of bandwidth. The exception handling module can timely trigger a degraded rendering or error-correcting transmission mechanism by monitoring the GPU video memory and network status in real time, ensuring the stable operation of the system under abnormal conditions.

[0108] Furthermore, the dynamic chunking module can adopt various non-uniform octree encoding algorithms, such as an adaptive adjustment algorithm based on data density or a prediction algorithm based on historical data. The three-stage asynchronous pipeline of the pipeline scheduling module can adopt different scheduling strategies, such as a dynamic adjustment strategy based on task priority or a prediction scheduling strategy based on user behavior. The dynamic projection mapping table of the multi-node collaboration module can adopt various generation methods, such as a mapping method based on spherical coordinates or a mapping method based on planar coordinates. The reinforcement learning prediction model of the intelligent preloading module can adopt different reinforcement learning algorithms, such as the Q-learning algorithm or the deep reinforcement learning algorithm. The degradation rendering mechanism of the exception handling module can adopt various degradation strategies, such as a degradation strategy based on historical data or a degradation strategy based on real-time monitoring.

[0109] The interactive large-screen visualization system based on real-time rendering of multi-dimensional data proposed in this application realizes the full-link optimization from data stream processing to rendering interaction through the collaborative work of five main modules. Compared with the prior art, the system of this application has significant advantages in data throughput, rendering latency, and interaction response, and can effectively solve the technical bottlenecks existing in traditional large-screen visualization systems.

[0110] It should be understood that the disclosed embodiments of the present invention and the above descriptions enable those skilled in the art to implement the present invention using the present invention. At the same time, the present invention is not limited to the part of the embodiments mentioned above. It should be understood that those of ordinary skill in the art can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An interactive large-screen visualization method based on real-time rendering of multi-dimensional data, characterized in that: The method comprises the following steps: Step S1: Dynamically block LOD encoding is performed on the input multi-dimensional data stream, and non-uniform octree is used to divide the space blocks. Each block is independently bound to LOD parameters, and the incremental data is compressed by ZSTD+Huffman hybrid compression in combination with the streaming compression engine. The compression rate is dynamically adjusted according to the real-time network bandwidth; Step S2: Processing the compressed block data through the GPU-CPU asynchronous pipeline scheduling mechanism, the pipeline includes three levels of asynchronous execution units: data decoding unit CPU, LOD pre-calculation unit, rendering unit, and dynamically adjusting the processing priority of each block based on user interaction behavior; Step S3: Using a master-slave rendering architecture, the master node calculates the global LOD distribution and distributes it to the slave nodes. The slave nodes render the local images according to the blocks, and combine the dynamic projection mapping table to splice the multi-node output images to the special-shaped large screen, and eliminate visible splicing defects through sub-pixel projection compensation technology; Step S4: Predict user operation sequences based on reinforcement learning, dynamically preload block data with high probability of access to the local cache, and implement dual-channel bandwidth allocation of the control channel and the data channel through the adaptive streaming protocol ASTP to achieve near real-time interactive response.

2. The interactive large-screen visualization method based on real-time rendering of multi-dimensional data according to claim 1, characterized in that: The specific implementation of the dynamic block LOD encoding in step S1 includes: Construct a non-uniform octree based on the multi-dimensional data density, and divide the blocks into d×d×d, where d=2 n And n∈[3,5], when the data update frequency is greater than 60Hz, n=3 is used: the block size is 8×8×8, and when the update frequency is less than 30Hz, n=4 is used: the block size is 16×16×16; For the incremental data of each block, the ZSTD algorithm is used for the first round of compression, and the compression rate threshold T is calculated by the historical bandwidth average: The ZSTD compressed data is further Huffman encoded, and the compression level is inversely adjusted according to the block LOD level, and the minimum compression rate β∈[0.15,0.25] is adaptively adjusted.

3. The interactive large-screen visualization method based on real-time rendering of multi-dimensional data according to claim 2, characterized in that: The asynchronous pipeline scheduling mechanism in step S2 includes: Assign an independent task queue to each block, and the task priority P is calculated by the formula: The focus point d is the visual distance from the center of the block to the user's gaze point; When a change in user interaction behavior is detected, the high-priority block task is inserted into the head of the pipeline queue through the queue insertion algorithm; When the LOD pre-computation unit performs LOD pre-computation, a tiled computing strategy is adopted to divide the computing task into 16×16 thread blocks for parallel processing.

4. The interactive large-screen visualization method based on real-time rendering of multi-dimensional data according to claim 3, characterized in that: The method for generating the dynamic projection mapping table in step S3 includes: Establish a spherical coordinate system model for special-shaped large screens and map the output image of each rendering node to spherical coordinates (u, v); The bicubic interpolation algorithm is used to smoothly transition the pixels in the overlapping area of ​​adjacent node images. The interpolation weight W is determined by the formula: Generate a pixel offset correction matrix and combine it with sub-pixel projection technology to align and compensate the stitched images to eliminate visible stitching defects.

5. The interactive large-screen visualization method based on real-time rendering of multi-dimensional data according to claim 4, characterized in that: The implementation of predicting the user operation sequence based on reinforcement learning and dynamically preloading the block data with high probability of access to the local cache in step S4 includes: Construct a Markov state transition model of the user operation sequence, where the state space S = {current viewpoint coordinates, operation type, time interval}; The Q-learning algorithm is used to update the preloading strategy. The reward function R is defined as: R = 0.7 × cache hit rate + 0.3 × (1-cache space occupancy) - [network packet loss rate > 5%? 0.2:0.1] × network bandwidth consumption. When the confidence of the predicted future k-step operation is > 85%, the preloading request of the corresponding block is triggered.

6. The interactive large-screen visualization method based on real-time rendering of multi-dimensional data according to claim 5, characterized in that: The implementation of the adaptive streaming protocol ASTP includes: The control channel uses UDP broadcast to transmit LOD metadata, and the data channel uses multiple TCP connections to transmit block content; Dynamically allocate bandwidth B for each TCP connection i , the allocation formula is: Where P j The priority of all active connections; When the data packet loss rate is detected to be >5%, it automatically switches to FEC forward error correction mode and reduces the bandwidth allocation weight of non-critical data streams.

7. The interactive large-screen visualization method based on real-time rendering of multi-dimensional data according to claim 6, characterized in that: The method also includes an abnormal recovery mechanism: Monitor the GPU memory usage of the rendering node in real time. When the usage is > 90%, automatically reduce the LOD level of low-priority blocks. If the network delay between nodes is >50ms and ≤100ms, enable the predictive rendering mode; if the delay is >100ms, switch to the key frame I-Frame transmission only mode.

8. An interactive large-screen visualization system based on real-time rendering of multi-dimensional data, used to implement the interactive large-screen visualization method based on real-time rendering of multi-dimensional data as described in claim 7, characterized in that: The system comprises: Dynamic Blocking Module: Configures a non-uniform octree encoder and a streaming compression engine to divide spatial blocks according to data density and perform hybrid compression; Pipeline scheduling module: integrates CPU decoding unit, GPU computing unit and rendering unit to realize three-level asynchronous pipeline and dynamic priority queue-jumping algorithm; Multi-node collaboration module: includes the master node LOD distributor and the slave node renderer, which drives the splicing of special-shaped large screens through dynamic projection mapping tables; Intelligent preloading module: deploys reinforcement learning prediction model and ASTP protocol stack to achieve cache management and adaptive bandwidth allocation; Exception handling module: real-time monitoring of GPU memory and network status, triggering downgraded rendering or error correction transmission mechanism.

Images