Interactive large-screen visualization system and method based on real-time rendering of multidimensional data

Abstract

The present invention discloses an interactive large-screen visualization system and method based on real-time rendering of multi-dimensional data, relates to the field of image rendering technology, and aims to solve the bottleneck problems of traditional systems in data throughput efficiency, rendering delay and interactive response. The method includes: using a non-uniform octree to divide the space blocks, dynamically adjusting the block size according to the data density, and combining the ZSTD+Huffman hybrid compression algorithm; constructing a CPU-GPU three-level asynchronous processing pipeline, and dynamically adjusting the task priority through the gaze point weight algorithm; the master node dynamically allocates LOD parameters, and the slave node realizes sub-pixel splicing of special-shaped large screens based on the spherical coordinate system model; using the Q-learning algorithm to predict the user operation sequence, and combining the ASTP protocol to realize dual-channel bandwidth adaptive allocation. The system includes dynamic blocking, pipeline scheduling, multi-node collaboration, intelligent preloading, and exception handling modules. The present invention can realize real-time interactive visualization of large amounts of data, improve rendering efficiency, and reduce hardware resource usage.

Description

Technical Field

[0001] The present invention relates to the field of image rendering technology, and in particular to an interactive large-screen visualization system and method based on real-time rendering of multi-dimensional data. Background Art

[0002] As the demand for real-time visualization in scenarios such as smart cities grows, traditional rendering engines face three technical bottlenecks when dealing with multi-dimensional data: first, insufficient data throughput efficiency. Existing compression algorithms struggle to balance the integrity and transmission efficiency of millions of data streams; second, an imperfect dynamic LOD switching mechanism and inefficient GPU-CPU collaboration lead to screen tearing and rendering delays; and third, interactive responses suffer from operation synchronization delays, seriously impacting real-time decision-making.

[0003] Existing improvement proposals are often one-sided: some employ inefficient compression algorithms that increase computational load, others rely on pre-rendering techniques that sacrifice dynamic interactivity, and distributed rendering is hampered by node synchronization challenges. These limitations highlight systemic flaws in existing rendering engines in terms of data processing pipeline optimization, hardware resource coordination, and real-time interactive architecture. Summary of the Invention

[0004] The present invention provides an interactive large-screen visualization system and method based on real-time rendering of multi-dimensional data to solve the data throughput bottleneck, rendering delay bottleneck and interactive response bottleneck problems existing in traditional large-screen visualization systems in the prior art.

[0005] The present invention provides an interactive large-screen visualization method based on real-time rendering of multidimensional data, the method comprising:

[0006] Step S1: Dynamically block LOD encoding is performed on the input multi-dimensional data stream. A non-uniform octree is used to divide the spatial blocks. Each block is independently bound to LOD parameters. The incremental data is then compressed using a streaming compression engine using ZSTD+Huffman hybrid compression. The compression rate is dynamically adjusted according to the real-time network bandwidth.

[0007] Step S2: Processing the compressed block data through the GPU-CPU asynchronous pipeline scheduling mechanism. The pipeline includes three levels of asynchronous execution units: a data decoding unit CPU, an LOD pre-calculation unit, and a rendering unit. The processing priority of each block is dynamically adjusted based on user interaction behavior.

[0008] Step S3: Using a master-slave rendering architecture, the master node calculates the global LOD distribution and distributes it to the slave nodes. The slave nodes render the local images based on the blocks, and combine the dynamic projection mapping table to splice the multi-node output images onto the special-shaped large screen. Sub-pixel projection compensation technology is used to eliminate visible splicing defects.

[0009] Step S4: Predict user operation sequences based on reinforcement learning, dynamically preload block data with high probability of access to the local cache, and implement dual-channel bandwidth allocation of the control channel and data channel through the adaptive streaming protocol ASTP to achieve near real-time interactive response.

[0010] Furthermore, the present application also proposes that the specific implementation of the dynamic block LOD encoding in step S1 includes:

[0011] Construct a non-uniform octree based on the multi-dimensional data density, and divide the block size into d×d×d, where d=2 n And n∈[3,5]. When the data update frequency is greater than 60Hz, n=3 is used: the block size is 8×8×8. When the update frequency is less than 30Hz, n=4 is used: the block size is 16×16×16.

[0012] The ZSTD algorithm is used to perform the first round of compression on the incremental data of each block, and the compression rate threshold T is calculated based on the historical bandwidth average:

[0013]

[0014] The ZSTD compressed data is further Huffman coded, and the compression level is inversely adjusted according to the block LOD level, and the minimum compression rate β∈[0.15,0.25] is adaptively adjusted.

[0015] Furthermore, the present application also proposes that the asynchronous pipeline scheduling mechanism in step S2 includes:

[0016] Assign an independent task queue to each block, and the task priority P is calculated by the formula:

[0017]

[0018] in d is the visual distance from the block center to the user's gaze point;

[0019] When a change in user interaction behavior is detected, the high-priority block task is inserted into the head of the pipeline queue through the queue insertion algorithm;

[0020] When the LOD pre-computation unit performs LOD pre-computation, it adopts a tiled computing strategy to divide the computing task into 16×16 thread blocks for parallel processing.

[0021] Furthermore, the present application also proposes that the method for generating the dynamic projection mapping table in step S3 includes:

[0022] Establish a spherical coordinate system model for special-shaped large screens and map the output image of each rendering node to spherical coordinates (u, v);

[0023] The bicubic interpolation algorithm is used to smoothly transition the pixels in the overlapping area of ​​adjacent node images. The interpolation weight W is determined by the formula:

[0024]

[0025] Generate a pixel offset correction matrix and combine it with sub-pixel projection technology to align and compensate the stitched images to eliminate visible stitching defects.

[0026] Furthermore, the present application also proposes that the implementation of predicting the user operation sequence based on reinforcement learning in step S4 and dynamically preloading the block data with high probability of access to the local cache includes:

[0027] Construct a Markov state transition model of the user operation sequence, where the state space S = {current viewpoint coordinates, operation type, time interval};

[0028] The Q-learning algorithm is used to update the preloading strategy. The reward function R is defined as: R = 0.7 × cache hit rate + 0.3 × (1-cache space occupancy) - [network packet loss rate > 5%? 0.2:0.1] × network bandwidth consumption. When the confidence level of the predicted k-step operation is greater than 85%, the preloading request for the corresponding block is triggered.

[0029] Furthermore, the present application also proposes that the implementation of the adaptive streaming protocol ASTP includes:

[0030] The control channel uses UDP broadcast to transmit LOD metadata, and the data channel uses multiple TCP connections to transmit block content;

[0031] Dynamically allocate bandwidth B for each TCP connection i , the allocation formula is:

[0032]

[0033] Among them, P j The priority of all active connections;

[0034] When the data packet loss rate is detected to be greater than 5%, it automatically switches to FEC forward error correction mode and reduces the bandwidth allocation weight of non-critical data streams.

[0035] Furthermore, the present application also proposes that the method also includes an abnormality recovery mechanism:

[0036] Monitor the GPU memory usage of the rendering node in real time. When the usage is > 90%, automatically reduce the LOD level of low-priority blocks.

[0037] If the network delay between nodes is greater than 50ms and less than or equal to 100ms, predictive rendering mode is enabled; if the delay is greater than 100ms, keyframe I-Frame transmission-only mode is switched.

[0038] Furthermore, the present application also proposes an interactive large-screen visualization system based on real-time rendering of multi-dimensional data, which is used to execute the above method. The system includes:

[0039] Dynamic Blocking Module: Configures a non-uniform octree encoder and a streaming compression engine to divide spatial blocks according to data density and perform hybrid compression;

[0040] Pipeline scheduling module: Integrates CPU decoding unit, GPU computing unit and rendering unit to implement three-level asynchronous pipeline and dynamic priority queue interruption algorithm;

[0041] Multi-node collaboration module: includes the master node LOD distributor and the slave node renderer, and drives the splicing of special-shaped large screens through dynamic projection mapping tables;

[0042] Intelligent preloading module: deploys reinforcement learning prediction models and the ASTP protocol stack to implement cache management and adaptive bandwidth allocation;

[0043] Exception handling module: monitors GPU memory and network status in real time, triggering rendering degradation or error correction transmission mechanism.

[0044] This application provides an interactive large-screen visualization method and system based on real-time rendering of multidimensional data. Through dynamic block LOD encoding, GPU-CPU asynchronous pipeline scheduling, a master-slave rendering architecture, and a reinforcement learning-based preloading strategy, it achieves efficient data processing, rendering, and interaction. This solution effectively solves the problem of real-time processing and visualization of large-scale multidimensional data, with the advantages of improving data throughput efficiency, optimizing rendering performance, and improving interactive response speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of the interactive large-screen visualization method based on real-time rendering of multi-dimensional data according to the present invention;

[0046] Figure 2 This is an architecture diagram of the interactive large-screen visualization system based on real-time rendering of multi-dimensional data in the present invention. DETAILED DESCRIPTION

[0047] The present invention aims to solve the data throughput bottleneck, rendering delay bottleneck and interactive response bottleneck existing in traditional large-screen visualization systems. Traditional systems have obvious deficiencies in terms of real-time parsing and compression of multi-dimensional data, dynamic detail level switching, and synchronization of user operations and rendering pipelines. They cannot meet the needs of real-time interaction of large-scale data in scenarios such as smart cities and industrial Internet of Things.

[0048] The above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods of the specification to better understand the above technical solution. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention. It should be understood that the present invention is not limited to the example embodiments used only to explain the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In addition, it should be noted that, for the convenience of description, only the parts related to the present invention, rather than all, are shown in the drawings.

[0049] Example 1

[0050] like Figure 1 The flowchart of the interactive large-screen visualization method based on real-time rendering of multi-dimensional data is shown as follows: This application realizes full-link optimization from data stream processing to rendering interaction through dynamic block LOD encoding, GPU-CPU asynchronous pipeline scheduling, master-slave rendering architecture and reinforcement learning to predict user operation sequences. Specifically, this application uses a non-uniform octree to divide the space blocks, and combines the streaming compression engine to perform ZSTD+Huffman hybrid compression on the incremental data. The compression rate is dynamically adjusted according to the real-time network bandwidth. The compressed block data is processed by the GPU-CPU asynchronous pipeline scheduling mechanism. The pipeline includes a data decoding unit, an LOD pre-calculation unit and a rendering unit, and the processing priority of each block is dynamically adjusted based on user interaction behavior. Using a master-slave rendering architecture, the master node calculates the global LOD distribution and distributes it to the slave node. The slave node renders the local screen according to the block, and combines the dynamic projection mapping table to splice the multi-node output screen to the special-shaped large screen to ensure that the pixel alignment error is ≤0.1px. Based on reinforcement learning, it predicts user operation sequences, dynamically preloads block data with high probability of access to the local cache, and implements dual-channel bandwidth allocation for the control channel and data channel through an adaptive streaming protocol, with end-to-end latency ≤20ms.

[0051] Traditional large-screen visualization systems often face insufficient data throughput when processing multidimensional data. To address this issue, this application proposes dynamic block LOD encoding and hybrid compression technology. This technology divides spatial blocks using a non-uniform octree and combines it with a streaming compression engine to perform ZSTD+Huffman hybrid compression on incremental data. The compression rate is dynamically adjusted based on the real-time network bandwidth. This not only improves data parsing and compression efficiency, but also ensures that data throughput meets real-time requirements. Regarding data processing, traditional methods suffer from low GPU-CPU collaboration efficiency, resulting in significant rendering latency. To address this, this application utilizes a GPU-CPU asynchronous pipeline scheduling mechanism. This mechanism, through three levels of asynchronous execution units—data decoding, LOD pre-calculation, and rendering—improves data processing efficiency and reduces rendering latency. Regarding user interaction, traditional systems struggle to achieve millisecond-level synchronization. This application uses reinforcement learning to predict user action sequences, dynamically preloads block data with a high probability of being accessed into a local cache, and implements dual-channel bandwidth allocation for the control and data channels through an adaptive streaming protocol. This enables rapid user interaction response and efficient data transmission, ensuring end-to-end latency of ≤20ms.

[0052] Dynamic block LOD encoding is one of the key technologies of this application. The spatial blocks are divided by non-uniform octrees, each block is independently bound to LOD parameters, and the incremental data is compressed with ZSTD+Huffman hybrid compression in combination with the streaming compression engine. The compression rate is dynamically adjusted according to the real-time network bandwidth. The GPU-CPU asynchronous pipeline scheduling mechanism includes three-level asynchronous execution units: data decoding unit, LOD pre-calculation unit and rendering unit. The data decoding unit is responsible for decoding the compressed block data, the LOD pre-calculation unit dynamically adjusts the processing priority of each block based on user interaction behavior, and the rendering unit is responsible for the final picture rendering. The master-slave rendering architecture calculates the global LOD distribution through the master node and distributes it to the slave node. The slave node renders the local picture according to the block, and combines the dynamic projection mapping table to splice the multi-node output picture to the special-shaped large screen to ensure that the pixel alignment error is ≤0.1px. Reinforcement learning predicts user operation sequences by constructing a Markov state transition model of user operation sequences, using the Q-learning algorithm to update the preloading strategy, dynamically preloading block data with a high probability of access to the local cache, and implementing dual-channel bandwidth allocation for the control channel and data channel through an adaptive streaming transmission protocol to ensure end-to-end latency ≤20ms.

[0053] Compared with the existing technology, this application has obvious advantages in data throughput, rendering latency and interactive response. Through dynamic block LOD encoding and hybrid compression technology, the parsing and compression efficiency of multi-dimensional data is improved, ensuring that the data throughput meets real-time requirements. The GPU-CPU asynchronous pipeline scheduling mechanism improves data processing efficiency and reduces rendering latency. The master-slave rendering architecture and dynamic projection mapping table realize multi-node collaborative rendering, ensuring the accuracy of picture splicing. Reinforcement learning predicts user operation sequences and adaptive streaming transmission protocols realize rapid response to user interactions and efficient data transmission, ensuring end-to-end latency ≤20ms.

[0054] Dynamic block LOD encoding and hybrid compression techniques address the efficient parsing and compression of multidimensional data, ensuring data throughput meets real-time requirements. A GPU-CPU asynchronous pipeline scheduling mechanism utilizes three levels of asynchronous execution: data decoding, LOD pre-calculation, and rendering, improving data processing efficiency and reducing rendering latency. A master-slave rendering architecture involves a master node calculating the global LOD distribution and distributing it to slave nodes. Slave nodes then render local images based on the blocks. Dynamic projection mapping is then used to stitch the outputs from multiple nodes onto the custom-shaped large screen, ensuring pixel alignment error of ≤0.1px. Reinforcement learning predicts user action sequences by constructing a Markov state transition model of user action sequences. Using a Q-learning algorithm, the preloading strategy is updated to dynamically preload blocks with high probability of access into the local cache. An adaptive streaming protocol is used to implement dual-channel bandwidth allocation between the control and data channels, ensuring end-to-end latency of ≤20ms. The coordinated implementation of these steps addresses data throughput, rendering latency, and interactive responsiveness issues in real-time rendering of multidimensional data and interactive large-screen visualization, achieving efficient, real-time visualization.

[0055] Furthermore, the present application also proposes to construct a non-uniform octree according to the multi-dimensional data density, and divide the block size into d×d×d, where d=2 n And n∈[3,5]. When the data update frequency is greater than 60Hz, n=3(8×8×8), and when the update frequency is less than 30Hz, n=4(16×16×16). For the incremental data of each block, the ZSTD algorithm is used for the first round of compression. The compression rate threshold T is calculated by the historical bandwidth average:

[0056] T = 0.6 × (current bandwidth / maximum bandwidth) + 0.2 + 0.05 × bandwidth change rate;

[0057] The ZSTD compressed data is further Huffman coded, and the compression level is inversely adjusted according to the block LOD level, and the minimum compression rate β∈[0.15,0.25] is adaptively adjusted.

[0058] Construct a non-uniform octree based on the multi-dimensional data density, and divide the block size into d×d×d, where d=2 n And n∈[3,5] is adjusted dynamically. This technical feature is that the block size can be flexibly adjusted according to the different data densities, thereby improving the coding efficiency. For the incremental data of each block, the ZSTD algorithm is used for the first round of compression, and the compression rate threshold T is calculated by the historical bandwidth average. This technical feature adapts to different bandwidth conditions by dynamically adjusting the compression rate, thereby improving data transmission efficiency. The data compressed by ZSTD is further Huffman encoded, and the compression level is adjusted inversely according to the block LOD level. This technical feature ensures data transmission efficiency at different levels of detail by further compressing the data and adjusting the compression level according to the LOD level. Through the comprehensive application of these technical means, the present application proposes a coding method that is dynamically adjusted according to data density and bandwidth conditions, thereby effectively solving the coding efficiency problem caused by uneven density of multi-dimensional data.

[0059] The construction of a non-uniform octree is based on the density of multidimensional data. Specifically, the density of each block is determined by analyzing the distribution of data points, and the block sizes are then divided accordingly. In practical applications, adaptive algorithms can be used to dynamically adjust block sizes based on data update frequency to accommodate varying data update requirements. The ZSTD algorithm is an efficient lossless compression algorithm suitable for large-scale data compression. By setting a compression ratio threshold T, the compression ratio can be dynamically adjusted based on current bandwidth conditions to ensure efficient data transmission. Huffman coding is a classic entropy coding method. By further compressing ZSTD-compressed data, the data volume can be further reduced and transmission efficiency improved. The compression level is inversely adjusted based on the block's LOD level. Specifically, the Huffman coding compression level can be dynamically adjusted based on each block's LOD level to ensure efficient data transmission at different levels of detail.

[0060] This application introduces a non-uniform octree, ZSTD algorithm, and Huffman coding techniques to implement a coding method that dynamically adjusts to data density and bandwidth conditions, effectively resolving coding efficiency issues caused by uneven multi-dimensional data density. Compared to existing technologies, this application's technical solution offers significant advantages in data compression and transmission efficiency, significantly improving data transmission efficiency while ensuring data quality, and adapting to varying bandwidth conditions and data densities.

[0061] Furthermore, this application also proposes to allocate an independent task queue to each block, and the task priority P is calculated by the formula:

[0062]

[0063] in d is the visual distance from the block center to the user's gaze point;

[0064] When a change in user interaction behavior is detected, the high-priority block task is inserted into the head of the pipeline queue through the queue insertion algorithm;

[0065] When GPUComputeShader performs LOD pre-computation, it adopts a tiled computing strategy to divide the computing task into 16×16 thread blocks for parallel processing.

[0066] The asynchronous pipeline scheduling mechanism of this application allocates an independent task queue for each block and uses a specific priority calculation formula to dynamically adjust the task priority according to the user's gaze point weight, current transmission delay and data volume. When a change in user interaction behavior is detected, the queue-jumping algorithm can insert high-priority block tasks into the head of the queue to be processed to ensure that important tasks are processed first. In addition, GPUComputeShader adopts a tiled computing strategy to divide the computing tasks into 16×16 thread blocks for parallel processing, which improves the efficiency of LOD pre-computation. These technical means jointly solve the problems of task priority scheduling and rendering efficiency.

[0067] In this application, assigning an independent task queue to each block is a method to ensure task independence and efficient management. The calculation formula of task priority is:

[0068] It can adjust task priorities based on real-time conditions to ensure timely processing of critical tasks. The application of a queue-jumping algorithm allows high-priority tasks to be quickly inserted into the queue when user interaction behavior changes, thereby improving system responsiveness. GPUComputeShader uses a tiled computing strategy for LOD precomputation, dividing the computational task into 16×16 thread blocks for parallel processing. This strategy fully utilizes the GPU's parallel computing capabilities and improves computational efficiency.

[0069] Therefore, this application effectively solves the problems of task priority scheduling and rendering efficiency during data processing through an asynchronous pipeline scheduling mechanism, using independent task queues, priority calculation formulas, queue-jumping algorithms, and tiled computing strategies. Compared with existing technologies, this application can dynamically adjust task priorities to ensure that important tasks are processed first, and improves computing efficiency through tiled computing strategies, thereby achieving more efficient rendering and faster system response.

[0070] Furthermore, the present application also proposes that the method for generating the dynamic projection mapping table in step S3 includes establishing a spherical coordinate system model of the special-shaped large screen and mapping the output image of each rendering node to spherical coordinates (u, v). In this way, the images of different nodes can be mapped to a unified coordinate system, which is convenient for subsequent splicing and alignment operations. The bicubic interpolation algorithm is used to smoothly transition the pixels in the overlapping area of ​​the adjacent node images, and the interpolation weight W is determined by the formula:

[0071]

[0072] This interpolation method can effectively reduce edge effects in overlapping areas of different node images, ensuring smooth transitions. A pixel offset correction matrix is ​​generated, and combined with NVIDIA Warp API's sub-pixel projection technology, sub-pixel alignment compensation is performed on the stitched images, ensuring that the pixel alignment error of the final stitched image is ≤0.1px.

[0073] When establishing a spherical coordinate system model for a special-shaped large screen, it is first necessary to accurately measure the physical size and shape of the special-shaped large screen, and then construct a spherical coordinate system model based on these measurement data. The output image of each rendering node will be mapped according to the (u, v) coordinates of the spherical coordinate system to ensure that the image can be processed in a unified coordinate system. The bicubic interpolation algorithm is used to smoothly transition the pixels in the overlapping area of ​​the adjacent node images. The interpolation weight W is determined by calculating the ratio of the edge distance to the width of the overlapping area. This interpolation method can effectively reduce the edge effect of the overlapping area of ​​the image and ensure a smooth transition of the image. When generating the pixel offset correction matrix, the grid deformation function of the NVIDIA Warp API is used to make sub-pixel adjustments to the spliced ​​image to correct the pixel offset problem caused by splicing.

[0074] As a preferred implementation, a high-precision camera and laser rangefinder can be used to measure the physical size and shape of the custom-shaped large screen to ensure the accuracy of the spherical coordinate system model. Furthermore, a GPU can be used to accelerate the bicubic interpolation algorithm, improving computational efficiency. When generating the pixel offset correction matrix, image processing algorithms can be used to perform detailed pixel-level adjustments on the stitched images, further improving image alignment accuracy.

[0075] This application establishes a spherical coordinate system model for special-shaped large screens, maps the output image of each rendering node to spherical coordinates (u, v), uses a bicubic interpolation algorithm to smoothly transition the pixels in the overlapping areas of adjacent node images, generates a pixel offset correction matrix, and combines the sub-pixel projection technology of NVIDIA Warp API to align and compensate the spliced ​​images. In this way, it can effectively solve the technical problems of pixel alignment error and smooth transition of overlapping areas of images when multiple nodes of special-shaped large screens are spliced ​​together, ensuring the accuracy and consistency of the final spliced ​​image. Compared with the existing technology, this application provides a more accurate and efficient splicing method for special-shaped large screens, which significantly improves the display effect of the large-screen visualization system.

[0076] Furthermore, this application proposes to construct a Markov state transition model for user operation sequences, where the state space S = {current viewpoint coordinates, operation type, time interval}. The Q-learning algorithm is used to update the preload strategy, and the reward function R is defined as:

[0077] R = 0.7 × cache hit rate + 0.3 × (1 – cache space occupancy) – [network packet loss rate > 5%? 0.2:0.1] × network bandwidth consumption.

[0078] When the confidence level of the predicted k-step operation is greater than 85%, a preload request for the corresponding block is triggered.

[0079] By constructing a Markov state transition model of user operation sequences, we can effectively capture the patterns and regularities of user operations. The Q-learning algorithm continuously updates preloading strategies, optimizing cache hit rates, cache space utilization, and network bandwidth consumption through the definition of the reward function R. When the confidence level of a predicted future operation exceeds 85%, the system preloads data for blocks with a high probability of access, thereby improving the accuracy and efficiency of data preloading.

[0080] Furthermore, a Markov state transition model of the user operation sequence can be constructed by collecting the user's operation data in the large-screen visualization system. Specifically, the state space S includes the current viewpoint coordinates, operation type, and time interval, where the current viewpoint coordinates represent the area the user is currently focusing on, the operation type includes zooming, panning, and other operations, and the time interval is used to record the time point when the operation occurs. The Q-learning algorithm updates the preloading strategy by continuously interacting with the environment, enabling the system to make the optimal preloading decision when facing different user operations. The definition of the reward function R comprehensively considers the cache hit rate, cache space occupancy rate, and network bandwidth consumption. When the network packet loss rate is >5%, the bandwidth consumption penalty coefficient is increased to 0.2 to achieve overall performance optimization.

[0081] As a preferred implementation, when the system predicts a user's next k steps with a confidence level exceeding 85%, it triggers a preload request for the corresponding block. Specifically, based on the user's current operating pattern, the system predicts blocks with a high probability of being accessed in the future and preloads the data for these blocks into the local cache. This allows users to quickly access the required data during actual operations, reducing wait times and improving system responsiveness and user experience.

[0082] By introducing the Markov state transition model and the Q-learning algorithm, this application achieves significant results in predicting user action sequences and preloading data. Compared to traditional methods, this application can more accurately predict user actions, improve the accuracy and efficiency of data preloading, reduce cache space usage and network bandwidth consumption, and thus achieve a more efficient large-screen visualization system.

[0083] Furthermore, the present application also proposes that the control channel uses UDP broadcast to transmit LOD metadata, and the data channel uses multiple TCP connections to transmit block content. Dynamically allocate bandwidth B for each TCP connection i , the allocation formula is:

[0084]

[0085] When the data packet loss rate is detected to be greater than 5%, it automatically switches to FEC forward error correction mode and reduces the bandwidth allocation weight of non-critical data streams.

[0086] This application adopts the method of separating the control channel and the data channel, using UDP and TCP protocols for data transmission respectively. The control channel transmits LOD metadata through UDP broadcast, ensuring the rapid transmission of control information, while the data channel transmits block content through multiple TCP connections, ensuring the reliable transmission of data content. In terms of bandwidth allocation, by dynamically allocating bandwidth B i , optimizing the utilization of network resources. Specifically, the bandwidth allocation formula is:

[0087]

[0088] Among them, P j Prioritize all active connections. This bandwidth allocation mechanism dynamically adjusts bandwidth resources based on the priority of each connection. When a packet loss rate exceeds 5%, the system automatically switches to forward error correction (FEC) mode and prioritizes bandwidth for non-critical data streams, improving data transmission reliability by enabling forward error correction.

[0089] Furthermore, the control channel uses UDP broadcast to transmit LOD metadata, which can transmit control information to all relevant nodes in a relatively short period of time, making it suitable for scenarios where control information needs to be transmitted quickly. The data channel uses multiple TCP connections to transmit block content. This multi-channel TCP connection method can improve the parallelism and reliability of data transmission. The dynamic bandwidth allocation mechanism ensures the rational allocation and efficient utilization of bandwidth resources by monitoring the priority and bandwidth usage of each TCP connection in real time. When the packet loss rate exceeds 5%, it automatically switches to FEC forward error correction mode and prioritizes critical data streams through weight adjustment. This mechanism can still ensure the accuracy and integrity of data transmission in a network environment with a high packet loss rate.

[0090] Therefore, this application separates the control channel from the data channel, using UDP and TCP protocols for data transmission, ensuring the rapid transmission of control information and the reliable transmission of data content. By dynamically allocating bandwidth, network resource utilization is optimized and data transmission efficiency is improved. When the packet loss rate exceeds 5%, it automatically switches to FEC (Forward Error Correction) mode, improving data transmission reliability. Compared with existing technologies, this application has significant advantages in network resource utilization and data transmission reliability.

[0091] Furthermore, the present application also proposes that the method also includes an abnormality recovery mechanism:

[0092] Monitor the GPU memory usage of the rendering node in real time. When the usage is > 90%, automatically reduce the LOD level of low-priority blocks.

[0093] If the network delay between nodes is >50ms and ≤100ms, predictive rendering mode is enabled; if the delay is >100ms, switch to keyframe (I-Frame) transmission only mode.

[0094] The technical solution includes two parts: real-time monitoring and local downgraded rendering. Real-time monitoring of the GPU memory usage of the rendering node. When the memory usage exceeds 90%, the LOD level of low-priority blocks is lowered to reduce the use of memory to ensure the stability of the system. When it is detected that the network delay between nodes exceeds 50ms, two-level processing is performed: if the delay is between 50ms and 100ms, the predictive rendering mode is enabled; if the delay is >100ms, it switches to the key frame transmission only mode, and uses historical cache data to generate an approximate picture to ensure the continuity of the picture. The above technical means have solved the problem of ensuring the stability and continuity of the large-screen visualization system when anomalies occur in the rendering node.

[0095] Real-time monitoring of the render node's GPU memory usage is a key step in the exception recovery mechanism. When the memory usage exceeds 90%, the system automatically reduces the LOD level of low-priority blocks. LOD (Level of Detail) refers to the reduction of the model's level of detail by reducing the number of polygons in 3D computer graphics. By reducing the LOD level of low-priority blocks, video memory usage can be reduced, thereby avoiding system instability caused by excessive memory usage.

[0096] When the network latency between nodes exceeds 50ms, the system will activate local degraded rendering mode. At this point, the system will use historical cached data to generate an approximate image. Historical cached data refers to previously rendered image data that is stored for future use. By using this cached data, the system can still generate relatively smooth images even when network latency is high, thereby ensuring the continuity of the user experience. If the latency is >100ms, it switches to keyframe (I-Frame) transmission only mode.

[0097] Furthermore, the exception recovery mechanism of this application effectively solves system stability and continuity issues when rendering nodes experience exceptions through real-time monitoring and local downgraded rendering. Compared with existing technologies, the technical solution of this application can improve system stability and user experience without increasing hardware costs by automatically adjusting LOD levels and using historical cached data.

[0098] Example 2

[0099] like Figure 2 The interactive large-screen visualization system architecture based on real-time rendering of multi-dimensional data is shown in the diagram, including:

[0100] Dynamic Blocking Module: Configures a non-uniform octree encoder and a streaming compression engine to divide spatial blocks according to data density and perform hybrid compression;

[0101] Pipeline scheduling module: Integrates CPU decoding unit, GPU computing unit and rendering unit to implement three-level asynchronous pipeline and dynamic priority queue interruption algorithm;

[0102] Multi-node collaboration module: includes the master node LOD distributor and the slave node renderer, and drives the splicing of special-shaped large screens through dynamic projection mapping tables;

[0103] Intelligent preloading module: deploys reinforcement learning prediction models and the ASTP protocol stack to implement cache management and adaptive bandwidth allocation;

[0104] Exception handling module: monitors GPU memory and network status in real time, triggering rendering degradation or error correction transmission mechanism.

[0105] The system consists of five main modules, each playing a different role in solving technical problems. The dynamic segmentation module uses a non-uniform octree encoder and a streaming compression engine to partition spatial blocks based on data density and perform hybrid compression, addressing the inefficient real-time parsing and compression of multidimensional data. The pipeline scheduling module integrates the CPU decoding unit, GPU computing unit, and rendering unit. Through a three-level asynchronous pipeline and a dynamic priority queue-jumping algorithm, it addresses rendering latency and inefficient GPU-CPU collaboration. The multi-node collaboration module uses a master node LOD allocator and slave node renderers, and utilizes a dynamic projection mapping table to drive the splicing of special-shaped large-screen displays, ensuring high precision and low error. The intelligent preloading module deploys a reinforcement learning prediction model and the ASTP protocol stack to implement cache management and adaptive bandwidth allocation, addressing the real-time nature of user interaction responses. The exception handling module monitors GPU memory and network status in real time, triggering downgraded rendering or error-correcting transmission mechanisms to ensure stable system operation even under abnormal circumstances. Through the above technical means, the system can effectively realize full-link optimization from data stream processing to rendering interaction, ensuring end-to-end delay ≤20ms, and solving the data throughput, rendering delay and interactive response bottlenecks existing in traditional large-screen visualization systems.

[0106] The non-uniform octree encoder of the dynamic block segmentation module can automatically adjust the block size according to the data density, and the streaming compression engine dynamically adjusts the compression rate according to the real-time network bandwidth. The pipeline scheduling module effectively reduces rendering latency and improves GPU-CPU collaboration efficiency through a three-level asynchronous pipeline and a dynamic priority queue-jumping algorithm. The multi-node collaboration module uses a dynamic projection mapping table to achieve high-precision splicing of large-screen special-shaped displays. The reinforcement learning prediction model deployed by the intelligent preloading module can accurately predict user operations and load data in advance, and the ASTP protocol stack ensures adaptive bandwidth allocation. The exception handling module monitors the GPU video memory and network status in real time, and can promptly trigger downgraded rendering or error correction transmission mechanisms to ensure stable operation of the system under abnormal conditions.

[0107] Furthermore, the dynamic blocking module can adopt a variety of non-uniform octree encoding algorithms, such as an adaptive adjustment algorithm based on data density or a prediction algorithm based on historical data. The three-level asynchronous pipeline of the pipeline scheduling module can adopt different scheduling strategies, such as a dynamic adjustment strategy based on task priority or a predictive scheduling strategy based on user behavior. The dynamic projection mapping table of the multi-node collaboration module can adopt a variety of generation methods, such as a mapping method based on a spherical coordinate system or a mapping method based on a plane coordinate system. The reinforcement learning prediction model of the intelligent preloading module can adopt different reinforcement learning algorithms, such as a Q-learning algorithm or a deep reinforcement learning algorithm. The degradation rendering mechanism of the exception handling module can adopt a variety of degradation strategies, such as a degradation strategy based on historical data or a degradation strategy based on real-time monitoring.

[0108] This application proposes an interactive large-screen visualization system based on real-time rendering of multidimensional data. Through the collaborative work of five main modules, it achieves full-link optimization from data stream processing to rendering interaction. Compared with existing technologies, this system has significant advantages in data throughput, rendering latency, and interactive response, and can effectively address the technical bottlenecks of traditional large-screen visualization systems.

[0109] It should be understood that the embodiments disclosed in the present invention and the above description can enable those skilled in the art to use the present invention to implement the present invention. At the same time, the present invention is not limited to the embodiments mentioned above. It should be understood that those skilled in the art can still modify the technical solutions described in the above embodiments or replace some of the technical features therein with equivalents; and such modifications or replacements do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention and are all included in the scope of protection of the present invention.

Claims

1. An interactive large-screen visualization method based on real-time rendering of multi-dimensional data, characterized in that: The method comprises the following steps: Step S1: Dynamically block LOD encoding is performed on the input multi-dimensional data stream. A non-uniform octree is used to divide the spatial blocks. Each block is independently bound to LOD parameters. The incremental data is then compressed using a streaming compression engine using ZSTD+Huffman hybrid compression. The compression rate is dynamically adjusted according to the real-time network bandwidth. Step S2: Processing the compressed block data through the GPU-CPU asynchronous pipeline scheduling mechanism. The pipeline includes three levels of asynchronous execution units: a data decoding unit CPU, an LOD pre-calculation unit, and a rendering unit. The processing priority of each block is dynamically adjusted based on user interaction behavior. Step S3: Using a master-slave rendering architecture, the master node calculates the global LOD distribution and distributes it to the slave nodes. The slave nodes render the local images based on the blocks, and combine the dynamic projection mapping table to splice the multi-node output images onto the special-shaped large screen. Sub-pixel projection compensation technology is used to eliminate visible splicing defects. Step S4: Based on reinforcement learning, the user operation sequence is predicted, and the block data with high probability of access is dynamically preloaded into the local cache. The adaptive streaming protocol ASTP is used to implement dual-channel bandwidth allocation of the control channel and the data channel to achieve near real-time interactive response; The method for generating the dynamic projection mapping table in step S3 includes: Establish a spherical coordinate system model for special-shaped large screens and map the output image of each rendering node to spherical coordinates (u, v); The bicubic interpolation algorithm is used to smoothly transition the pixels in the overlapping area of ​​adjacent node images. The interpolation weight W is determined by the formula: ; Generate a pixel offset correction matrix and combine it with sub-pixel projection technology to align and compensate the stitched images to eliminate visible stitching defects; The implementation of the adaptive streaming transmission protocol ASTP includes: The control channel uses UDP broadcast to transmit LOD metadata, and the data channel uses multiple TCP connections to transmit block content; Dynamically allocate bandwidth for each TCP connection , the allocation formula is: ; in The priority of all active connections; When the data packet loss rate is detected to be greater than 5%, it automatically switches to FEC (forward error correction) mode and reduces the bandwidth allocation weight of non-critical data streams.

2. The interactive large-screen visualization method based on real-time rendering of multi-dimensional data according to claim 1, characterized in that: The specific implementation of the dynamic block LOD encoding in step S1 includes: Construct a non-uniform octree based on the multi-dimensional data density, and divide the block size into d×d×d, where and When the data update frequency is greater than 60 Hz, n=3 is used: the block size is 8×8×8; when the update frequency is less than 30 Hz, n=4 is used: the block size is 16×16×16; The ZSTD algorithm is used to perform the first round of compression on the incremental data of each block, and the compression rate threshold T is calculated based on the historical bandwidth average: ; The ZSTD compressed data is further Huffman coded, and the compression level is inversely adjusted according to the block LOD level, and the minimum compression rate β∈[0.15,0.25] is adaptively adjusted.

3. The interactive large-screen visualization method based on real-time rendering of multi-dimensional data according to claim 2, characterized in that: The asynchronous pipeline scheduling mechanism in step S2 includes: Assign an independent task queue to each block, and the task priority P is calculated by the formula: ; Where the gaze weight = , d is the visual distance from the block center to the user's gaze point; When a change in user interaction behavior is detected, the high-priority block task is inserted into the head of the pipeline queue through the queue insertion algorithm; When the LOD pre-computation unit performs LOD pre-computation, it adopts a tiled computing strategy to divide the computing task into 16×16 thread blocks for parallel processing.

4. The interactive large-screen visualization method based on real-time rendering of multi-dimensional data according to claim 3, characterized in that: The implementation of predicting the user operation sequence based on reinforcement learning and dynamically preloading the block data with high probability of access to the local cache in step S4 includes: Construct a Markov state transition model of the user operation sequence, where the state space S = {current viewpoint coordinates, operation type, time interval}; The Q-learning algorithm is used to update the preloading strategy. The reward function R is defined as: R = 0.7 × cache hit rate + 0.3 × (1-cache space occupancy) - [network packet loss rate > 5%? 0.2:0.1] × network bandwidth consumption. When the confidence level of the predicted k-step operation is > 85%, the preloading request of the corresponding block is triggered.

5. The interactive large-screen visualization method based on real-time rendering of multi-dimensional data according to claim 4, characterized in that: The method also includes an exception recovery mechanism: Monitor the GPU memory usage of the rendering node in real time. When the usage is greater than 90%, automatically reduce the LOD level of low-priority blocks. If the network delay between nodes is greater than 50ms and less than or equal to 100ms, predictive rendering mode is enabled; if the delay is greater than 100ms, keyframe I-Frame transmission-only mode is switched.

6. An interactive large-screen visualization system based on real-time rendering of multidimensional data, used to implement the interactive large-screen visualization method based on real-time rendering of multidimensional data according to any one of claims 1 to 5, characterized in that: The system comprises: Dynamic Blocking Module: Configures a non-uniform octree encoder and a streaming compression engine to divide spatial blocks according to data density and perform hybrid compression; Pipeline scheduling module: Integrates CPU decoding unit, GPU computing unit and rendering unit to implement three-level asynchronous pipeline and dynamic priority queue-jumping algorithm; Multi-node collaboration module: includes the master node LOD distributor and the slave node renderer, and drives the splicing of special-shaped large screens through dynamic projection mapping tables; Intelligent preloading module: deploys reinforcement learning prediction models and the ASTP protocol stack to implement cache management and adaptive bandwidth allocation; Exception handling module: monitors GPU memory and network status in real time, triggering rendering degradation or error correction transmission mechanism.

Images