Three-dimensional scene optimization method and system for collaborative rendering of dynamic LOD and view cone elimination based on space-time prediction

Abstract

The invention relates to the field of scene rendering, and particularly discloses a three-dimensional scene optimization method and system for collaborative rendering of dynamic LOD and view cone rejection based on space-time prediction, and the method comprises the steps: dynamically calculating the visibility frequency of an object, and dynamically adjusting the loading and rendering modes of objects with different priorities; predicting a view cone range of multiple frames in the future by using an LSTM network architecture; dividing the scene into uniform grid blocks, and then performing grading elimination; optimizing a heterogeneous computing pipeline; rendering the execution process; the system comprises a dynamic LOD and visual cone rejection collaborative optimization module, an LSTM visual cone prediction module, a block-level mixed rejection module, a heterogeneous calculation pipeline module, an optimization CPU-GPU task allocation and data transmission module and a rendering execution control module. According to the method, the technologies of collaborative optimization of dynamic LOD and view cone removal, block-level mixed removal, heterogeneous calculation assembly line optimization and the like are adopted, so that unnecessary rendering calculation and resource loading are reduced, and the rendering efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of scene rendering technology, and in particular to a method and system for optimizing 3D scenes using dynamic LOD based on spatiotemporal prediction and view frustum culling co-rendering. Background Technology

[0002] With the rapid development of 3D graphics technology, real-time rendering of large-scale 3D scenes has become a core requirement in fields such as digital content creation and virtual simulation. Current mainstream 3D scene rendering systems mainly rely on Level of Detail (LOD) and frustum culling techniques to optimize GPU load and improve frame rates, but significant bottlenecks still exist in dynamic scene processing, hardware resource scheduling, and predictive optimization.

[0003] 1. The problem of hardware resource scheduling mismatch: Traditional rendering systems have serious deficiencies in CPU and GPU task allocation strategies, failing to fully consider the dynamic characteristics of the scene. This leads to frequent blocking of the rendering thread by the resource loading thread, causing frame rate fluctuations. The PCIe bus is blocked due to frequent transmission of LOD model data. The CPU and GPU computing loads are unbalanced, failing to fully leverage the advantages of heterogeneous computing. Existing management strategies are static and cannot adapt to the needs of dynamic scenes.

[0004] 2. The limitations of traditional frustum culling techniques: By employing a frame-by-frame full-scale calculation of occlusion relationships, traditional methods fail to utilize the spatiotemporal continuity of camera motion, resulting in significant unnecessary computational overhead. Experimental data shows that in typical urban scenes, approximately 35%–45% of the culling calculations using traditional methods are redundant operations. Furthermore, the inability to preload potentially visible objects often leads to objects "suddenly popping out" when the camera moves rapidly, severely impacting the user experience. This problem is particularly pronounced in VR environments, exacerbating user motion sickness.

[0005] 3. Deficiencies in the LOD switching mechanism and improper handling of high-frequency visible objects: High-frequency visible objects such as buildings along main urban roads can easily cause a sudden drop in frame rate when high-precision models are suddenly loaded (typically resulting in a frame time peak of 20-30ms). Waste of resources for low-frequency visible objects: Low-frequency visible objects such as distant mountains occupy high-precision video memory resources for a long time, leading to a decrease in video memory utilization. Static thresholding issues: Using a fixed distance threshold for LOD switching cannot adapt to different scene characteristics and hardware configurations.

[0006] Therefore, there is an urgent need for a 3D scene optimization method and system based on spatiotemporal prediction, dynamic LOD and view frustum culling co-rendering to solve the above problems. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing technologies in 3D scene rendering, such as sudden drops in frame rate due to LOD switching, redundant frustum culling calculations, and insufficient hardware resource utilization. The proposed method and system for 3D scene optimization based on spatiotemporal prediction-driven dynamic LOD and frustum culling co-rendering achieves improved rendering frame rate, reduced video memory usage, and elimination of sudden stuttering by using a spatiotemporal prediction-driven adaptive loading strategy, a block-level hybrid culling architecture, and heterogeneous computing pipeline optimization.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: A 3D scene optimization method based on spatiotemporal prediction-driven dynamic LOD and view frustum culling co-rendering includes: S1: Dynamic LOD and frustum culling co-optimization: Dynamically calculate the visibility frequency of objects using historical data and current frame information, and then dynamically adjust the loading and rendering methods of objects with different priorities based on the visibility frequency of objects and system memory usage. S2: LSTM-based frustum prediction: First, data preprocessing is performed, then the LSTM network architecture is designed according to the specific application scenario and prediction requirements, the LSTM network architecture is trained and optimized, and finally the LSTM network architecture is used to predict the frustum range of future multiple frames. S3: Block-level hybrid culling: First, divide the scene into uniform grid blocks (e.g., 10m×10m×10m), and then perform hierarchical culling; S4: Heterogeneous Computing Pipeline Optimization S41: In terms of GPU computing optimization, Compute Shader is used to perform frustum culling and LOD switching operations. Compute Shader has high flexibility and parallel computing capabilities, which can efficiently handle large-scale culling and LOD switching tasks. At the same time, Wave Intrinsics technology is used to control the execution flow of threads more finely, further optimize parallel efficiency, reduce synchronization overhead between threads, and improve the utilization of GPU resources. S42: In terms of CPU multi-threaded resource loading management, a shared memory pool is used to reduce data transfer. The shared memory pool manages model resources uniformly and adopts the LRU (Least Recently Used) cache replacement strategy to ensure that frequently used model resources can be accessed quickly. At the same time, atomic operations are used to ensure thread safety and avoid data inconsistency when multiple threads access the shared memory pool at the same time. S5: Rendering Execution Process: The rendering process is executed sequentially according to steps S2-S4.

[0009] As a further technical solution of the present invention, S1 specifically includes: S11: Visibility Probability Statistics: The visibility probability is calculated using the following formula: , in: For visibility probability, This represents the total number of visible frames of an object in the history of frames. Indicates the total number of frames. It is the visibility frequency calculated from the previous frame. It is the attenuation coefficient, the attenuation coefficient Its function is to control the degree of influence of historical data, when When the data is large, historical data has a significant impact on the current visibility frequency, making changes in the visibility frequency smoother and avoiding frequency abrupt changes due to anomalies in a single frame; when When the current frame is smaller, the information in the current frame has a greater impact on the visibility frequency, enabling a faster response to dynamic changes in the scene. In practical implementation, it is necessary to determine the appropriate frequency through experimentation and adjustment based on different scene characteristics and requirements. value; S12: Hierarchical LOD loading strategy: For high-frequency visible objects ( >0.7), implement a preloading strategy, loading high-precision LOD models 1-2 frames in advance and ensuring that the models are not replaced in video memory, while giving them the highest rendering priority. This ensures that high-frequency visible objects can be rendered quickly, reducing loading latency and improving rendering smoothness; for mid-frequency objects (0.3≤... ≤0.7), a background thread is used for on-demand loading, automatically adjusting the cache size based on memory pressure. When memory is tight, the number of cached models is reduced appropriately, and when memory is sufficient, the cache size is increased to improve model loading speed. Medium rendering priority ensures that mid-frequency objects are handled appropriately during rendering, preventing stuttering due to loading delays and avoiding excessive VRAM consumption. For low-frequency objects (…),… < 0.3), using the lowest detail level, and loading last when memory is tight. It can be preempted by high-priority tasks. When system memory usage > 85%, adaptive adjustment is triggered, automatically downgrading the memory level. ,in: For low-frequency objects, the loading coefficient is... This is to monitor memory usage. In this way, high-priority objects can be guaranteed to receive video memory resources first, and the system will automatically degrade under video memory pressure to avoid performance crashes caused by video memory overflow. In terms of memory management, it is necessary to monitor the system's memory usage in real time and dynamically allocate and release video memory resources according to the loading strategy of objects with different priorities. At the same time, it is necessary to establish an effective model resource index and caching mechanism to quickly find and load the required models.

[0010] As a further technical solution of the present invention, S2 specifically includes: S21: Data Preprocessing: Data preprocessing is a crucial step in the LSTM frustum prediction module, directly impacting the model's training performance and prediction accuracy. Inputting 5-10 consecutive frames of camera 6DOF parameters (position x / y / z + rotation pitch / yaw / roll) provides spatiotemporal continuity information about camera motion, enabling the model to learn the patterns of camera motion. First, batch processing of data improves training efficiency; the batch size is 64 samples. Simultaneously, batch normalization maps data of different dimensions and ranges to a unified interval, reducing the model's sensitivity to initial parameters and improving its generalization ability. The normalization formula is: , in: This is the average of the past 30 frames. The standard deviation of the past 30 frames. The original camera 6DOF parameters (position x / y / z + rotation pitch / yaw / roll) have a dimension of 6. Normalized data makes it easier for the model to learn the features in the data. Normalization can not only improve the training speed of the model, but also prevent gradient vanishing or exploding problems caused by the data range being too large or too small. In practical applications, data preprocessing also needs to consider the noise and outliers in the data. Finally, filtering algorithms are used to smooth the camera parameters and remove noise interference. For outliers, interpolation or replacement methods are used to ensure the quality of the input data. S22: LSTM network architecture: The input layer contains 6-dimensional camera parameters with a batch size of 64, which can fully capture information about camera motion; the hidden layer uses 2 layers of LSTM with 64 neurons each. This architecture can capture the complex spatiotemporal features of camera motion. The Dropout=0.2 setting can effectively prevent overfitting. By randomly dropping some neurons, the model focuses more on important features during the learning process, improving the model's generalization ability; the output layer contains the coordinates of the 8 vertices of the view frustum in the next 3 frames (24-dimensional vector), which can accurately predict the future range of the view frustum of the camera. S23: Training Optimization: Regarding training optimization, the loss function is: , in: The segmentation region predicted by the model; For the true segmentation label, this part of the loss function measures the difference between the predicted result and the true result, and improves the accuracy of the prediction by maximizing the intersection-over-union ratio (IoU); This is a regularization term that controls the strength of regularization, prevents the model from overfitting, and ensures that the model maintains good performance on both training and testing data. S24: Prediction process algorithm steps: S241: Inputs and Outputs: Input: A sequence of 5-10 consecutive camera 6DOF parameters (sliding window) (Length = 10 frames) Output: The next 3 frames The coordinates of the 8 vertices of the view frustum , Each of them Contains 8 vertices coordinate; S242: Execution process: Data preprocessing: Normalize the input sequence frame by frame (using the past 30 frames). Outlier handling: Linear interpolation is used to replace abrupt data; Smoothing filtering: Kalman filtering is applied to the camera trajectory; LSTM inference: Input dimensions: (Time step) feature) LSTM layer calculation (simplified formula): (The Gate of Oblivion) (Input Gate) (Output Gate) (Candidate Memory) (New Memories) (Hidden state) in: This is the hidden state vector from the previous time step; This is the memory state vector from the previous time step; The concatenated vector is (64+6). , , , These are the forget gate weight, input gate weight, output gate weight, and candidate memory weight, respectively. , , , It is the bias vector; For the Sigmoid activation function: ; The hyperbolic tangent activation function is: ; This is element-wise multiplication; The output of the Forgotten Gate determines how many old memories are retained. ; The input gate output determines how many new candidate memories are received. ; The output gate determines how much memory is exposed from the hidden state; Candidate memory states; The updated memory state is: New memory = Forgotten old memory + Remembered new information; Output the current hidden state; Output parsing: The 24-dimensional output vector is split into 3 groups of 8-vertex coordinates and mapped to the world coordinate system (which needs to be combined with the camera projection matrix).

[0011] As a further technical solution of the present invention, in S3, each mesh block includes a static object list, a dynamic object list, and occlusion metadata; the static object list records relevant information of all static objects in the mesh block, such as model ID, position, rotation, etc.; the dynamic object list updates the state of dynamic objects in the mesh block in real time; occlusion metadata is the key to improving culling efficiency.

[0012] As a further technical solution of the present invention, dividing the scene into uniform grid blocks specifically includes: S311: For static scenes, load the pre-baked Occlusion Map and optimize the potential visibility set using PVS (potential visible set); the Occlusion Map records the occlusion relationships of different regions in the scene. By querying the Occlusion Map, it is possible to quickly determine whether certain objects are occluded, thereby avoiding unnecessary rendering calculations; PVS further optimizes the visibility judgment, retaining only potentially visible objects and reducing the size of the rendering queue. S312: In terms of dynamic object management, after dividing the scene into grid blocks, precise culling is only performed on blocks in the direction of camera movement; for blocks in the direction of non-movement, a simpler culling strategy or reduced culling precision is adopted to reduce computational overhead; at the same time, the position and state of dynamic objects are updated in real time to ensure the accuracy of occlusion metadata.

[0013] As a further technical solution of the present invention, the graded rejection specifically includes: S321: Coarse Removal: Based on the fast intersection test of block AABB, the removal process first performs a coarse removal stage, which performs a fast intersection test based on the block AABB (axis-aligned bounding box). Parallel processing is adopted, with each thread processing a certain number of blocks to improve processing efficiency. Through coarse removal, potentially visible blocks can be quickly filtered out and a list of visible blocks can be generated. S322: Fine Culling: Entering the fine culling stage, an object-by-object occlusion query is performed on objects within the visible block. HiZ occlusion culling technology is adopted, and GPU Compute Shader is used for accelerated processing. The Compute Shader makes full use of the parallel computing power of the GPU to perform efficient occlusion judgment for each object, and finally generates the final list of visible objects.

[0014] As a further technical solution of the present invention, in the hierarchical elimination process: if the camera movement speed is greater than the set threshold, the block elimination calculation in the non-movement direction is skipped, the LOD level of these blocks is reduced, and they are marked as potentially visible, so as to reduce unnecessary computational overhead.

[0015] As a further technical solution of the present invention, in S4, the CPU and GPU share a memory area through the PCIe bus, which avoids frequent data copying operations, further reduces communication overhead, and improves the overall performance of the system.

[0016] As a further technical solution of the present invention, S5 specifically includes: S51: In the prediction phase, the LSTM network is called for each frame, and the camera 6DOF parameters of multiple consecutive frames are input. After the network inference, the frustum range of the next 3 frames is output. This prediction result provides an important basis for subsequent resource preloading. S52: The resource preloading stage is based on the prediction results and the visibility frequency of objects ( Asynchronous loading of LOD models with high visibility probability can reduce loading time during rendering and avoid sudden stuttering by preloading high-precision models; S53: In the culling and rendering stage, the CPU first performs coarse culling, quickly filtering blocks in the scene and outputting a list of visible blocks; then, the GPU performs HiZ fine culling, performing object-by-object occlusion queries on objects within the visible blocks to generate the final rendering queue; for low-frequency objects ( <0.3) Enables GPU Instancing batch rendering technology, which merges multiple identical object instances into a single rendering batch, reducing the number of rendering calls and improving rendering efficiency.

[0017] A 3D scene optimization system based on spatiotemporal prediction and dynamic LOD co-rendering with frustum culling is used to implement a 3D scene optimization method based on spatiotemporal prediction and dynamic LOD co-rendering with frustum culling, including: Dynamic LOD and frustum culling collaborative optimization module: realizes visibility probability statistics and hierarchical LOD loading; LSTM frustum prediction module: Predicts the frustum range for multiple future frames; Block-level hybrid rejection module: realizes coarse and fine rejection optimization; Heterogeneous computing pipeline module: Optimizes CPU-GPU task allocation and data transfer; Rendering execution control module: coordinates the timing and data flow of each module.

[0018] The beneficial effects of this invention are as follows: 1. By dynamically adjusting the LOD loading priority (preloading of high-frequency objects) and block-level culling, the problem of sudden lag was solved.

[0019] 2. At the same time, the prediction accuracy is significantly improved by using a lightweight LSTM to predict the frustum range of the next 3 frames and introducing a volume constraint loss function.

[0020] 3. Simply combining LOD and culling techniques can lead to memory overflow. The shared memory pool design of this invention can effectively reduce the communication overhead between the CPU and GPU. Attached Figure Description

[0021] Figure 1 This is a diagram of the overall system architecture of the present invention; Figure 2 This is a flowchart illustrating the collaborative process of dynamic LOD and view frustum culling in this invention. Figure 3 This is a schematic diagram of the LSTM network architecture of the present invention. Detailed Implementation

[0022] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0023] Please see Figures 1-3 A method for optimizing 3D scenes based on spatiotemporal prediction-driven dynamic LOD and view frustum culling co-rendering includes: S1: Dynamic LOD and frustum culling co-optimization: Dynamically calculate the visibility frequency of objects using historical data and current frame information, and then dynamically adjust the loading and rendering methods of objects with different priorities based on the visibility frequency of objects and system memory usage. S11: Visibility Probability Statistics: The visibility probability is calculated using the following formula: , in: For visibility probability, This represents the total number of visible frames of an object in the history of frames. Indicates the total number of frames. It is the visibility frequency calculated from the previous frame. It is the attenuation coefficient, the attenuation coefficient Its function is to control the degree of influence of historical data, when When the data is large, historical data has a significant impact on the current visibility frequency, making changes in the visibility frequency smoother and avoiding frequency abrupt changes due to anomalies in a single frame; when When the current frame is smaller, the information in the current frame has a greater impact on the visibility frequency, enabling a faster response to dynamic changes in the scene. In practical implementation, it is necessary to determine the appropriate frequency through experimentation and adjustment based on different scene characteristics and requirements. value; S12: Hierarchical LOD loading strategy: For high-frequency visible objects ( >0.7), implement a preloading strategy, loading high-precision LOD models 1-2 frames in advance and ensuring that the models are not replaced in video memory, while giving them the highest rendering priority. This ensures that high-frequency visible objects can be rendered quickly, reducing loading latency and improving rendering smoothness; for mid-frequency objects (0.3≤... ≤0.7), a background thread is used for on-demand loading, automatically adjusting the cache size based on memory pressure. When memory is tight, the number of cached models is reduced appropriately, and when memory is sufficient, the cache size is increased to improve model loading speed. Medium rendering priority ensures that mid-frequency objects are handled appropriately during rendering, preventing stuttering due to loading delays and avoiding excessive VRAM consumption. For low-frequency objects (…),… < 0.3), using the lowest detail level, and loading last when memory is tight. It can be preempted by high-priority tasks. When system memory usage > 85%, adaptive adjustment is triggered, automatically downgrading the memory level. ,in: For low-frequency objects, the loading coefficient is... This is to monitor memory usage. In this way, high-priority objects can be guaranteed to receive video memory resources first, and the system will automatically degrade under video memory pressure to avoid performance crashes caused by video memory overflow. In terms of memory management, it is necessary to monitor the system's memory usage in real time and dynamically allocate and release video memory resources according to the loading strategy of objects with different priorities. At the same time, it is necessary to establish an effective model resource index and caching mechanism to quickly find and load the required models.

[0024] S2: LSTM-based frustum prediction: First, data preprocessing is performed, then the LSTM network architecture is designed according to the specific application scenario and prediction requirements, the LSTM network architecture is trained and optimized, and finally the LSTM network architecture is used to predict the frustum range of future multiple frames. S21: Data Preprocessing: Data preprocessing is a crucial step in the LSTM frustum prediction module, directly impacting the model's training performance and prediction accuracy. Inputting 5-10 consecutive frames of camera 6DOF parameters (position x / y / z + rotation pitch / yaw / roll) provides spatiotemporal continuity information about camera motion, enabling the model to learn the patterns of camera motion. First, batch processing of data improves training efficiency; the batch size is 64 samples. Simultaneously, batch normalization maps data of different dimensions and ranges to a unified interval, reducing the model's sensitivity to initial parameters and improving its generalization ability. The normalization formula is: , in: This is the average of the past 30 frames. The standard deviation of the past 30 frames. The original camera 6DOF parameters (position x / y / z + rotation pitch / yaw / roll) have a dimension of 6. Normalized data makes it easier for the model to learn the features in the data. Normalization can not only improve the training speed of the model, but also prevent gradient vanishing or exploding problems caused by the data range being too large or too small. In practical applications, data preprocessing also needs to consider the noise and outliers in the data. Finally, filtering algorithms are used to smooth the camera parameters and remove noise interference. For outliers, interpolation or replacement methods are used to ensure the quality of the input data. S22: LSTM network architecture: The input layer contains 6-dimensional camera parameters with a batch size of 64, which can fully capture information about camera motion; the hidden layer uses 2 layers of LSTM with 64 neurons each. This architecture can capture the complex spatiotemporal features of camera motion. The Dropout=0.2 setting can effectively prevent overfitting. By randomly dropping some neurons, the model focuses more on important features during the learning process, improving the model's generalization ability; the output layer contains the coordinates of the 8 vertices of the view frustum in the next 3 frames (24-dimensional vector), which can accurately predict the future range of the view frustum of the camera. S23: Training Optimization: Regarding training optimization, the loss function is: , in: The segmentation region predicted by the model; For the true segmentation label, this part of the loss function measures the difference between the predicted result and the true result, and improves the accuracy of the prediction by maximizing the intersection-over-union ratio (IoU); This is a regularization term that controls the strength of regularization, prevents the model from overfitting, and ensures that the model maintains good performance on both training and testing data. S24: Prediction process algorithm steps: S241: Inputs and Outputs: Input: A sequence of 5-10 consecutive camera 6DOF parameters (sliding window) (Length = 10 frames) Output: The next 3 frames The coordinates of the 8 vertices of the view frustum , Each of them Contains 8 vertices coordinate; S242: Execution process: Data preprocessing: Normalize the input sequence frame by frame (using the past 30 frames). Outlier handling: Linear interpolation is used to replace abrupt data; Smoothing filtering: Kalman filtering is applied to the camera trajectory; LSTM inference: Input dimensions: (Time step) feature)

[0025] LSTM layer calculation (simplified formula): (The Gate of Oblivion) (Input Gate) (Output Gate) (Candidate Memory) (New Memories) (Hidden state) in: This is the hidden state vector from the previous time step; This is the memory state vector from the previous time step; The concatenated vector is (64+6). , , , These are the forget gate weight, input gate weight, output gate weight, and candidate memory weight, respectively. , , , It is the bias vector; For the Sigmoid activation function: ; The hyperbolic tangent activation function is: ; This is element-wise multiplication; The output of the Forgotten Gate determines how many old memories are retained. ; The input gate output determines how many new candidate memories are received. ; The output gate determines how much memory is exposed from the hidden state; Candidate memory states; The updated memory state is: New memory = Forgotten old memory + Remembered new information; Output the current hidden state; Output parsing: The 24-dimensional output vector is split into 3 groups of 8-vertex coordinates and mapped to the world coordinate system (which needs to be combined with the camera projection matrix).

[0026] S3: Block-level hybrid culling: First, divide the scene into uniform grid blocks (e.g., 10m×10m×10m), and then perform hierarchical culling; Each mesh tile contains a list of static objects, a list of dynamic objects, and occlusion metadata. The list of static objects records relevant information about all static objects within the mesh tile, such as model ID, position, and rotation. The list of dynamic objects updates the state of dynamic objects within the mesh tile in real time. Occlusion metadata is key to improving culling efficiency. Dividing the scene into uniform grid blocks specifically includes: S311: For static scenes, load the pre-baked Occlusion Map and optimize the potential visibility set using PVS (potential visible set); the Occlusion Map records the occlusion relationships of different regions in the scene. By querying the Occlusion Map, it is possible to quickly determine whether certain objects are occluded, thereby avoiding unnecessary rendering calculations; PVS further optimizes the visibility judgment, retaining only potentially visible objects and reducing the size of the rendering queue. a. Load the pre-baked Occlusion Map Spatial mesh generation: The entire 3D scene is divided into uniform mesh blocks, such as a cube mesh with a side length of 10m. The size of the mesh blocks can be adjusted according to the complexity and requirements of the actual scene. Assuming the size of the scene is L×W×H (length×width×height), then the number of mesh blocks after division is (L / 10)×(W / 10)×(H / 10). Each mesh block has a unique identifier for subsequent querying and management. Occlusion Map Generation: Through pre-calculation, the occlusion relationships between each grid block and other grid blocks are obtained, and these occlusion relationships are stored in the Occlusion Map. The Occlusion Map can be represented as a three-dimensional array, where Occlusion_Map[i][j][k] represents whether grid block (i,j,k) is occluded by other grid blocks. The specific calculation method can use occlusion calculation algorithms such as ray casting or depth buffering. Taking ray casting as an example, rays are emitted from the center of grid block (i,j,k) in all directions, and the intersection of the rays with objects in the scene is detected. If the ray intersects with an object before reaching the camera or screen, the grid block is considered to be occluded. The mathematical expression is: For grid block (i,j,k), the direction of the emitted ray r is d, the origin is p0, and the intersection point t between the ray and the object in the scene is calculated. If there is an intersection point and t <t max (t) max If the maximum propagation distance of light is given, then Occlusion_Map[i][j][k] = 1 (indicating occlusion), otherwise it is 0; b. Optimize the potential visible set using PVS PVS Generation: Based on the Occlusion Map, a Potential Visible Set (PVS) is generated for each grid cell. PVS refers to the set of all grid cells that can be seen starting from the current grid cell. The calculation of PVS can be done recursively or iteratively. Taking the recursive method as an example, starting from the current grid cell, visible grid cells are recursively searched according to the occlusion relationships in the Occlusion Map. The specific algorithm is as follows: Input: Current grid block (i,j,k); Output: PVS[i][j][k] (the potential visible set of the current grid tile, which contains the set of visible grid tiles); Initialize PVS[i][j][k] as an empty set; Add the adjacent grid blocks of the current grid block to the candidate queue; While the candidate queue is not empty: Take out a grid block (x, y, z) from the queue; If (x,y,z) is not in PVS[i][j][k] and Occlusion_Map[x][y][z]=0 (not occluded): add (x,y,z) to PVS[i][j][k]; add the adjacent grid blocks of (x,y,z) to the candidate queue; The definition of adjacent grid blocks can be determined based on the connection relationship of the actual scene, and usually includes grid blocks in six directions: up, down, left, right, front, and back.

[0027] Visibility determination: During the rendering process, based on the mesh block where the camera is located, its PVS (Visible Frames View) is queried to quickly determine visible mesh blocks and reduce rendering calculations; the specific calculation steps are as follows: Assuming the camera is located in grid block (cx, cy, cz), the set of visible grid blocks V = PVS[cx][cy][cz]. For each visible grid block (vx, vy, vz) ∈ V, it is further determined whether the object within the grid block is visible.

[0028] S312: In terms of dynamic object management, after dividing the scene into grid blocks, precise culling is only performed on blocks in the direction of camera movement; for blocks in the direction of non-movement, a simpler culling strategy or reduced culling precision is adopted to reduce computational overhead; at the same time, the position and state of dynamic objects are updated in real time to ensure the accuracy of occlusion metadata. The specific steps of the tiered elimination process include: S321: Coarse Removal: Based on the fast intersection test of block AABB, the removal process first performs a coarse removal stage, which performs a fast intersection test based on the block AABB (axis-aligned bounding box). Parallel processing is adopted, with each thread processing a certain number of blocks to improve processing efficiency. Through coarse removal, potentially visible blocks can be quickly filtered out and a list of visible blocks can be generated. Parameter definition: The AABB of a block: The AABB of each block is defined by its minimum and maximum coordinates, denoted as... ; Camera frustum: The camera frustum is defined by its six planes (left, right, top, bottom, near, and far), each of which can be represented by a plane equation. Indicates the intersection test result: a Boolean value indicating whether the AABB intersects with the view frustum.

[0029] Fast intersection test algorithm: For each block's AABB, the following steps are performed to determine whether it intersects with the camera's view frustum: Obtain the equations of the six planes of the camera's view frustum: each plane equation is expressed as ,in It is the normal vector of the plane. It is the distance from the plane to the origin; For each plane: Calculate the distances of the eight vertices of the AABB on that plane. If all vertices are on the same side of the plane (all distances are either positive or all are negative), then the AABB is completely outside the plane and does not intersect the view frustum. Return the non-intersection condition. Specifically: Let the eight vertices of the AABB be... The coordinates of each vertex are For a plane The distance from the vertex to the plane is calculated as follows: If for a certain plane If all the distances from the vertices are either positive or all are negative, then AABB does not intersect the plane. If AABB intersects all six planes, then return "intersection," indicating that the block may be visible.

[0030] Parallelization processing: To efficiently process multiple blocks, a parallel approach is adopted: Thread allocation: All blocks are allocated to multiple threads, and each thread processes a certain number of blocks; Thread safety: Use thread-safe data structures (such as locks or atomic operations) to manage shared resources, such as a list of visible blocks; Task scheduling: Dynamically adjust the number of blocks for each thread to balance the load.

[0031] S322: Fine Culling: Entering the fine culling stage, an object-by-object occlusion query is performed on objects within the visible block. HiZ occlusion culling technology is adopted, and GPU Compute Shader is used for accelerated processing. The Compute Shader makes full use of the parallel computing power of the GPU to perform efficient occlusion judgment on each object, and finally generates the final list of visible objects. Parameter definition: HiZ buffer: The HiZ buffer is a hierarchical depth buffer, usually divided into multiple levels. The resolution of each level is half that of the previous level, with the top level having the lowest resolution and the lowest level having the highest resolution. Depth value: The depth value of each pixel represents the straight-line distance from the camera to that pixel; object's : Axis-aligned bounding box for each object , represented as ; HiZ Occlusion Removal Algorithm: HiZ occlusion culling quickly determines whether an object is occluded by using a hierarchical depth buffer: Constructing the HiZ buffer: The HiZ buffer is constructed from the depth buffer (Z-Buffer). Each level generates the depth value of the previous level by taking the maximum depth value of its four adjacent pixels. Let the resolution of the depth buffer be... The number of levels in the HiZ buffer is The resolution of each layer is as follows: down to the lowest level; occlusion query: for each object Calculate the object's projection rectangle in screen space; query the maximum depth value within this rectangle in the HiZ buffer; if the object's nearest depth value is greater than the queryed maximum depth value, the object is occluded and can be culled; specifically: let the object's AABB projection rectangle in screen space be Rect Query HiZ buffer level The deepest depth value in If the object's nearest depth value satisfy: If an object is obscured, it can be removed.

[0032] GPU Compute Shader acceleration: Parallelized occlusion lookup using GPU Compute Shader: Thread allocation: Assign each object to a thread or a group of threads; each thread is responsible for calculating the projection of an object's AABB onto the screen space and querying the HiZ buffer. Parallel query: Utilizes the parallel computing capabilities of Compute Shader to process occlusion queries for multiple objects simultaneously; calculates the intersection of the AABB and HiZ buffer for each object to quickly determine its visibility.

[0033] During the hierarchical elimination process, if the camera's movement speed exceeds the set threshold, the elimination calculation for blocks in non-movement directions is skipped, the LOD level of these blocks is reduced, and they are marked as potentially visible to reduce unnecessary computational overhead.

[0034] S4: Heterogeneous Computing Pipeline Optimization S41: In terms of GPU computing optimization, Compute Shader is used to perform frustum culling and LOD switching operations. Compute Shader has high flexibility and parallel computing capabilities, which can efficiently handle large-scale culling and LOD switching tasks. At the same time, Wave Intrinsics technology is used to control the execution flow of threads more finely, further optimize parallel efficiency, reduce synchronization overhead between threads, and improve the utilization of GPU resources. S42: In terms of CPU multi-threaded resource loading management, a shared memory pool is used to reduce data transfer. The shared memory pool manages model resources uniformly and adopts the LRU (Least Recently Used) cache replacement strategy to ensure that frequently used model resources can be accessed quickly. At the same time, atomic operations are used to ensure thread safety and avoid data inconsistency when multiple threads access the shared memory pool at the same time. The CPU and GPU share a memory area via the PCIe bus, avoiding frequent data copying operations, further reducing communication overhead, and improving the overall system performance.

[0035] S5: Rendering Execution Process: The rendering process is executed sequentially according to steps S2-S4; S51: In the prediction phase, the LSTM network is called for each frame, and the camera 6DOF parameters of multiple consecutive frames are input. After the network inference, the frustum range of the next 3 frames is output. This prediction result provides an important basis for subsequent resource preloading. S52: The resource preloading stage is based on the prediction results and the visibility frequency of objects ( Asynchronous loading of LOD models with high visibility probability can reduce loading time during rendering and avoid sudden stuttering by preloading high-precision models; S53: In the culling and rendering stage, the CPU first performs coarse culling, quickly filtering blocks in the scene and outputting a list of visible blocks; then, the GPU performs HiZ fine culling, performing object-by-object occlusion queries on objects within the visible blocks to generate the final rendering queue; for low-frequency objects ( <0.3) Enables GPU Instancing batch rendering technology, which merges multiple identical object instances into a single rendering batch, reducing the number of rendering calls and improving rendering efficiency.

[0036] A 3D scene optimization system based on spatiotemporal prediction and dynamic LOD co-rendering with frustum culling is used to implement a 3D scene optimization method based on spatiotemporal prediction and dynamic LOD co-rendering with frustum culling, including: Dynamic LOD and frustum culling collaborative optimization module: realizes visibility probability statistics and hierarchical LOD loading; LSTM frustum prediction module: Predicts the frustum range for multiple future frames; Block-level hybrid rejection module: realizes coarse and fine rejection optimization; Heterogeneous computing pipeline module: Optimizes CPU-GPU task allocation and data transfer; Rendering execution control module: coordinates the timing and data flow of each module.

[0037] Example 1 I. Initialization Phase a. Scene preprocessing: Divide the space into a grid (10m×10m×10m), generate the Occlusion Map of static objects, and pre-calculate the LOD level of each object.

[0038] b. Model loading: Establish a shared memory pool, initialize the LRU cache, and load the initial visible region model.

[0039] c. Prediction model initialization: Load the pre-trained LSTM model, initialize the camera trajectory buffer, and calibrate the normalization parameters.

[0040] II. Operational Phase a. Camera data acquisition: Obtain camera parameters for the current frame, update the historical trajectory queue, and perform data normalization.

[0041] b. View frustum prediction: Call LSTM inference, parse the output vertex coordinates, and construct the predicted view frustum.

[0042] c. Visibility Analysis: Calculate the visibility frequency of each object, update the LOD priority level, and adjust the video memory allocation strategy.

[0043] d. Resource preloading: Schedule asynchronous loading tasks, manage memory pool allocation, and handle loading completion events.

[0044] e. Culling calculation: CPU coarse culling (block level), GPU fine culling (object level), generating rendering queue.

[0045] f. Rendering execution.

[0046] Terminology Explanation: LOD (Level of Detail): A technique for dynamically adjusting the model's accuracy based on the distance between the object and the camera.

[0047] Frustum culling: The rendering process that removes objects that are not visible outside the camera's view frustum.

[0048] GPU Instancing: A GPU optimization technique for batch rendering of identical meshes.

[0049] Spatiotemporal prediction: refers to the technology of predicting the range of the view frustum in the next N frames by combining the motion trajectory from a historical perspective (time dimension) with the spatial structure features of the scene.

[0050] Heterogeneous computing pipeline: Maximizes hardware resource utilization by interleaving the clock cycles of CPU instruction sets (SSE / AVX) and GPU computing units (CU Core / StreamProcessor).

[0051] As can be seen from the above description, the above embodiments of the present invention achieve the following technical effects: by dynamically adjusting the LOD loading priority (preloading of high-frequency objects) and block-level culling, the problem of sudden lag is solved.

[0052] Meanwhile, the prediction accuracy is significantly improved by using a lightweight LSTM to predict the frustum range of the next 3 frames and introducing a volume constraint loss function.

[0053] Simply combining LOD and culling techniques can lead to memory overflow. The shared memory pool design of this invention can effectively reduce the communication overhead between the CPU and GPU.

[0054] Overall performance improvement: Techniques such as the coordinated optimization of dynamic LOD and view frustum culling, block-level hybrid culling, and heterogeneous computing pipeline optimization reduce unnecessary rendering calculations and resource loading, thereby improving rendering efficiency.

[0055] Regarding video memory usage, this invention reduces memory usage by more than 20% through block-level culling and predictive loading. Block-level culling reduces the number of objects that need to be rendered, while predictive loading avoids unnecessary model loading, thereby reducing video memory usage.

[0056] In terms of memory usage, the shared memory pool design and LRU cache replacement strategy effectively manage model resources and reduce memory waste.

[0057] Regarding loading latency, this invention reduces sudden lag by using dynamic LOD loading priority and resource preloading strategies, resulting in a significant reduction in loading latency.

[0058] Especially when the camera is moving quickly, this invention can respond quickly, ensuring smooth rendering of the scene and avoiding the phenomenon of objects "suddenly popping out" in traditional methods.

[0059] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention is limited to these examples; within the framework of the invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity.

[0060] This invention is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this specification. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A 3D scene optimization method based on spatiotemporal prediction-driven dynamic LOD and view frustum culling co-rendering, characterized in that, include: S1: Dynamic LOD and frustum culling co-optimization: Dynamically calculate the visibility frequency of objects using historical data and current frame information, and then dynamically adjust the loading and rendering methods of objects with different priorities based on the visibility frequency of objects and system memory usage. S2: LSTM-based frustum prediction: First, data preprocessing is performed, then the LSTM network architecture is designed according to the specific application scenario and prediction requirements, the LSTM network architecture is trained and optimized, and finally the LSTM network architecture is used to predict the frustum range of future multiple frames. S3: Block-level hybrid culling: First, the scene is divided into uniform grid blocks, and then hierarchical culling is performed; S4: Heterogeneous Computing Pipeline Optimization S41: In terms of GPU computing optimization, Compute Shader is used to perform frustum culling and LOD switching operations; at the same time, Wave Intrinsics technology is used to more finely control the execution flow of threads; S42: In terms of CPU multi-threaded resource management and loading, model resources are managed uniformly through a shared memory pool, and an LRU cache replacement strategy is adopted; at the same time, atomic operations are used to ensure thread safety. S5: Rendering Execution Process: The rendering process is executed sequentially according to steps S2-S4.

2. The 3D scene optimization method based on spatiotemporal prediction and dynamic LOD and view frustum culling co-rendering according to claim 1, characterized in that, S1 specifically includes: S11: Visibility Probability Statistics: The visibility probability is calculated using the following formula: ， in: For visibility probability, This represents the total number of visible frames of an object in the history of frames. Indicates the total number of frames. It is the visibility frequency calculated from the previous frame. It is the attenuation coefficient; S12: Tiered LOD Loading Strategy: For high-frequency visible objects, a preloading strategy is implemented, loading high-precision LOD models 1-2 frames in advance and ensuring that the models are not replaced in video memory, while giving them the highest rendering priority; for mid-frequency objects, a background thread is used to load them on demand, and the cache size is automatically adjusted according to memory pressure; for low-frequency objects, the lowest level of detail is used, and they are loaded last when video memory is tight, and can be preempted by high-priority tasks.

3. The 3D scene optimization method based on spatiotemporal prediction and dynamic LOD and view frustum culling co-rendering according to claim 1, characterized in that, S2 specifically includes: S21: Data Preprocessing: Input camera 6DOF parameters for 5-10 consecutive frames. First, batch process the data with a batch size of 64 samples. Simultaneously, batch normalization is used to map data of different dimensions and ranges to a unified interval. Finally, a filtering algorithm is used to smooth the camera parameters. For outliers, interpolation or replacement is used for processing. S22: LSTM network architecture: The input layer contains 6D camera parameters with a batch size of 64; the hidden layer uses 2 layers of LSTM with 64 neurons per layer; the output layer contains the coordinates of the 8 vertices of the view frustum for the next 3 frames. S23: Training Optimization: Regarding training optimization, the loss function is: ， in: The segmentation region predicted by the model; For real segmentation labels; This is a regularization term.

4. The 3D scene optimization method based on spatiotemporal prediction and dynamic LOD and view frustum culling co-rendering as described in claim 1, characterized in that, In S3, each grid block contains a static object list, a dynamic object list, and occlusion metadata; the static object list records relevant information about all static objects within the grid block; the dynamic object list updates the state of dynamic objects within the grid block in real time. Obscuring metadata is key to improving culling efficiency.

5. The 3D scene optimization method based on spatiotemporal prediction and dynamic LOD and view frustum culling co-rendering according to claim 4, characterized in that, The process of dividing the scene into uniform grid blocks specifically includes: S311: For static scenes, load the pre-baked Occlusion Map and optimize the potential visibility set using PVS; S312: In terms of dynamic object management, after dividing the scene into grid blocks, precise culling is only performed on blocks in the direction of camera movement; for blocks in the direction of non-movement, a simpler culling strategy or reduced culling precision is adopted; at the same time, the position and status of dynamic objects are updated in real time.

6. The 3D scene optimization method based on spatiotemporal prediction and dynamic LOD and view frustum culling co-rendering according to claim 5, characterized in that, The tiered rejection specifically includes: S321: Coarse culling: Based on the fast intersection test of block AABB, the coarse culling stage is first performed in the culling process. The fast intersection test is performed based on the block AABB and a parallel processing method is adopted, with each thread processing a certain number of blocks. S322: Fine Culling: Entering the fine culling stage, an object-by-object occlusion query is performed on objects within the visible block. HiZ occlusion culling technology is adopted, and GPU Compute Shader is used for accelerated processing. The Compute Shader makes full use of the parallel computing power of the GPU to perform efficient occlusion judgment for each object, and finally generates the final list of visible objects.

7. The 3D scene optimization method based on spatiotemporal prediction and dynamic LOD and view frustum culling co-rendering according to claim 6, characterized in that, During the hierarchical elimination process: if the camera's movement speed is greater than the set threshold, the elimination calculation for blocks in non-movement directions is skipped, the LOD level of these blocks is reduced, and they are marked as potentially visible.

8. The 3D scene optimization method based on spatiotemporal prediction and dynamic LOD and view frustum culling co-rendering according to claim 1, characterized in that, In S4, the CPU and GPU share a memory region via the PCIe bus.

9. The 3D scene optimization method based on spatiotemporal prediction and dynamic LOD and view frustum culling co-rendering according to claim 1, characterized in that, S5 specifically includes: S51: In the prediction phase, the LSTM network is called for each frame, and the camera 6DOF parameters of multiple consecutive frames are input. After the network inference, the frustum range of the next 3 frames is output. S52: During the resource preloading stage, based on the prediction results and the visibility frequency of objects, LOD models with high visibility probability are loaded asynchronously. S53: In the culling and rendering stage, the CPU first performs coarse culling to quickly filter blocks in the scene and output a list of visible blocks; then, the GPU performs HiZ fine culling to perform object-by-object occlusion query on objects within the visible blocks and generate the final rendering queue; for low-frequency objects, GPU Instancing batch rendering technology is enabled to merge multiple instances of the same object into a rendering batch.

10. A 3D scene optimization system based on spatiotemporal prediction-driven dynamic LOD and view frustum culling co-rendering, characterized in that, The method for optimizing a 3D scene based on spatiotemporal prediction and co-rendering of dynamic LOD and frustum culling as described in any one of claims 1-9 includes: Dynamic LOD and frustum culling collaborative optimization module: realizes visibility probability statistics and hierarchical LOD loading; LSTM frustum prediction module: Predicts the frustum range for multiple future frames; Block-level hybrid rejection module: realizes coarse and fine rejection optimization; Heterogeneous computing pipeline module: Optimizes CPU-GPU task allocation and data transfer; Rendering execution control module: coordinates the timing and data flow of each module.

Images