Hierarchical occlusion perception grid infinitesimal streaming and adaptive rendering method

Abstract

The invention relates to a hierarchical occlusion perception grid infinitesimal streaming transmission and adaptive rendering method, which comprises the following steps of: S1, decomposing a model into grid infinitesimal, recursively constructing a bounding volume hierarchical tree BVH from bottom to top by taking the grid infinitesimal as a leaf node, and setting a semantic error and a geometric error for each sub-node cluster; s2, acquiring camera parameters of a current frame, comparing a total screen space error with an error threshold value, if the error is smaller than the threshold value, stopping traversing, and adding the cluster node into a to-be-rendered list of the current frame; and S3, if the grid infinitesimal corresponding to the cluster node in the to-be-rendered list is not loaded to the GPU memory, sending a loading request to an asynchronous streaming manager, asynchronously loading the corresponding grid infinitesimal from the memory, and rendering a father node of the cluster node. Compared with the prior art, the method has the advantages that large-scale and high-precision model data can be rendered on the lightweight XR all-in-one machine in real time, and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of virtual reality rendering, and in particular to a hierarchical occlusion-aware grid cell streaming and adaptive rendering method. BACKGROUND

[0002] With the increasing application of digital twin, high-precision scanning and movie-level art assets in the field of extended reality (XR), the geometric complexity of three-dimensional models is experiencing explosive growth, with the number of polygons in a single scene or model easily reaching tens of millions or even billions. However, XR devices, especially all-in-one machines with mobile chips as the core, have very limited computing and storage capabilities. This contradiction between "infinite assets" and "limited hardware" constitutes the core bottleneck of current XR content development. Existing solutions have significant defects in dealing with this challenge. Traditional LOD (Level of Detail) technology requires developers to manually create or algorithmically generate several discrete versions of different precision for the model. This process not only consumes a large amount of art manpower and time, but also has limited LOD levels, often accompanied by obvious "popping" phenomenon when switching, which destroys the sense of immersion and cannot effectively handle models composed of a single, large number of polygons. On the other hand, techniques such as dynamic resolution scaling are only a "pixel" level compromise, which can alleviate GPU pressure, but cannot solve the CPU bottleneck and memory overflow problems caused by overly complex models at the "geometric" level. When the model itself cannot be completely loaded, such techniques are powerless. In addition, platform-specific optimization solutions for different GPU architectures require high professional knowledge and the results are difficult to reuse across platforms, resulting in high development and maintenance costs. Therefore, existing technologies generally lack a unified solution that can efficiently present any complex model on a power-limited XR device in an automated and non-intrusive manner. SUMMARY

[0003] The present application proposes a hierarchical occlusion-aware grid cell streaming and adaptive rendering method, which fundamentally breaks the rigid restriction in traditional rendering pipelines that "models must be completely loaded into memory" through virtualization of geometric bodies and streaming technology, enabling real-time rendering of large-scale and high-precision model data on lightweight XR all-in-one machines.

[0004] The purpose of the present application can be achieved by the following technical solutions:

[0005] A hierarchical occlusion-aware grid cell streaming and adaptive rendering method, the method comprising the following steps:

[0006] S1, obtain a high-precision source model, decompose the model into grid microelements, take the grid microelements as leaf nodes, recursively construct a bounding volume hierarchy tree BVH from bottom to top, the nodes of the hierarchy tree are cluster nodes, each parent node is a geometric simplification representation of the cluster of its child nodes, set a semantic error and a geometric error for each child cluster;

[0007] S2, obtain camera parameters of a current frame, traverse from top to bottom from a root node of the hierarchy tree, in the traversal process, perform view frustum culling and occlusion culling on each cluster node in the traversal to obtain a visible cluster, calculate a total screen space error based on the camera parameters and the error of the visible cluster, compare the total screen space error with an error threshold, if the error is less than the threshold, stop the traversal, and add the cluster node to a to-be-rendered list of the current frame; otherwise, continue the traversal;

[0008] S3, if the grid microelements corresponding to the cluster nodes in the to-be-rendered list have not been loaded into a GPU memory, send a loading request to an asynchronous stream manager, asynchronously load the corresponding grid microelements from storage and render a parent node of the cluster node, after the corresponding grid microelements are loaded, package and send to the GPU for rendering, then update a next frame to the current frame, and return to S2.

[0009] Further, the total screen space error is:

[0010] E total =(1-λ s )·E geom +λ s ·E sem

[0011] wherein E total represents the total screen space error, E geom represents the geometric error, E sem represents the semantic error, and λ s represents a semantic weight factor.

[0012] Further, the geometric error is:

[0013]

[0014] wherein ∈ is a geometric error source of the model, H is a screen height, d is a distance from a tile to a camera in the camera parameters of the current frame, and θ is a field of view angle.

[0015] Further, the error threshold is obtained based on a predicted GPU rendering time of a next frame.

[0016] Further, the predicted GPU rendering time of the next frame is:

[0017]

[0018] wherein, is the predicted GPU rendering time of the next frame, c f is the camera state vector, v f is the camera velocity vector, p f is the current rendering parameter vector, H f-k..f is the historical performance data, is a time series containing the actual GPU rendering time of the past k frames.

[0019] Further, in S3, if multiple loading requests exist simultaneously, the utility value of the grid cell corresponding to each loading request is calculated, and the loading requests are executed in descending order of the utility value.

[0020] Further, the utility value is:

[0021]

[0022] wherein, U(i) is the utility value of the i-th loading request, I(i) is the perceived importance, Pvis(i) is the predicted visibility probability of the camera parameter of the current frame, C load (i) is the loading cost, C mem (i) is the memory cost.

[0023] Further, the predicted visibility probability is obtained based on the current velocity and acceleration vectors of the camera, and the predicted visibility probability is the probability that the grid cell corresponding to the i-th loading request enters the view frustum.

[0024] Further, the grid cell only contains vertices and triangular patches whose number is less than the cell threshold.

[0025] Further, the package is sent to the GPU for rendering, specifically:

[0026] The GPU driving rendering function using a modern graphics API is packaged and sent to the GPU for rendering.

[0027] Compared with the prior art, the present application has the following beneficial effects:

[0028] The application fundamentally breaks the hard limit of "model must be completely loaded into memory" in traditional rendering pipeline by virtualizing geometry and streaming technology, loading to nodes less than a threshold through a constructed tree, and realizes a breakthrough in asset scale and scalability. This means that theoretically, any complexity model can be rendered, and the performance overhead is only related to the visible details at the pixel level, so that it is possible to render movie-level art assets, high-precision BIM models or large-scale three-dimensional scanning data in real time on a lightweight XR all-in-one machine, greatly expanding the content boundary of XR applications. In addition, the application realizes fully automated optimal LOD generation and switching. The offline preprocessing automatically completes the micro-elementization, hierarchical and error measurement of the model, completely replacing the traditional LOD scheme of tedious, time-consuming and expert experience-dependent manual creation process. The LOD selection at runtime is a continuous process based on accurate error, and its beneficial effect is not only to greatly improve the development efficiency and reduce the art cost, but also because the LOD is a pixel-by-pixel accurate match, completely eliminating the annoying "jump" phenomenon in the traditional scheme, providing a seamless and smooth visual experience. Finally, the application brings extreme rendering performance optimization. The hierarchical culling algorithm can quickly cull a large number of invisible geometry with logarithmic complexity, and the rendering mode with grid micro-element perfectly matches the parallel computing architecture of modern GPU, greatly reducing the communication overhead from CPU to GPU. The final beneficial effect is that even in the face of extremely complex scenes, the CPU and GPU load can be maintained at a very low level, thereby ensuring the high and stable frame rate on which XR applications depend, effectively avoiding delay and dizziness caused by insufficient performance.

[0029] At the same time, the performance adjustment mechanism of the engine is upgraded from "passive response" to "active prediction", which fundamentally changes the way the system responds to load changes, aiming to resolve performance bottlenecks before they occur. Traditional performance adjustment, such as pure PID controller, is essentially based on historical error (such as the rendering time of the last frame) for lag compensation. The application introduces a performance prediction model that can estimate the rendering overhead of one or more future frames based on the current and recent state. This allows the system to allocate resources in advance to avoid actual fluctuations in frame rate, thereby providing a smoother and more stable user experience. The application models the predicted GPU rendering time of the next frame as a multivariate function, and the predicted rendering time will serve as the core input of a higher-level control system (such as model predictive control, MPC), allowing the engine to not only "make up for the loss", but also to "plan ahead", making global optimal performance planning within a time window of multiple future frames.

[0030] The engine's Streaming Manager calculates a "UtilityScore" for each data block that is out of view or not yet loaded (whether it's a tile in a 3D Tiles or an independent asset). This value dynamically quantifies the ratio between the "benefits" of loading the data block and the "costs" incurred, ensuring that content that maximizes the user's perceived experience is prioritized within limited bandwidth and memory budgets. The Streaming Manager maintains a priority queue sorted by UtilityScore U(i) in real time. When system resources allow, it retrieves the highest-utility block from the top of the queue for loading. Conversely, when memory pressure is too high, it selects the currently loaded block with the lowest utility score for unloading. This mechanism ensures that the allocation of rendering resources always serves to maximize the user's visual and interactive experience.

[0031] Furthermore, suppose there are two objects of similar size in the distance of the scene: a regular rock and an NPC issuing a quest. Under traditional, purely geometric error metrics, they might both be degraded to low-polygon models at the same time. However, under the hybrid error metrics of this scheme, the NPC has extremely high semantic importance (high Equation). sem The total error E calculated from the value) total The model will be large, and the system will continue to maintain high-precision models and textures for it; while the stone with zero semantic importance will be simplified normally. In this way, the precious rendering budget is intelligently and automatically allocated to the elements most critical to the core experience. Attached Figure Description

[0032] Figure 1 Flowchart for offline preprocessing;

[0033] Figure 2 Flowchart for adaptive rendering loop;

[0034] Figure 3 This is a schematic diagram of a hierarchical cluster structure;

[0035] Figure 4 This is a schematic diagram of the control device. Detailed Implementation

[0036] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0037] This invention proposes a novel 3D model processing and rendering framework. Its core idea is to transform the traditional rendering mode, which uses polygonal meshes as units, into a streaming rendering mode, which uses clusters of tiny meshlets as units. This framework ensures that, at any given time, only the geometric details visible and necessary to the current viewpoint are transmitted and rendered through offline preprocessing and dynamic selection and compositing at runtime.

[0038] The technical solution of this invention comprises two core stages: an offline preprocessing stage during construction (as shown in the appendix). Figure 1 (as shown) and the adaptive rendering loop phase during application runtime (as shown in the appendix) Figure 2 (As shown). In the offline preprocessing workflow, the system automatically converts the high-precision source model provided by the developer into a data structure optimized for streaming rendering. This process begins with inputting one or more high-polygon 3D model files, followed by "mesh miniaturization," which decomposes the large, unordered set of triangles into regular "mesh miniature" data units containing a small number of vertices and triangles. Next, using these mesh miniatures as leaf nodes, the system recursively constructs a tree-like hierarchical cluster structure from bottom to top, such as a bounding volume hierarchy (BVH), where each parent node is a geometrically simplified representation of its child node cluster. Simultaneously with tree construction, the system meticulously calculates a key geometric error metric E for each cluster C. geom (C) quantifies the visual bias resulting from replacing the fine geometry of its child nodes with the current simplified cluster. After completing the above steps, an optimized asset package containing all mesh microdata, the hierarchical cluster tree structure, and related error and occlusion information is finally generated for runtime invocation.

[0039] Entering the adaptive rendering loop phase during application runtime, the system performs real-time decisions for each frame on the XR device. This process begins by acquiring the camera parameters for the current frame, and then traverses from the root node of the hierarchical cluster tree from top to bottom. During this traversal, efficient frustum culling and occlusion culling are performed sequentially on each cluster node, quickly eliminating a large number of geometric objects located outside the line of sight or occluded. For visible clusters C that pass the culling test, the system will calculate their geometric errors E based on these errors. geom (C) and the distance d to the camera cam Calculate the spatial error E of its projection screen using parameters such as these. screen (C), its formula can be expressed as:

[0040]

[0041] The system compares this error value to a preset threshold tau (e.g., 1 pixel). If the error is less than the threshold, it means the simplification of the cluster is sufficiently fine at the current distance, and the human eye cannot distinguish more details. Therefore, the system stops traversing the branch downwards and adds the cluster to the rendering list for this frame. Conversely, if the error is greater than the threshold, it indicates that higher precision details are needed, and the system continues traversing its child nodes. During this process, if the selected mesh microdata has not yet been loaded into GPU memory, the system sends a request to an asynchronous streaming manager to load the data from storage, while simultaneously rendering a low-precision version of its parent cluster to avoid holes in the image. Finally, the system efficiently packages and submits the list of all selected visible mesh microdata that meet the LOD requirements to the GPU for execution using modern graphics API GPU-driven rendering functions (such as Mesh Shaders), thereby minimizing CPU overhead and completing the rendering of a frame. The error threshold tau is obtained based on the predicted GPU rendering time for the next frame. Where τ0 is the basic threshold, T target For the target frame rendering time (e.g., XR devices typically require each frame to take ≤16.7ms), T pred This is the predicted GPU rendering time for the next frame. The error is compared with the threshold tau, which is adjusted by the GPU rendering time of the predicted next frame: if the prediction time is tight, tau is temporarily increased to reduce the rendering load; if there is enough time, tau is decreased to improve image quality.

[0042] Refer to the structural diagram of the entire control device (as shown in the attached diagram). Figure 4As shown, the system proposed in this invention mainly consists of two core modules: an offline preprocessor that runs during development and construction, and a runtime adaptive renderer embedded in the XR application for real-time rendering. The offline preprocessor receives source model assets as input and, through its internal subsystems such as mesh miniaturization, hierarchical cluster construction, and error / occlusion calculation, ultimately produces an optimized streaming asset package. The runtime adaptive renderer is the core of the system, itself consisting of four closely cooperating subsystems forming a complete closed loop. The workflow of this module begins with the hierarchical traversal and culling engine, which receives real-time camera parameters from the XR platform and traverses and culls the hierarchical tree in the streaming assets. The list of visible clusters after culling is passed to the system's "decision brain"—the LOD selection and streaming manager. This manager calculates the LOD requirement for each cluster based on the screen space error formula and makes the final rendering decision. When the data required for the decision is not in memory, it sends an asynchronous loading request to the background data streaming subsystem. Simultaneously, it sends the final list of mesh miniaturization elements to be rendered to the rendering instruction compositor. This synthesizer leverages the features of modern GPUs (such as Mesh Shaders) to package these minute elements into a very small number of efficient rendering instructions, interacting directly with the hardware and graphics APIs. This structure ensures a seamless transition from data input and intelligent decision-making to efficient execution, constituting the complete control device of this invention.

[0043] Based on the above solution, the present invention proposes three improvements:

[0044] 1. Predictive Performance Modeling and Proactive Adaptation

[0045] This innovation upgrades the engine's performance tuning mechanism from "passive response" to "active prediction," fundamentally changing how the system responds to load changes and aiming to resolve performance bottlenecks before they occur.

[0046] Traditional performance tuning, such as pure PID controllers, essentially compensates for lag based on historical errors (e.g., the rendering time of the previous frame). This solution introduces a performance prediction model that proactively estimates the rendering overhead of one or more future frames based on the current and recent state. This allows the system to allocate resources in advance, avoiding actual frame rate fluctuations and thus providing a smoother, more stable user experience.

[0047] We will predict the GPU rendering time for the next frame. Model it as a multivariate function:

[0048]

[0049] The detailed explanation of each parameter in this formula is as follows:

[0050] c f (Camera State Vector): Represents the camera state in the current frame f. It includes not only the camera's position and orientation, but also key parameters of its projection matrix, such as the field of view and near and far clipping planes. This vector determines which objects will fall within the view frustum and is the most fundamental input for calculating the rendering load.

[0051] v f (Camera Velocity Vector): Represents the camera's linear and angular velocities. This vector is crucial because it reveals user intent and changing scene trends. For example, a rapid head turn means a large amount of new geometry and textures will enter the field of view, potentially causing a sharp increase in rendering load. By analyzing this vector, the model can predict upcoming load spikes.

[0052] p f (Current Rendering Parameters): This vector contains the settings for all dynamically adjustable rendering parameters, such as the dynamic resolution scaling, shadow map resolution, and the on / off status of post-processing effects. These are system-controlled variables and directly affect the rendering time of the next frame.

[0053] H f-k..f (Historical Performance Data): This is a time series containing the actual GPU rendering times of the past k frames. This data is used for model self-correction. By comparing the predicted value of the previous frame with the actual value, the model can dynamically adjust its internal parameters to adapt to performance behaviors that are not explicitly modeled on a particular scene or device.

[0054] The specific implementation of function f can be chosen based on the requirements for accuracy and performance. In scenarios where extreme performance is required, a lightweight multiple linear regression model can be used; if higher prediction accuracy is required, an offline-trained machine learning model can be used, such as gradient boosting decision trees or a small feedforward neural network.

[0055] This predicted rendering time It will serve as the core input to more advanced control systems (such as Model Predictive Control, MPC), enabling the engine to move beyond reactive measures and instead proactively plan for global optimal performance across multiple future frame time windows.

[0056] 2. Cost-Benefit-Driven Intelligent Streaming and Prefetching Strategy

[0057] This innovation introduces an intelligent decision-making system based on "utility theory," which differs from traditional data streaming methods that rely solely on spatial proximity (such as octree / quadtree traversal of 3D Tiles).

[0058] Detailed description:

[0059] The engine's Streaming Manager calculates a "UtilityScore" for each data block that is out of view or not yet loaded (whether it's a tile in a 3D Tiles or an individual asset). This value dynamically quantifies the ratio between the "benefits" of loading the data block and the "costs" required, thus ensuring that content that best enhances the user experience is prioritized within limited bandwidth and memory budgets.

[0060] For any potential loadable data block i, its utility value U(i) is calculated as follows:

[0061]

[0062] Detailed explanation of each component of the formula:

[0063] Numerator (Benefit):

[0064] I(i) (Perceptual Importance): This is a core innovation, representing the importance of data block i at the application level. It is a normalized value that can be specified directly by the artist or designer in the editor, or automatically calculated based on the object's metadata (e.g., tagged as "key interactive object," "task objective," "protagonist," etc.). This allows the engine to "understand" the content, rather than just processing geometry.

[0065] Pvis(i) (Predicted Visibility Probability): This probability value is derived from the aforementioned performance prediction model. By analyzing the camera's current velocity and acceleration vectors, the system can predict the camera's possible trajectory within a short future time window and estimate the probability that data block i will enter the view frustum.

[0066] Denominator (Cost):

[0067] C load (i) Loading Cost: This cost is a function of the file size of the data block and the current network conditions (such as bandwidth and latency). Loading a large file on a slow network can be very costly.

[0068] C mem (i) (Memory Cost): Represents the memory resources used after loading data block i. This is a key limiting factor on memory-sensitive XR all-in-one machines. Loading a large asset may force the system to unload other equally useful assets, and this cost item represents the pressure it puts on the memory budget.

[0069] The streaming manager maintains a priority queue sorted by utility value U(i) in real time. When system resources allow, it retrieves the highest-utility block from the top of the queue and loads it. Conversely, when memory pressure is too high, it selects the currently loaded block with the lowest utility value for unloading. This mechanism ensures that the allocation of rendering resources always serves to maximize the user's visual and interactive experience.

[0070] 3. Hybrid Geometric and Semantic Error Metric

[0071] This innovation aims to improve the core decision-making criteria for LOD (Level of Detail) selection. This solution introduces a "semantic" dimension, ensuring that LOD selection not only conforms to physical laws but also meets the narrative needs of human cognition and application.

[0072] Detailed description:

[0073] When deciding which LOD level to render, the system no longer relies solely on traditional Screen-Space Error (SSE), but instead employs a novel hybrid error metric. This metric dynamically combines geometric precision with the semantic importance of objects.

[0074] Total screen space error E total The definition is as follows:

[0075] E total =(1-λ) s )·E geom +λ s ·E sem

[0076] Detailed explanation of each component:

[0077] E geom (Geometric Error): This part is consistent with traditional methods, representing the maximum geometric deviation on the screen after replacing a high-detail model with a low-detail model. Its calculation method is as follows:

[0078]

[0079] in

[0080] ∈ represents the geometric error sources of the model (such as Hausdorff distance).

[0081] H is the screen height, d is the distance, and θ is the field of view.

[0082] E sem (Semantic Error): This is the core innovation of this solution. It is a dimensionless, normalized value used to quantify the "loss of semantic information" caused by using a low-detail model. The higher the semantic importance of an object (e.g., it is a tool the user is interacting with, or a key clue in the task), the higher its corresponding E. sem The larger the value, the better. This means that even if the object can be simplified geometrically, the system will perceive this simplification as causing a significant "experience error" due to its high semantic importance. This value is typically configured by developers or designers during the content creation phase.

[0083] λ s (Semantic Weighting Factor): This is a dynamic weighting parameter between 0 and 1, acting as a "knob" to adjust the balance between geometric fidelity and semantic fidelity. λ s The value of λ can change dynamically based on the application context. For example, in a teaching scenario, when a user needs to carefully observe a specific part, the value of λ for that part... s The value can be dynamically increased by the program, thereby forcing the engine to render the highest level of LOD for it, while temporarily ignoring the details of the surrounding environment.

[0084] Application Example: Suppose there are two objects of similar size in the distance of a scene: a regular rock and an NPC issuing a quest. Under traditional, purely geometric error metrics, they might both be degraded to low-polygon models at the same time. However, under the hybrid error metric of this scheme, the NPC has extremely high semantic importance (high Equation). sem The total error E calculated from the value) total The model will be large, and the system will continue to maintain high-precision models and textures for it; while the stone with zero semantic importance will be simplified normally. In this way, the precious rendering budget is intelligently and automatically allocated to the elements most critical to the core experience.

[0085] Compared to existing technologies, the technical solution of this invention achieves significant improvements in many aspects and brings substantial beneficial effects. The most crucial improvement lies in its fundamental breaking of the rigid limitation of "the model must be fully loaded into memory" in traditional rendering pipelines through virtualized geometry and streaming technology, achieving breakthrough asset scale and scalability. This means that, theoretically, models of arbitrary complexity can be rendered, with performance overhead only related to pixel-level visible details. This makes it possible to render cinematic-quality art assets, high-precision BIM models, or large-scale 3D scan data in real time on lightweight XR all-in-one machines, greatly expanding the content boundaries of XR applications. Furthermore, this invention achieves fully automated optimal LOD generation and switching. Offline preprocessing automatically completes the miniaturization, hierarchicalization, and error measurement of the model, completely replacing the tedious, time-consuming, and expert-experience-dependent manual creation process in traditional LOD solutions. The runtime LOD selection is a continuous process based on a precise screen space error formula. Its benefits include significantly improved development efficiency and reduced art costs. Furthermore, because LOD is precisely matched pixel-by-pixel, it completely eliminates the annoying "jumping" phenomenon of traditional solutions, providing a seamless and smooth visual experience. Finally, this invention delivers extreme rendering performance optimization. The hierarchical culling algorithm can quickly cullate massive amounts of invisible geometry with logarithmic complexity, while the rendering mode based on mesh micro-elements perfectly matches the parallel computing architecture of modern GPUs, significantly reducing CPU-to-GPU communication overhead. The ultimate benefit is that even in extremely complex scenes, the CPU and GPU load can be maintained at a very low level, thus ensuring the high and stable frame rate upon which XR applications depend, effectively avoiding latency and dizziness caused by insufficient performance.

[0086] Figure 1 This paper demonstrates the detailed steps by which the system of this invention transforms a traditional high-precision model into optimized streaming assets during project construction. The process includes: inputting the source model, mesh miniaturization, constructing a hierarchical cluster, calculating geometric errors and occlusion information, and finally outputting the optimized asset package.

[0087] Figure 2 The core working mechanism of each frame during runtime is described in detail. The process includes: acquiring camera parameters, performing hierarchical tree traversal and culling, selecting LOD based on screen space error, and finally submitting the composite rendering instructions to the GPU.

[0088] Figure 3 The core data structure of this invention is presented in a tree-like format. A complete model is represented as a root cluster, which is progressively subdivided into smaller and finer sub-clusters, down to the lowest-level grid micro-elements.

[0089] Figure 4 The core components of the control device described in this invention are illustrated in block diagram form, including an offline preprocessor and a runtime adaptive renderer, as well as the four subsystems within the runtime module and their information flow and control flow relationships.

[0090] Imagine a scenario: an architect wearing a mainstream XR all-in-one device (such as the Meta Quest 3) needs to review a digital twin model of a factory, generated from a high-precision 3D scan and containing 500 million triangular facets, in a virtual environment. Before application packaging, the developer has automatically converted this massive factory model into an optimized streaming asset file using the offline preprocessor of this invention. This file contains complete hierarchical data and all decomposed mesh elements, and is then incorporated into the XR application.

[0091] When the designer puts on the helmet and starts the application, the runtime adaptive renderer does not attempt to load the entire model. Instead, it only requests to load the top-level node of the hierarchy tree and a small amount of necessary, low-precision cluster data near the designer's initial position. As a result, the application starts instantly, and the designer first sees a rough but complete outline of the factory.

[0092] As the designer begins to "walk" through the virtual factory, the adaptive rendering loop of this invention executes seamlessly at high speed in every frame. When the designer faces area A of the factory, the hierarchical traversal and culling engine uses frustum culling to quickly discard the entire upper-level cluster representing areas B and C at the top of the hierarchy tree. The hundreds of millions of faces belonging to these areas don't even have a chance to enter the next calculation. If the designer's gaze falls on a large machine tool in the distance, the system calculates that the screen space error of the machine tool cluster is extremely small. Therefore, the LOD selector only decides to render a highly simplified cluster version of the machine tool, without loading its complex internal parts. However, when the designer approaches the machine tool, the system detects that the shortened distance causes a sharp increase in screen space error. The LOD selector then automatically traverses the hierarchy tree downwards, searching for and requesting the streaming loading of the mesh elements corresponding to the more detailed sub-clusters representing the machine tool surface, control panel, etc. This data loading process is asynchronous. During the few milliseconds of loading time, the system continues to render a simplified version of its parent cluster to avoid holes in the image. Once the data is in place, the next frame immediately renders high-definition details. Ultimately, the thousands of mesh elements selected within a single frame from different objects and different LOD levels are packaged into a very small number of efficient instructions by the rendering instruction synthesizer and submitted to the GPU all at once.

[0093] Ultimately, the designer experienced a completely smooth user experience, with a stable frame rate of 90Hz and no stuttering or dizziness. From a distance, the view was of a grand scene structure; as the designer approached to examine any detail, that detail appeared photographically clear. The billions of polygons outside the viewpoint, obscured areas, and distant objects that didn't require attention never placed any performance burden on the device. This invention successfully achieved real-time, high-quality, high-performance rendering of ultra-large-scale 3D assets on resource-constrained XR devices in a completely transparent manner to users and developers.

[0094] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A hierarchical occlusion-aware mesh micro-element streaming and adaptive rendering method, characterized in that, The method includes the following steps: S1. Obtain a high-precision source model, decompose the model into mesh micro-elements, and recursively construct a bounding volume hierarchy tree (BVH) from bottom to top using the mesh micro-elements as leaf nodes. The nodes of the hierarchy tree are cluster nodes, and each parent node is a geometrically simplified representation of its child node cluster. Set semantic and geometric errors for each child node cluster. S2. Obtain the camera parameters of the current frame and traverse from the root node of the hierarchy tree from top to bottom. During the traversal, perform view frustum culling and occlusion culling on each cluster node to obtain the visible cluster. Calculate the total screen space error based on the error between the camera parameters and the visible cluster. Compare the total screen space error with the error threshold. If the error is less than the threshold, stop the traversal and add the cluster node to the rendering list of the current frame. Conversely, continue the traversal; S3. If the mesh element corresponding to the cluster node in the rendering list has not yet been loaded into the GPU memory, a loading request is sent to the asynchronous stream manager to asynchronously load the corresponding mesh element from storage and render the parent node of the cluster node. After the corresponding mesh element is loaded, it is packaged and sent to the GPU for rendering. Then, the next frame is updated to the current frame, and S2 is returned.

2. The hierarchical occlusion-aware mesh micro-element streaming and adaptive rendering method according to claim 1, characterized in that, The total screen space error is: AND total =(1-λ s )·AND geom +λ s ·AND sem Among them, E total E represents the total screen space error. geom E represents geometric error. sem λ represents semantic error. s This represents the semantic weight factor.

3. The hierarchical occlusion-aware mesh micro-element streaming and adaptive rendering method according to claim 2, characterized in that, The geometric error is: Where ∈ is the geometric error source of the model, H is the screen height, d is the distance from the tile to the camera in the camera parameters of the current frame, and θ is the field of view angle.

4. The hierarchical occlusion-aware mesh micro-element streaming and adaptive rendering method according to claim 1, characterized in that, The error threshold is obtained based on the predicted GPU rendering time for the next frame.

5. The hierarchical occlusion-aware mesh micro-element streaming and adaptive rendering method according to claim 4, characterized in that, The predicted GPU rendering time for the next frame is: in, For the predicted GPU rendering time of the next frame, c f Let v be the camera state vector. f Let p be the camera velocity vector. f H is the current rendering parameter vector. f-k..f Historical performance data is a time series containing the actual GPU rendering time of the past k frames.

6. The hierarchical occlusion-aware mesh micro-element streaming and adaptive rendering method according to claim 1, characterized in that, In S3, if multiple loading requests exist simultaneously, the utility value of the grid element corresponding to each loading request is calculated, and the loading requests are executed in descending order of utility value.

7. The hierarchical occlusion-aware mesh micro-element streaming and adaptive rendering method according to claim 6, characterized in that, The utility value is: Where U(i) is the utility value of the i-th loading request, I(i) is the perceived importance, Pvis(i) is the predicted visibility probability of the camera parameters in the current frame, and C load (i) represents the loading cost, C mem (i) represents the memory cost.

8. The hierarchical occlusion-aware mesh micro-element streaming and adaptive rendering method according to claim 1, characterized in that, The predicted visibility probability is obtained based on the camera's current velocity and acceleration vectors, and the predicted visibility probability is the probability that the mesh element corresponding to the i-th loading request enters the view frustum.

9. The hierarchical occlusion-aware mesh micro-element streaming and adaptive rendering method according to claim 1, characterized in that, The mesh micro-element contains only vertices and triangular faces with a number less than the micro-element threshold.

10. The hierarchical occlusion-aware mesh micro-element streaming and adaptive rendering method according to claim 1, characterized in that, The process of packaging and sending the image to the GPU for rendering specifically involves: The GPU-driven rendering functionality, using modern graphics APIs, is packaged and sent to the GPU for rendering.

Images