A Fixation Point Rendering Method and Device Based on Mesh Level of Detail

Abstract

The present invention discloses a foveated rendering method and device based on mesh detail levels, comprising: dividing and simplifying a mesh to be rendered to obtain micro-grids at various levels of detail, recording micro-grid related information, including the detail level bounding sphere and detail level error of the micro-grid, and the detail level bounding sphere and detail level error of the micro-grid's parent micro-grid; selecting micro-grids at various levels of detail using camera, foveated point information, and micro-grid related information for each frame; and using a mesh shader to render all selected micro-grids for each frame. This can greatly reduce rendering overhead and improve rendering frame rate without significantly reducing the user's image perception.

Description

Technical Field

[0001] The present invention belongs to the field of real-time design technology, and in particular relates to a foveated rendering method and device based on grid detail level. Background Art

[0002] Virtual Reality (VR) technology and applications have made significant progress. Due to the stringent requirements for low latency and high-resolution rendering in VR environments, traditional rendering methods can consume significant computing resources, impacting the user experience. To address the unique characteristics of VR rendering, numerous rendering optimization methods have been proposed, including foveated rendering.

[0003] Foveated rendering is a visual optimization technology whose core concept is to optimize image rendering based on the characteristics of the human retina. The human eye has the highest visual resolution in the fovea centralis, while the visual resolution drops significantly at the edges of the visual field. Therefore, eye tracking devices can capture the user's gaze focus in real time, reducing rendering detail in the peripheral areas of the visual field while maintaining high resolution in the center of the field of view. This significantly reduces rendering overhead and latency while ensuring the user's visual experience.

[0004] Foveated rendering is typically implemented using variable rate shading. This approach divides the view into multiple regions and applies different rendering resolutions based on their importance. For example, the highest resolution is used for rendering at the very center of the user's viewpoint, the area covered by the fovea, while areas further away are rendered at a lower resolution, with the user barely perceiving the loss of detail.

[0005] However, variable resolution implementations also have some significant drawbacks. First, different areas have different resolutions, and this gradual resolution change can sometimes lead to unsmooth edge transitions, resulting in visual layering or blurring. Users may perceive this resolution change, affecting immersion and experience. Second, implementing variable resolution requires rendering the same scene at multiple resolutions, significantly increasing the CPU overhead.

[0006] Level of Detail (LOD) is a common technique in computer graphics, primarily used to optimize rendering performance. Its core concept is to dynamically adjust an object's level of detail based on its distance from the camera. When an object is close to the camera, a high-precision model is used; when it is farther away, a lower-precision model is used. This method can significantly reduce the complexity of drawing geometry, thereby reducing vertex shader calls and improving rendering efficiency, making rendering overhead independent of camera viewpoint. Clearly, the LOD concept can be applied to foveated rendering techniques.

[0007] A meshlet is a complex 3D mesh (typically composed of tens of thousands of triangles) divided into several smaller subunits, each called a micromesh. A micromesh consists of a small set of vertices and triangles that can be processed independently on the GPU, making geometry processing in the rendering pipeline more efficient. Modern graphics libraries have introduced a new shader stage for micromesh-based rendering: the mesh shader. By dividing the 3D mesh into pieces and using the mesh shader for geometry processing, rendering efficiency can be greatly improved. Summary of the Invention

[0008] In order to improve the real-time rendering performance of VR devices, the present invention proposes a foveated rendering method and device based on mesh detail level, which greatly reduces the rendering overhead and improves the rendering frame rate without significantly reducing the user's screen viewing experience.

[0009] To achieve the above-mentioned object of the invention, an embodiment provides a foveated rendering method based on mesh detail level, comprising the following steps:

[0010] The mesh to be rendered is divided and simplified to obtain micro-meshes of various levels of detail, and relevant information of the micro-meshes is recorded, including the level of detail bounding sphere and level of detail error of the micro-meshes, and the level of detail bounding sphere and level of detail error of the micro-meshes' parent micro-meshes;

[0011] Each frame uses camera, gaze point information, and micro-mesh related information to select micro-meshes at each level of detail;

[0012] Render all selected micromeshes each frame using the mesh shader.

[0013] Preferably, the mesh to be rendered is segmented and simplified to obtain micro-meshes at various levels of detail, and relevant information about the micro-meshes is recorded, including:

[0014] According to the connection relationship between triangles, the triangles in the mesh are clustered to obtain a set of micro-grids at the 0th level of detail.

[0015] All microgrids in the microgrid set of the current level of detail are clustered according to the connection relationship between triangles to obtain multiple microgrid groups. All microgrids contained in each microgrid group are merged, simplified, and clustered to obtain the microgrid set of the next level of detail. This process is iterated until only one microgrid remains in the microgrid set. Microgrids of each level of detail are obtained, and the detail bounding sphere and detail error of the microgrid are recorded.

[0016] Preferably, all microgrids included in each microgrid group are merged, simplified, and clustered to obtain a microgrid set at the next level of detail, including:

[0017] Merge all microgrids contained in each current microgrid group to obtain a merged grid;

[0018] The merged mesh is simplified with the goal of reducing the number of triangles. During the simplification process, the boundary edges of the micro-mesh group need to be locked to ensure that the simplified mesh has the same boundary as the unsimplified mesh. The simplified mesh is obtained, and the error introduced by the simplification process is recorded.

[0019] All triangles in the simplified mesh are clustered to obtain a set of microgrids at the next level of detail. Each microgrid at the next level of detail is called the parent microgrid of each microgrid in the current microgrid group. The detail level bounding sphere and detail level error of the parent microgrid are recorded. For each parent microgrid, its detail level bounding sphere is recorded as the union of the detail level bounding spheres of all its child microgrids. Its detail level error is the maximum value of the maximum detail level error of all child microgrids and the error introduced by the simplification process.

[0020] Preferably, the mesh to be rendered after segmentation and simplification is constructed as a tree-shaped directed acyclic graph containing all levels of detail. Each node in the graph corresponds to a microgrid, and all nodes in each layer represent all microgrids at a level of detail. The directed edges between nodes represent parent-child relationships. Connecting from a parent node to a child node indicates that the parent node is simplified from the microgrid group where the child node is located. The leaf nodes of the graph represent all microgrids at level 0 of detail, and the root node of the graph represents the microgrid at the highest level of detail. Microgrid-related information is recorded at each node.

[0021] Preferably, each frame uses camera, gaze point information, and micromesh related information to select micromeshes at each level of detail, including:

[0022] A compute shader is used to select micromeshes at each level of detail in parallel. The screen error projected onto the screen is calculated based on camera and gaze point information as well as micromesh-related information. The screen error is compared with a threshold error to select a micromesh, and the selected micromesh is added to the rendering queue.

[0023] Preferably, for each level of detail of the micro-grid, a screen error projected onto the screen is calculated based on the camera and gaze point information and the micro-grid related information, and the screen error is compared with a threshold error to select the micro-grid, including:

[0024] Calculate the first screen error of the micro-mesh-to-screen projection based on the camera and gaze point information and the micro-mesh's own level of detail bounding sphere and level of detail error;

[0025] Calculate a second screen error of the projection of the micromesh's parent micromesh onto the screen based on the camera and gaze point information and the level-of-detail bounding sphere and level-of-detail error of the parent micromesh;

[0026] When the second screen error is greater than the threshold error, and when the first screen error is less than or equal to the threshold error, the micro cell is selected.

[0027] Preferably, the first screen error of the microgrid-to-screen projection and the second screen error of the parent microgrid-to-screen projection are calculated in the same manner, namely:

[0028] The maximum projection error of the detail level error projected onto the screen is calculated according to the detail level bounding sphere and camera parameters, the screen space position of the gaze point is determined according to the gaze point information, and the maximum projection error is attenuated based on the screen space position of the gaze point to obtain the attenuated error. The attenuated error is converted to pixels to obtain the screen error for comparison with the threshold error.

[0029] To achieve the above-mentioned object of the invention, an embodiment further provides a foveated rendering device based on mesh level of detail, characterized by comprising:

[0030] The level of detail partitioning module is used to split and simplify the mesh to be rendered to obtain micro-grids of various levels of detail, and record relevant information of the micro-grid, including the level of detail bounding sphere and level of detail error of the micro-grid, and the level of detail bounding sphere and level of detail error of the micro-grid's parent micro-grid;

[0031] A microgrid selection module, which selects microgrids at each level of detail using camera, gaze point information, and microgrid-related information in each frame;

[0032] A rendering module that renders all selected micromeshes each frame using a mesh shader.

[0033] To achieve the above-mentioned purpose of the invention, an embodiment also provides a computing device, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned grid detail level-based foveation rendering method.

[0034] To achieve the above-mentioned purpose of the invention, an embodiment further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the above-mentioned foveated rendering method based on mesh detail level is implemented.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention first divides and simplifies the mesh to be rendered to obtain micro-grids at various levels of detail, while recording relevant information about the micro-grids. Then, each frame uses camera, gaze point information, and micro-grid related information to select micro-grids at various levels of detail, and uses a mesh shader to render all selected micro-grids for each frame. This can greatly reduce rendering overhead and improve rendering frame rate without significantly reducing the user's visual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 is a flowchart of a foveated rendering method based on mesh level of detail provided by an embodiment;

[0039] Figure 2 This is a flowchart of dividing and simplifying a grid to obtain microgrids at various levels of detail, as provided in an embodiment;

[0040] Figure 3 is a flowchart of performing microgrid selection in parallel using a compute shader provided by an embodiment;

[0041] Figure 4 is an example diagram of a tree-shaped directed acyclic graph provided by an embodiment, wherein the maximum number of microgrids in a microgrid group is 4, and nodes of the same color belong to the same microgrid group;

[0042] Figure 5 is a schematic diagram of a rendered microgrid set obtained by level of detail selection of a directed acyclic graph provided by an embodiment, wherein solid nodes represent microgrids selected for rendering;

[0043] Figure 6 3 is a schematic diagram of a maximum projection error calculation process of a detail level error sphere projected onto a screen in camera space provided by an embodiment;

[0044] Figure 71 is a schematic diagram of calculating the maximum angle between the tangent line of the detail level bounding sphere and the coordinate axis Z provided by the embodiment;

[0045] Figure 8 3 is a schematic structural diagram of a foveated rendering device based on a grid detail level provided in an embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0047] like Figure 1 As shown, the embodiment provides a foveated rendering method based on mesh detail level, comprising the following steps:

[0048] S1, the mesh to be rendered is divided and simplified to obtain micro-grids of various levels of detail, and the relevant information of the micro-grid is recorded, including the detail level bounding sphere and detail level error of the micro-grid, and the detail level bounding sphere and detail level error of the micro-grid's parent micro-grid.

[0049] In the embodiment, a micromesh is a collection of triangles belonging to the same level of detail. The maximum number of triangles and the maximum number of vertices for a micromesh need to be specified. These two parameters should not be too large to affect rendering and simplification efficiency. The specific values can be set and combined by the user. In particular, the values of the two parameters need to take into account the upper limits of the number of triangles and vertices output by the mesh shader specified by the graphics library. A typical parameter setting is: the maximum number of triangles for a micromesh is set to 64, and the maximum number of vertices for a micromesh is set to 128.

[0050] The number of triangles and vertices in each micromesh cannot exceed but should be as close as possible to the maximum number of triangles and vertices. At the same time, the micromesh also needs to record the level of detail bounding sphere (LODBounding Sphere) and level of detail error (LOD Error) for itself and its parent micromesh. In addition, the micromesh also includes other information used for rendering, including vertex data and index data.

[0051] In the embodiment, preferably, the grid to be rendered is segmented and simplified to obtain microgrids of various levels of detail, and the microgrid related information is recorded, such as Figure 2 Shown, including:

[0052] (1) Cluster the triangles in the mesh according to the connection relationship between the triangles to obtain a set of micro-grids at the 0th level of detail;

[0053] The requirements for triangle clustering are as follows: each triangle in the original mesh must belong to and only belong to one micromesh to prevent gaps or overlaps between micromeshes after clustering; adjacent triangles have a greater probability of being grouped into one micromesh to improve the efficiency of micromesh rendering; the number of triangles and vertices in each micromesh does not exceed and should be as close to the upper limit as possible.

[0054] Obviously, since no mesh simplification has been performed on the LOD 0 micromesh, its LOD error is 0, and its LOD bounding sphere is the micromesh's own bounding sphere. The LOD 0 micromesh is stored in the micromesh buffer and used as the current LOD micromesh set in subsequent processes.

[0055] (2) Then, multiple cycles are performed. In the kth cycle, the microgrid set at the kth level of detail is used to generate the microgrid set at the k+1th level of detail. The operation of each cycle is as follows:

[0056] (2-1) Clustering all the microgrids in the microgrid set of the current level of detail according to the connection relationship between triangles to obtain multiple microgrid groups (Meshlet Group);

[0057] A meshlet group is a collection of meshlets at the same level of detail. A maximum number of meshlets in a meshlet group is specified, and this value is user-configurable. A typical setting is 16 meshlets per meshlet group.

[0058] The requirements for clustering microgrids are as follows: each microgrid must belong to one and only one microgrid group to prevent gaps or overlaps between microgrid groups after clustering; microgrids that share more edges have a greater probability of being grouped together, reducing the number of edges that need to be locked during subsequent mesh simplification and improving simplification efficiency; the number of microgrids in each microgrid group must not exceed and must be kept as close to the upper limit as possible.

[0059] (2-2) All microgrids contained in each microgrid group are merged, simplified, and clustered to obtain a microgrid set of the next level of detail. If there is still more than one microgrid in the microgrid set of the next level of detail, jump to step (2) to continue generating a microgrid set of the next level of detail, that is, iteratively execute step (2) until only one microgrid is left in the microgrid set, obtain microgrids of each level of detail, record the detail level bounding sphere and detail level error of the microgrid, take the last microgrid as the root microgrid, representing the highest level of detail, set its parent detail level bounding box to infinite, and set the parent detail level error to infinite.

[0060] In the embodiment, preferably, all microgrids included in each microgrid group are merged, simplified, and clustered to obtain a microgrid set at the next level of detail, including:

[0061] Merge all microgrids contained in each current microgrid group to obtain a merged grid;

[0062] Simplify the merged microgrid group to reduce its triangle count (e.g., halve it) and obtain the simplified mesh. The error introduced during the simplification process is recorded. During the simplification process, the boundary edges of the microgrid group must be locked to ensure that the simplified mesh has the same boundaries as the original mesh, avoiding seams when switching levels of detail.

[0063] Cluster all triangles in the simplified mesh to obtain a set of microgrids at the next level of detail. The specific requirements are as described in Generating the Initial Microgrid Set. Each microgrid at the next level of detail is referred to as the parent microgrid of every microgrid in the current microgrid group. Therefore, all microgrids in the same microgrid group share the same set of parent microgrids. The LOD bounding sphere and LOD error of the parent microgrid are also recorded. Specifically, for each parent microgrid, its LOD bounding sphere is recorded as the union of the LOD bounding spheres of all its child microgrids, and its LOD error is the maximum of the maximum LOD error of all its child microgrids and the error introduced by the simplification process. For each child microgrid, its parent LOD bounding sphere is set to the parent LOD bounding sphere, and its parent LOD error is set to the parent LOD error.

[0064] After this process, if Figure 4 As shown, a tree-shaped directed acyclic graph containing all levels of detail is established for the mesh to be rendered. Each node in the graph corresponds to a microgrid, and all nodes in each layer represent all microgrids at a level of detail. The directed edges between nodes represent parent-child relationships. Connecting from a parent node to a child node indicates that the parent node is simplified from the microgrid group where the child node is located. From the simplification process, it can be seen that a node may have multiple parents, and all microgrids belonging to a microgrid group can be replaced by all parent microgrids of this group without generating gaps or overlaps in the rendering results. The leaf nodes of the graph represent all microgrids at the 0th level of detail, the root node of the graph represents the microgrid at the highest level of detail, and microgrid-related information is recorded at each node.

[0065] S2, in each frame, uses the camera, gaze point information, and micro-mesh related information to select micro-meshes at each level of detail.

[0066] In this embodiment, the goal of detail level selection is to render the highest level of detail that meets the threshold error, so as to minimize the complexity of the rendered geometry within the allowable error range. Therefore, it is necessary to define a threshold error (Threshold Error) in pixels, which is expressed as For a micro-grid at a certain level of detail, the error of the detail level projected onto the screen is calculated, which is called the screen error. If the screen error is less than the threshold error, the error caused by rendering the micro-grid is acceptable.

[0067] At the same time, the choice of detail level needs to conform to the principle of foveated rendering: locations close to the foveation point tend to choose a lower level of detail to achieve higher rendering accuracy; locations far from the foveation point tend to choose a higher level of detail to save rendering overhead. Therefore, when calculating the error, the projection error of the screen projection needs to be attenuated according to the distance between the microgrid and the foveation point: for the same projection error, the farther away from the foveation point, the greater the error attenuation, and the smaller the final screen error. In this way, the corresponding level of detail can be selected according to the distance of the microgrid from the foveation point.

[0068] In the above processing steps, a directed acyclic graph (DAG) containing all levels of detail is generated for each mesh to be rendered. Therefore, it is necessary to select appropriate nodes in the graph for rendering. An obvious approach is to perform a breadth-first search from the root node of the graph to the leaf nodes: for each node, if its screen error is less than or equal to the threshold error, it is added to the rendering queue; if its screen error is greater than the threshold error, its child nodes are added to the processing queue. In fact, if each node stores its parent node information, each node can determine whether its parent node's error meets the requirements, thus determining whether it should be selected without relying on the parent node's judgment result. Furthermore, if the screen error is monotonically increasing along each path from the leaf node to the root node in the graph, then for each frame with fixed camera and gaze point information, the result of the DAG node selection is unique across the entire graph, and the selection of all nodes can be processed in parallel. As shown in the above DAG construction process, the calculation of the screen error depends on the DAG bounding sphere and the DAG error, both of which remain monotonically increasing at each level of the DAG during the bottom-up construction. Therefore, the selection and judgment of all microgrid nodes can be performed in parallel.

[0069] Based on this, in each frame, the camera, gaze point information and micro-grid related information are used to select micro-grids at each level of detail, such as Figure 3As shown in the figure, the specific process includes: before rendering each frame, a compute shader is used to select the level of detail of the directed acyclic graph containing all levels of detail in parallel. Each thread of the compute shader is responsible for processing a microgrid in the graph, calculating the screen error projected onto the screen based on the camera and gaze point information and microgrid related information, comparing the screen error with the threshold error to select the microgrid, and adding the selected microgrid to the rendering queue, as shown in the figure. Figure 5 As shown, the selected rendering micromesh collection.

[0070] More specifically, the first screen error of the micromesh to screen projection is calculated based on the camera and gaze point information as well as the detail level bounding sphere and detail level error of the micromesh itself. ;

[0071] Calculate the second screen error of the micromesh's parent micromesh to screen projection based on camera and gaze point information and the parent micromesh's level of detail bounding sphere and level of detail error ;

[0072] When the second screen error is greater than the threshold error, and when the first screen error is less than or equal to the threshold error, it satisfies and When , the microgrid is selected and the rendering queue is drawn. Among them, the threshold error is , usually taken as 1.

[0073] In one embodiment, the first screen error of the microgrid to screen projection is and the second screen error of the parent microgrid to the screen projection The calculation method is the same as:

[0074] Surrounding the sphere according to the level of detail Calculate detail level error with camera parameters Maximum projection error projected onto the screen , determine the screen space position of the gaze point based on the gaze point information , and based on the screen space position of the gaze point Maximum projection error Perform attenuation to obtain the error after attenuation , the attenuated error After converting to pixel units, the screen error is compared with the threshold error. For microgrids, screen error First screen error , for the parent microgrid, screen error Second screen error .

[0075] Specifically, in the camera coordinate system, the error sphere is defined as , the center of the sphere is ,radius Level of detail error , which may exist in the level of detail bounding sphere Any position within the error sphere The longest projection radius of the ellipse is the semi-major axis of the ellipse, which is the detail level error. The maximum projection error when projecting onto the screen.

[0076] Depend on Figure 6 As shown, in the camera coordinate system, let the viewpoint be , the projection surface is ,Pass And the plane parallel to the projection plane is , let the coordinate axis and The plane is ; In the plane In, order of Tangent is , and the cut point is ; pay At ,pay At ; pay At ; Easy to prove for The longest projection radius of ;

[0077] exist In the equation, we can get from trigonometric functions:

[0078] ;

[0079] in, For the projection surface Coordinate value ; for of Coordinate value ; is the radius of the error sphere, ; Tangent and Axis angle ; The maximum projection error ; Substitute into the above formula to get:

[0080] ;

[0081] From the above formula, we can see that the maximum projection error The size depends on and By scaling the two values separately, we can get a conservative upper bound for the maximum projection error:

[0082] ;

[0083] in, express The minimum value of express The minimum value of express The minimum value of Bounding sphere for level of detail The center of the ball, for The radius of May exist in Any location within, easy to get The minimum value of The minimum Z value in the range, that is ,in for The Z coordinate value of The maximum value for The tangent and coordinate axis The maximum value of the angle.

[0084] Depend on Figure 7 As shown, And the plane parallel to the projection plane is , Intersecting axes At ; Let the coordinate axis and The plane is ; In the plane In, order of Tangent is , and the cut point is ; Easy to prove Angle with coordinate axis Z That is ;make With the coordinate axis The angle is , and The angle is ; According to trigonometric functions:

[0085] ;

[0086] in:

[0087] ;

[0088] in, express The X coordinate value, express The Y coordinate value of express The Z coordinate value of

[0089] therefore, The calculation formula is as follows:

[0090] ;

[0091] Finally, we use the following formula to calculate :

[0092] ;

[0093] According to the screen space coordinates of the gaze point Maximum projection error Attenuation is performed to obtain the error after attenuation The goal of error attenuation is to: the error of the fixation point position will not be attenuated and will maintain its original value; the farther away from the fixation point, the greater the error attenuation, and the error changes linearly with the distance from the fixation point.

[0094] According to this principle, for a length of ,Leave The distance is The minimum error is made to attenuate the error for:

[0095] ;

[0096] The attenuation coefficient Should be:

[0097] ;

[0098] is a constant and requires a minimum attenuation coefficient specified by the user The maximum distance within the screen viewport Work out:

[0099] ;

[0100] The maximum distance within the screen viewport This is the diagonal length of the viewport, calculated from the viewport information:

[0101] ;

[0102] in, Indicates the right border coordinate value of the viewport. Indicates the left border coordinate value of the viewport. Indicates the coordinate value of the upper boundary of the viewport. Indicates the coordinate value of the lower boundary of the viewport;

[0103] Then for the maximum projection error , the error after attenuation for:

[0104] ;

[0105] In order to understand the above integral formula, we need to use the information of the detail level bounding sphere to Let the detail level projected onto the screen enclose the sphere The center of the sphere is , assuming One end is located at , and along direction, then:

[0106] ;

[0107] Among them, the upper and lower limits of points and for:

[0108] ;

[0109] in, express the distance between them;

[0110] If the lower limit ,but:

[0111] ;

[0112] If the lower limit ,but:

[0113] ;

[0114] Using viewport information and screen pixel information, the attenuated error Convert to pixels to get the final screen error :

[0115] ;

[0116] in, Indicates the number of pixels corresponding to the viewport width, Indicates the viewport width, Indicates the right border coordinate value of the viewport. Indicates the left edge coordinate value of the viewport.

[0117] S3, uses the mesh shader to render all selected micromeshes for each frame.

[0118] In an embodiment, when using a mesh shader to render all selected micro-meshes for each frame, directly rendering the rendering queue constructed by the micro-meshes selected in step S2 to obtain a rendering result includes:

[0119] For each microgrid in the rendering queue, add it to the rendering instructions of the corresponding material according to the material it uses;

[0120] Set the correct graphics API state for each material and issue the corresponding rendering instructions, where each micromesh corresponds to a mesh shader thread;

[0121] In the mesh shader thread, obtain the data of its corresponding micromesh, process and output all vertex attributes and triangle indices of the micromesh;

[0122] The subsequent rendering process can be completed using forward shading or deferred shading, and the final rendering result is output.

[0123] like Figure 8 As shown, the embodiment further provides a foveated rendering device 80 based on mesh detail levels, including a detail level division module 81, a micro-grid selection module 82, and a rendering module 83. The detail level division module 81 is used to divide and simplify the mesh to be rendered to obtain micro-grids of various levels of detail, and record micro-grid related information, including the detail level bounding sphere and detail level error of the micro-grid, and the detail level bounding sphere and detail level error of the micro-grid's parent micro-grid; the micro-grid selection module 82 is used to select micro-grids of various levels of detail per frame using camera, foveation point information, and micro-grid related information; and the rendering module 83 is used to render all selected micro-grids for each frame using a mesh shader.

[0124] It should be noted that the foveated rendering device based on mesh level of detail provided in the above embodiment, when performing foveated rendering, should be illustrated by the division of the above functional modules. The above functions can be assigned to different functional modules as needed, that is, the internal structure of the terminal or server is divided into different functional modules to complete all or part of the functions described above. In addition, the foveated rendering device based on mesh level of detail provided in the above embodiment and the foveated rendering construction method based on mesh level of detail embodiment are based on the same concept. The specific implementation process is detailed in the foveated rendering method based on mesh level of detail embodiment, and will not be repeated here.

[0125] Based on the same inventive concept, an embodiment further provides a computing device including a memory and one or more processors. The memory stores executable code. When the one or more processors execute the executable code, the device is configured to implement the above-mentioned mesh level of detail-based foveated rendering method, which specifically includes the following steps:

[0126] S1: Split and simplify the mesh to be rendered to obtain micro-meshes of various levels of detail, and record relevant information of the micro-meshes, including the level of detail bounding sphere and level of detail error of the micro-meshes, and the level of detail bounding sphere and level of detail error of the micro-meshes' parent micro-meshes;

[0127] S2, each frame uses camera, gaze point information and micro-mesh related information to select micro-grids at each level of detail;

[0128] S3, uses the mesh shader to render all selected micromeshes for each frame.

[0129] The computing device provided in the embodiment, in addition to the processor and memory, also includes hardware required for other services such as internal bus, network interface, memory, etc. at the hardware level. The memory is a non-volatile memory, and the processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the foveated point rendering method based on the grid detail level described in S1-S3 above. Of course, in addition to software implementation, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0130] Based on the same inventive concept, an embodiment further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the above-mentioned foveated rendering method based on mesh level of detail is implemented, specifically comprising the following steps:

[0131] S1: Split and simplify the mesh to be rendered to obtain micro-meshes of various levels of detail, and record relevant information of the micro-meshes, including the level of detail bounding sphere and level of detail error of the micro-meshes, and the level of detail bounding sphere and level of detail error of the micro-meshes' parent micro-meshes;

[0132] S2, each frame uses camera, gaze point information and micro-mesh related information to select micro-grids at each level of detail;

[0133] S3, uses the mesh shader to render all selected micromeshes for each frame.

[0134] In the embodiment, computer-readable media includes permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data.

[0135] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A foveated rendering method based on mesh level of detail, characterized in that: The following steps are involved: The mesh to be rendered is divided and simplified to obtain micro-meshes of various levels of detail, and relevant information of the micro-meshes is recorded, including the level of detail bounding sphere and level of detail error of the micro-meshes, and the level of detail bounding sphere and level of detail error of the micro-meshes' parent micro-meshes; For each frame, a screen error projected onto the screen is calculated based on camera and gaze point information and micromesh related information, and the screen error is compared with a threshold error to select micromeshes at each level of detail, including: calculating a first screen error projected onto the screen by the micromesh based on camera and gaze point information and the micromesh's own level of detail bounding sphere and level of detail error; calculating a second screen error projected onto the screen by the micromesh's parent micromesh based on camera and gaze point information and the parent micromesh's level of detail bounding sphere and level of detail error; when the second screen error is greater than the threshold error and when the first screen error is less than or equal to the threshold error, the micromesh is selected; The calculation methods for the first screen error projected from the microgrid to the screen and the second screen error projected from the parent microgrid to the screen are the same: the maximum projection error of the detail level error projected onto the screen is calculated based on the detail level bounding sphere and camera parameters, the screen space position of the gaze point is determined based on the gaze point information, and the maximum projection error is attenuated based on the screen space position of the gaze point to obtain the attenuated error. The attenuated error is converted to pixel units to obtain the screen error for comparison with the threshold error. Render all selected micromeshes each frame using the mesh shader.

2. The foveated rendering method based on mesh level of detail according to claim 1, characterized in that The mesh to be rendered is split and simplified to obtain micro-meshes at various levels of detail, and relevant information about the micro-meshes is recorded, including: According to the connection relationship between triangles, the triangles in the mesh are clustered to obtain a set of micro-grids at the 0th level of detail. All microgrids in the microgrid set of the current level of detail are clustered according to the connection relationship between triangles to obtain multiple microgrid groups. All microgrids contained in each microgrid group are merged, simplified, and clustered to obtain the microgrid set of the next level of detail. This process is iterated until only one microgrid remains in the microgrid set. Microgrids of each level of detail are obtained, and the detail bounding sphere and detail error of the microgrid are recorded.

3. The foveated rendering method based on mesh level of detail according to claim 2, characterized in that All microgrids contained in each microgrid group are merged, simplified, and clustered to obtain a microgrid set at the next level of detail, including: Merge all microgrids contained in each current microgrid group to obtain a merged grid; The merged mesh is simplified with the goal of reducing the number of triangles. During the simplification process, the boundary edges of the micro-mesh group need to be locked to ensure that the simplified mesh has the same boundary as the unsimplified mesh. The simplified mesh is obtained, and the error introduced by the simplification process is recorded. All triangles in the simplified mesh are clustered to obtain a set of microgrids at the next level of detail. Each microgrid at the next level of detail is called the parent microgrid of each microgrid in the current microgrid group. The detail level bounding sphere and detail level error of the parent microgrid are recorded. For each parent microgrid, its detail level bounding sphere is recorded as the union of the detail level bounding spheres of all its child microgrids. Its detail level error is the maximum value of the maximum detail level error of all child microgrids and the error introduced by the simplification process.

4. The foveated rendering method based on mesh level of detail according to claim 3, characterized in that After segmentation and simplification, the mesh to be rendered is constructed as a tree-shaped directed acyclic graph containing all levels of detail. Each node in the graph corresponds to a microgrid, and all nodes in each layer represent all microgrids at a level of detail. The directed edges between nodes represent parent-child relationships. Connecting from a parent node to a child node indicates that the parent node is simplified from the microgrid group where the child node is located. The leaf nodes of the graph represent all microgrids at level 0 of detail, and the root node of the graph represents the microgrid at the highest level of detail. Microgrid-related information is recorded at each node.

5. The foveated rendering method based on mesh level of detail according to claim 1, wherein: Also includes: The compute shader is used to select micro-meshes at each level of detail in parallel and the selected micro-meshes are added to the rendering queue.

6. A foveated rendering device based on mesh detail level, characterized in that include: The level of detail partitioning module is used to split and simplify the mesh to be rendered to obtain micro-grids of various levels of detail, and record relevant information of the micro-grid, including the level of detail bounding sphere and level of detail error of the micro-grid, and the level of detail bounding sphere and level of detail error of the micro-grid's parent micro-grid; A micromesh selection module is configured to calculate, for each frame, a screen error projected onto the screen based on camera and gaze point information and micromesh-related information, and compare the screen error with a threshold error to select micromeshes at each level of detail, including: calculating a first screen error projected onto the screen by the micromesh based on camera and gaze point information and the micromesh's own level-of-detail bounding sphere and level-of-detail error; calculating a second screen error projected onto the screen by the micromesh's parent micromesh based on camera and gaze point information and the parent micromesh's level-of-detail bounding sphere and level-of-detail error; and selecting the micromesh when the second screen error is greater than the threshold error and the first screen error is less than or equal to the threshold error. The calculation methods for the first screen error projected from the microgrid to the screen and the second screen error projected from the parent microgrid to the screen are the same: the maximum projection error of the detail level error projected onto the screen is calculated based on the detail level bounding sphere and camera parameters, the screen space position of the gaze point is determined based on the gaze point information, and the maximum projection error is attenuated based on the screen space position of the gaze point to obtain the attenuated error. The attenuated error is converted to pixel units to obtain the screen error for comparison with the threshold error. A rendering module that renders all selected micromeshes each frame using a mesh shader.

7. A computing device comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the one or more processors execute the executable code, they are used to implement the foveated rendering method based on mesh detail level according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that A program is stored thereon, and when the program is executed by a processor, the foveated rendering method based on mesh detail level described in any one of claims 1 to 5 is implemented.

Images