Grid detail level-based fixation point rendering method and device

Abstract

The invention discloses a grid detail level-based fixation point rendering method and device, and the method comprises the steps: carrying out the segmentation and simplification of a to-be-rendered grid, obtaining a micro-grid of each level of detail level, recording the related information of the micro-grid, including a detail level bounding ball and a detail level error of the micro-grid, a detail level bounding ball and a detail level error of a parent micro-grid of the micro-grids; in each frame, using a camera, fixation point information and micro-grid related information to select micro-grids of each level of detail level; and rendering all the selected micro-grids for each frame by using a grid shader, so that the rendering overhead can be greatly reduced and the rendering frame rate can be improved under the condition that the picture impression of a user is not obviously reduced.

Description

Technical Field

[0001] The present invention belongs to the field of real-time design technology, and in particular relates to a foveated rendering method and device based on grid detail level. Background Art

[0002] Virtual Reality technology and applications have made great progress. Due to the high requirements for low latency and high-resolution rendering in VR environments, traditional rendering methods may consume a lot of computing resources and affect user experience. In view of the characteristics of VR rendering, many rendering optimization methods have been proposed, and foveated rendering is one of them.

[0003] Foveated rendering is a visual optimization technology. Its core concept is to optimize image rendering based on the characteristics of the human eye's retina. The human eye has the highest visual resolution in the fovea of ​​the retina, while the visual resolution drops significantly in the edge of the field of vision. Therefore, the user's line of sight can be captured in real time through an eye tracking device, and the rendering details in the peripheral area of ​​the field of vision can be reduced, while maintaining high-resolution rendering in the center of the field of vision, so as to significantly reduce rendering overhead and latency while ensuring the user's visual experience.

[0004] Usually, the implementation of foveated rendering adopts variable rate shading. This method divides the view into multiple areas and applies different rendering resolutions according to the importance of the area. For example, the highest resolution is used for rendering in the center of the user's viewpoint, that is, the area covered by the fovea of ​​the retina, and the lower resolution is used for rendering in areas far from the center, and the user can hardly perceive the decrease in details.

[0005] However, the implementation of variable resolution also has some significant disadvantages. First, the resolution of different areas is different. This gradual change in resolution sometimes causes the edge transition area to be not smooth enough, resulting in visual layering or blurred images. Users may perceive this resolution change, affecting immersion and experience. Second, the implementation of variable resolution requires rendering the same scene at multiple resolutions, which significantly increases the CPU overhead of rendering.

[0006] Level of Detail is a commonly used technique in computer graphics, mainly used to optimize rendering performance. The core idea is to dynamically adjust the level of detail of an object based on the distance between the object and the camera. When the object is closer to the camera, a high-precision model is used; when the object is farther away, a lower-precision model is used. This method can significantly reduce the complexity of drawing geometry, reduce the call of vertex shaders, and improve rendering efficiency, so that the rendering cost is independent of the camera perspective. Obviously, the idea of ​​level of detail can be used in foveated rendering technology.

[0007] A micromesh is a complex 3D mesh (usually composed of tens of thousands of triangles) divided into several small sub-units, each of which is called a micromesh. A micromesh contains a small group of vertices and triangles that can be processed independently on the GPU, making geometry processing in the rendering pipeline more efficient. In modern graphics libraries, a new shader stage is introduced for micromesh-based rendering: the Mesh Shader. By dividing the 3D mesh into mesh pieces and using the mesh shader for geometry processing, rendering efficiency can be greatly improved. Summary of the invention

[0008] In order to improve the real-time rendering performance of VR devices, the present invention proposes a foveation rendering method and device based on grid detail level, which can greatly reduce the rendering overhead and improve the rendering frame rate without significantly reducing the user's picture perception.

[0009] To achieve the above-mentioned object of the invention, an embodiment provides a foveated rendering method based on mesh detail level, comprising the following steps: The mesh to be rendered is divided and simplified to obtain micro-meshes of various levels of detail, and the micro-mesh related information is recorded, including the detail level bounding sphere and detail level error of the micro-mesh, and the detail level bounding sphere and detail level error of the micro-mesh's parent micro-mesh; Each frame uses the camera, gaze point information, and micro-mesh related information to select micro-meshes at each level of detail; Render all selected micromeshes each frame using the mesh shader.

[0010] Preferably, the mesh to be rendered is segmented and simplified to obtain micro-meshes of various levels of detail, and micro-mesh related information is recorded, including: According to the connection relationship between triangles, the triangles in the mesh are clustered to obtain a set of micro-grids at the 0th level of detail. All microgrids in the microgrid set of the current level of detail are clustered according to the connection relationship between triangles to obtain multiple microgrid groups, and all microgrids contained in each microgrid group are merged, simplified, and clustered to obtain a microgrid set of the next level of detail. The process is iterated until only one microgrid is left in the microgrid set to obtain microgrids of each level of detail, and the detail level bounding sphere and detail level error of the microgrid are recorded.

[0011] Preferably, all microgrids included in each microgrid group are merged, simplified, and clustered to obtain a microgrid set at the next level of detail, including: Merge all microgrids contained in each current microgrid group to obtain a merged grid; The merged mesh is simplified with the goal of reducing the number of triangles. During the simplification process, the boundary edges of the micro-mesh group need to be locked to ensure that the simplified mesh has the same boundary as the mesh before simplification. The simplified mesh is obtained, and the error introduced in the simplification process is recorded. Cluster all triangles in the simplified mesh to get the next level of detail level micro-grid set. Each micro-grid of the next level of detail level is called the parent micro-grid of each micro-grid in the current micro-grid group. The detail level bounding sphere and detail level error of the parent micro-grid are recorded. For each parent micro-grid, its detail level bounding sphere is recorded as the union of the detail level bounding spheres of all its child micro-grids, and its detail level error is the maximum value of the maximum detail level error of all child micro-grids and the error introduced by the simplification process.

[0012] Preferably, the mesh to be rendered after segmentation and simplification is constructed as a tree-shaped directed acyclic graph containing all levels of detail, each node in the graph corresponds to a microgrid, all nodes in each layer are all microgrids at one level of detail, directed edges between nodes represent parent-child relationships, and connections from parent nodes to child nodes indicate that the parent node is simplified from the microgrid group where the child node is located, the leaf nodes of the graph represent all microgrids at the 0th level of detail, the root node of the graph represents the microgrid at the highest level of detail, and microgrid related information is recorded at each node.

[0013] Preferably, each frame uses the camera, gaze point information and micro-mesh related information to select micro-meshes at each level of detail, including: A compute shader is used to select micro-meshes at each level of detail in parallel. Specifically, the screen error projected onto the screen is calculated based on the camera and gaze point information and the micro-mesh related information. The screen error is compared with the threshold error to select the micro-mesh, and the selected micro-mesh is added to the rendering queue.

[0014] Preferably, for each level of detail of the micro-grid, a screen error projected onto the screen is calculated based on the camera and gaze point information and the micro-grid related information, and the screen error is compared with a threshold error to select the micro-grid, including: Calculate the first screen error of the projection of the micro-mesh onto the screen based on the camera and gaze point information and the detail level bounding sphere and detail level error of the micro-mesh itself; Calculate a second screen error of the projection of the micromesh's parent micromesh onto the screen based on the camera and gaze point information and the level-of-detail bounding sphere and level-of-detail error of the parent micromesh; When the second screen error is greater than a threshold error, and when the first screen error is less than or equal to the threshold error, the micro grid is selected.

[0015] Preferably, the first screen error of the microgrid to the screen projection and the second screen error of the parent microgrid to the screen projection are calculated in the same way, both of which are: The maximum projection error of the detail level error projected onto the screen is calculated according to the detail level bounding sphere and the camera parameters, the screen space position of the gaze point is determined according to the gaze point information, and the maximum projection error is attenuated based on the screen space position of the gaze point to obtain the attenuated error. The attenuated error is converted to pixels to obtain the screen error for comparison with the threshold error.

[0016] To achieve the above-mentioned object of the invention, the embodiment further provides a foveated point rendering device based on mesh detail level, characterized in that it includes: A detail level division module is used to divide and simplify the mesh to be rendered to obtain micro-grids of various levels of detail levels, and record micro-grid related information, including the detail level bounding sphere and detail level error of the micro-grid, and the detail level bounding sphere and detail level error of the micro-grid's parent micro-grid; A microgrid selection module is used to select microgrids at each level of detail using camera, gaze point information and microgrid related information in each frame; A rendering module that is used to render all selected micromeshes each frame using a mesh shader.

[0017] To achieve the above-mentioned purpose of the invention, an embodiment further provides a computing device, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned grid detail level-based foveation rendering method.

[0018] To achieve the above-mentioned purpose of the invention, an embodiment further provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the above-mentioned foveated point rendering method based on grid detail level is implemented.

[0019] Compared with the prior art, the present invention has the following beneficial effects: The present invention first divides and simplifies the mesh to be rendered to obtain micro-meshes at various levels of detail, and records the relevant information of the micro-meshes. Then, the camera, gaze point information and micro-mesh related information are used to select micro-meshes at various levels of detail in each frame, and a mesh shader is used to render all selected micro-meshes for each frame. This can greatly reduce the rendering overhead and improve the rendering frame rate without significantly reducing the user's picture perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a foveated rendering method based on a grid detail level provided in an embodiment; Figure 2 It is a flow chart of dividing and simplifying a grid to obtain microgrids of various levels of detail provided by an embodiment; Figure 3 is a flowchart of performing microgrid selection in parallel using a compute shader provided by an embodiment; Figure 4 is an example diagram of a tree-shaped directed acyclic graph provided by an embodiment, wherein the maximum number of microgrids in a microgrid group is 4, and nodes of the same color belong to the same microgrid group; Figure 5 is a schematic diagram of a rendering microgrid set obtained by selecting a directed acyclic graph provided by an embodiment through a detail level, wherein a solid node represents a microgrid selected for rendering; Figure 6 is a schematic diagram of a maximum projection error calculation process of projecting a detail level error sphere onto a screen in a camera space provided by an embodiment; Figure 7 is a schematic diagram of calculating the maximum angle between the tangent line of the detail level bounding sphere and the coordinate axis Z provided by the embodiment; Figure 8 It is a structural schematic diagram of a foveated point rendering device based on a grid detail level provided in an embodiment. DETAILED DESCRIPTION

[0022] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0023] like Figure 1 As shown, the embodiment provides a foveated rendering method based on mesh detail level, comprising the following steps: S1, divide and simplify the mesh to be rendered to obtain micro-meshes of various levels of detail, and record the micro-mesh related information, including the detail level bounding sphere and detail level error of the micro-mesh, and the detail level bounding sphere and detail level error of the micro-mesh's parent micro-mesh.

[0024] In the embodiment, a micromesh (Meshlet) is a collection of a series of triangles belonging to the same level of detail. It is necessary to specify the maximum number of triangles and the maximum number of vertices of the micromesh. These two parameters should not be too large so as not to affect the efficiency of rendering and simplification. The specific values ​​can be set and combined by the user. In particular, the values ​​of the two need to take into account the upper limits of the number of triangles and vertices output by the mesh shader specified by the graphics library. A typical parameter setting is: the maximum number of triangles of the micromesh is set to 64, and the maximum number of vertices of the micromesh is set to 128.

[0025] The number of triangles and vertices in each micro-mesh cannot exceed but be as close as possible to the maximum number of triangles and vertices. At the same time, the micro-mesh also needs to record the level of detail bounding sphere (LODBounding Sphere) and level of detail error (LOD Error) of itself and its parent micro-mesh. In addition, the micro-mesh also includes other information for rendering, including vertex data and index data.

[0026] In the embodiment, preferably, the grid to be rendered is segmented and simplified to obtain micro grids of various levels of detail, and the micro grid related information is recorded, such as Figure 2 As shown, including: (1) Cluster the triangles in the mesh according to the connection relationship between the triangles to obtain a set of micro-grids at the 0th level of detail. The requirements for triangle clustering are as follows: each triangle in the original mesh must belong to and only belong to one micro-mesh to prevent gaps or overlaps between micro-meshes after clustering; adjacent triangles have a greater probability of being grouped into one micro-mesh to improve the efficiency of micro-mesh rendering; the number of triangles and vertices of each micro-mesh does not exceed and should be as close to the upper limit specified above as possible.

[0027] It is obvious that the microgrid at level 0 of detail level has a detail level error of 0 because no mesh simplification is performed, and its detail level bounding sphere is the bounding sphere of the microgrid itself. The microgrid at level 0 of detail level is stored in the microgrid buffer and participates in the subsequent process as the current level of detail level microgrid collection.

[0028] (2) Next, multiple loops are performed. In the kth loop, the micro-grid set at the kth level of detail is used to generate the micro-grid set at the k+1th level of detail. The operation of each loop is as follows: (2-1) Clustering all microgrids in the microgrid set of the current level of detail according to the connection relationship between triangles to obtain multiple microgrid groups (Meshlet Group); A meshlet group is a collection of meshlets at the same level of detail. The maximum number of meshlets in a group needs to be specified. The specific value can be set by the user. A typical parameter setting is that the maximum number of meshlets in a meshlet group is 16.

[0029] The requirements for clustering microgrids are: each microgrid must belong to and only belong to one microgrid group to prevent gaps or overlaps between microgrid groups after clustering; microgrids that share more edges have a greater probability of being grouped into one microgrid group to reduce the number of edges that need to be locked in the subsequent mesh simplification process and improve simplification efficiency; the number of microgrids in each microgrid group cannot exceed and should be as close to the upper limit specified above as possible; (2-2) All microgrids contained in each microgrid group are merged, simplified, and clustered to obtain a microgrid set of the next level of detail. If there is still more than one microgrid in the microgrid set of the next level of detail, jump to step (2) to continue to generate a microgrid set of the next level of detail, that is, iteratively execute step (2) until only one microgrid is left in the microgrid set, obtain microgrids of each level of detail, record the detail level bounding sphere and detail level error of the microgrid, take the last microgrid as the root microgrid, representing the highest level of detail, set its parent detail level bounding box to infinite, and set the parent detail level error to infinite.

[0030] In the embodiment, preferably, all microgrids included in each microgrid group are merged, simplified, and clustered to obtain a microgrid set at the next level of detail, including: Merge all microgrids contained in each current microgrid group to obtain a merged grid; Simplify the mesh after merging the micro-grid group, aiming to reduce the number of triangles (e.g. halve) to obtain the simplified mesh of the group, and record the error introduced in the simplification process. During the simplification process, the boundary edges of the micro-grid group need to be locked to ensure that the simplified mesh has the same boundary as the mesh before simplification, avoiding seam problems when switching detail levels; Cluster all triangles in the simplified mesh to obtain a microgrid set of the next level of detail. The specific requirements are as described in generating the initial microgrid set. Each microgrid of the next level of detail is called the parent microgrid of each microgrid in the current microgrid group. Therefore, all microgrids belonging to the same microgrid group have the same parent microgrid set. The detail level bounding sphere and detail level error of the parent microgrid are also recorded. Specifically, for each parent microgrid, its detail level bounding sphere is recorded as the union of the detail level bounding spheres of all its child microgrids, and its detail level error is the maximum value of the maximum detail level error of all child microgrids and the error introduced by the simplification process. For each child microgrid, set its parent detail level bounding sphere to the above parent detail level bounding sphere, and its parent detail level error to the above parent detail level error.

[0031] After this process, if Figure 4 As shown in the figure, a tree-shaped directed acyclic graph containing all levels of detail is established for the mesh to be rendered. Each node of the graph corresponds to a microgrid, and all nodes of each layer are all microgrids of a level of detail. The directed edges between nodes represent the parent-child relationship. Connecting from the parent node to the child node indicates that the parent node is simplified from the microgrid group where the child node is located. From the simplification process, it can be seen that a node may have multiple parents, and all microgrids belonging to a microgrid group can be replaced by all parent microgrids of this group without generating gaps or overlaps in the drawing results. The leaf nodes of the graph represent all microgrids at the 0th level of detail, the root node of the graph represents the microgrid at the highest level of detail, and the microgrid related information is recorded at each node.

[0032] S2, each frame uses the camera, gaze point information and micro-mesh related information to select micro-meshes at each level of detail.

[0033] In the embodiment, the goal of detail level selection is to render the highest level of detail level that satisfies the threshold error, so as to minimize the complexity of the rendered geometry within the allowable error range. Therefore, it is necessary to define a threshold error (Threshold Error) in pixels, which is expressed as For a micro-mesh with a certain level of detail, the error of the detail level projected onto the screen is calculated, which is called the screen error. If the screen error is less than the threshold error, the error caused by rendering the micro-mesh is acceptable.

[0034] At the same time, the choice of detail level needs to comply with the principle of foveated rendering: positions close to the foveation point tend to choose a lower level of detail level to achieve higher rendering accuracy; positions far from the foveation point tend to choose a higher level of detail level to save rendering overhead. Therefore, when calculating the error, the projection error of the screen projection needs to be attenuated according to the distance between the microgrid and the foveation point: for the same projection error, the farther away from the foveation point, the greater the error attenuation, and the smaller the final screen error. In this way, the corresponding level of detail level can be selected according to the distance of the microgrid from the foveation point.

[0035] In the above processing steps, a directed acyclic graph containing all levels of detail levels is generated for each mesh to be rendered. Therefore, it is necessary to select the appropriate node in the graph for rendering. An obvious method is to perform a breadth-first search from the root node to the leaf node of the graph: for each node, if its screen error is less than or equal to the threshold error, it will be added to the rendering queue; if its screen error is greater than the threshold error, its child node will be added to the queue to be processed. In fact, if the information of its parent node is stored in each node, each node can judge whether the error of its parent node meets the requirements, so that it can judge whether it should be selected without relying on the judgment result of the parent node. Further, if the screen error is monotonically increasing on each path from the leaf node to the root node in the graph, then for each frame of fixed camera and gaze point information, the result of the detail level node selection of the entire graph is unique, and the selection judgment of all nodes can be processed in parallel. From the above directed acyclic graph construction process, it can be seen that the calculation of the screen error depends on the detail level bounding sphere and the detail level error, both of which remain monotonically increasing in the process of constructing each level of the directed acyclic graph from bottom to top. Therefore, the selection and judgment of all microgrid nodes can be performed in parallel.

[0036] Based on this, in each frame of the embodiment, the camera, the gaze point information and the micro-grid related information are used to select the micro-grids at each level of detail, such as Figure 3 As shown in , it specifically includes: before rendering each frame, a compute shader (ComputeShader) is used to select the level of detail of the directed acyclic graph containing all levels of detail in parallel. Each thread of the compute shader is responsible for processing a microgrid in the graph, calculating the screen error projected to the screen based on the camera and gaze point information and microgrid related information, comparing the screen error with the threshold error to select the microgrid, and adding the selected microgrid to the rendering queue, as shown in Figure 5 As shown, a collection of rendered micromeshes is selected.

[0037] More specifically, the first screen error of the micromesh to screen projection is calculated based on the camera and gaze point information as well as the micromesh's own level of detail bounding sphere and level of detail error ; Calculate the second screen error of the micromesh's parent micromesh to screen projection based on the camera and gaze point information and the parent micromesh's level of detail bounding sphere and level of detail error ; When the second screen error is greater than the threshold error, and when the first screen error is less than or equal to the threshold error, and When , the microgrid is selected and the rendering queue is drawn. The threshold error is , usually taken as 1.

[0038] In the embodiment, the first screen error of the microgrid to the screen projection is and the second screen error of the parent microgrid to the screen projection The calculation method is the same as: Surround the sphere according to the level of detail Calculate detail level error with camera parameters Maximum projection error onto the screen , determine the screen space position of the gaze point based on the gaze point information , and based on the screen space position of the gaze point The maximum projection error Attenuation is performed to obtain the error after attenuation , the attenuated error After conversion to pixels, the screen error is compared with the threshold error. For microgrids, screen error First screen error , for the parent microgrid, screen error Second screen error .

[0039] Specifically, in the camera coordinate system, the error sphere is defined as , the center of the sphere is ,radius is the detail level error , which may exist in the detail level bounding sphere Any position within the error sphere The perspective projection onto the screen is an ellipse, and its longest projection radius is the semi-major axis of the ellipse, which is the detail level error. The maximum projection error when projecting onto the screen.

[0040] Depend on Figure 6 As shown, in the camera coordinate system, let the viewpoint be , the projection surface is ,Pass And the plane parallel to the projection plane is , let the coordinate axis and The plane is ; In the plane In, order of Tangent is , and the cut point is ; pay At ,pay At ; pay At ; Easy to prove for The longest projection radius of ; exist In the equation, we can get the trigonometric function: ; in, For projection surface Coordinate value ; for of Coordinate value ; is the radius of the error sphere ; Tangent and Axis Angle ; The maximum projection error ; Substitute into the above formula and we get: ; From the above formula, we can see that the maximum projection error The size depends on and By scaling the values ​​of the two separately, we can get a conservative upper bound of the maximum projection error: ; in, express The minimum value of express The minimum value of express The minimum value of Bounding the sphere for the level of detail The center of the ball, for The radius of May exist in Any location within The minimum value of The minimum value of Z in the range, that is ,in for The Z coordinate value of The maximum value of for The tangent line and the coordinate axis The maximum value of the angle.

[0041] Depend on Figure 7 As shown, And the plane parallel to the projection plane is , Intersection Axis At ; Let the coordinate axis and The plane is ; In the plane In, order of Tangent is , and the cut point is ; Easy to prove Angle with coordinate axis Z That is ;make With coordinate axis The angle is , and The angle is ; According to trigonometric functions: ; in: ; in, express The X coordinate value, express The Y coordinate value of express The Z coordinate value of therefore, The calculation formula is as follows: ; Finally, we use the following formula to calculate : ; Based on the screen space coordinates of the gaze point The maximum projection error Attenuation is performed to obtain the error after attenuation The goal of error attenuation is that the error of the fixation point position will not be attenuated and will maintain its original value; the farther away from the fixation point, the greater the degree of error attenuation, and the error changes linearly with the distance from the fixation point.

[0042] According to this principle, for a length of ,Leave The distance is The error is very small, so that the error after attenuation for: ; The attenuation coefficient Should be: ; is a constant and requires a minimum attenuation coefficient specified by the user The maximum distance within the screen viewport Work out: ; The maximum distance within the screen viewport This is the diagonal length of the viewport, calculated from the viewport information: ; in, Indicates the right border coordinate value of the viewport. Indicates the left border coordinate value of the viewport. Indicates the coordinate value of the upper boundary of the viewport. Indicates the coordinate value of the lower boundary of the viewport; For the maximum projection error , the error after attenuation for: ; To understand the above integral formula, we need to use the information of the detail level bounding sphere Let the detail level enclosing the sphere projected onto the screen The center of the sphere is , assuming One end is located at , and along direction, then: ; Among them, the upper and lower limits of the integral and for: ; in, express The distance between If the lower limit ,but: ; If the lower limit ,but: ; Using the viewport information and screen pixel information, the attenuated error Convert to pixels to get the final screen error : ; in, Indicates the number of pixels corresponding to the viewport width. Indicates the viewport width, Indicates the right border coordinate value of the viewport. Indicates the left edge coordinate value of the viewport.

[0043] S3, uses the mesh shader to render all selected micro-meshes for each frame.

[0044] In the embodiment, when using a mesh shader to render all selected micro-meshes for each frame, directly rendering the rendering queue constructed by the micro-meshes selected in step S2 to obtain a rendering result includes: For each microgrid in the rendering queue, add it to the rendering instructions of the corresponding material according to the material it uses; Set the correct graphics API state for each material and issue the corresponding rendering instructions, where each micro-mesh corresponds to a mesh shader thread; In the mesh shader thread, obtain the data of the corresponding micro-mesh, process and output all vertex attributes and triangle indices of the micro-mesh; The subsequent rendering process can be completed using forward shading or deferred shading, and the final rendering result can be output.

[0045] like Figure 8As shown, the embodiment further provides a foveation rendering device 80 based on mesh detail level, including a detail level division module 81, a micro-grid selection module 82, and a rendering module 83, wherein the detail level division module 81 is used to divide and simplify the mesh to be rendered to obtain micro-grids of each level of detail level, and record the micro-grid related information, including the detail level bounding sphere and detail level error of the micro-grid, and the detail level bounding sphere and detail level error of the parent micro-grid of the micro-grid; the micro-grid selection module 82 is used to select micro-grids of each level of detail level using the camera, foveation point information and micro-grid related information for each frame; the rendering module 83 is used to use the mesh shader to render all selected micro-grids for each frame.

[0046] It should be noted that the foveated rendering device based on the grid detail level provided in the above embodiment should be illustrated by the division of the above functional modules when performing foveated rendering. The above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the terminal or server is divided into different functional modules to complete all or part of the functions described above. In addition, the foveated rendering device based on the grid detail level provided in the above embodiment and the foveated rendering construction method embodiment based on the grid detail level belong to the same concept. The specific implementation process is detailed in the foveated rendering method embodiment based on the grid detail level, which will not be repeated here.

[0047] Based on the same inventive concept, an embodiment further provides a computing device, including a memory and one or more processors, wherein an executable code is stored in the memory, and when the one or more processors execute the executable code, the foveated rendering method based on the mesh detail level is implemented, specifically including the following steps: S1, divide and simplify the mesh to be rendered to obtain micro-meshes of various levels of detail, and record the micro-mesh related information, including the micro-mesh's level of detail bounding sphere and level of detail error, the micro-mesh's parent micro-mesh's level of detail bounding sphere and level of detail error; S2, each frame uses the camera, gaze point information and micro-mesh related information to select micro-meshes at each level of detail; S3, uses the mesh shader to render all selected micro-meshes for each frame.

[0048] The computing device provided in the embodiment, in addition to the processor and memory, also includes hardware required for other services such as internal bus, network interface, memory, etc. at the hardware level. The memory is a non-volatile memory, and the processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned grid detail level based gaze point rendering method S1-S3. Of course, in addition to the software implementation, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0049] Based on the same inventive concept, an embodiment further provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the above-mentioned foveated rendering method based on mesh detail level is implemented, which specifically includes the following steps: S1, divide and simplify the mesh to be rendered to obtain micro-meshes of various levels of detail, and record the micro-mesh related information, including the micro-mesh's level of detail bounding sphere and level of detail error, the micro-mesh's parent micro-mesh's level of detail bounding sphere and level of detail error; S2, each frame uses the camera, gaze point information and micro-mesh related information to select micro-meshes at each level of detail; S3, uses the mesh shader to render all selected micro-meshes for each frame.

[0050] In the embodiment, computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data.

[0051] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A foveated rendering method based on mesh detail level, characterized in that: The following steps are involved: The mesh to be rendered is divided and simplified to obtain micro-meshes of various levels of detail, and the micro-mesh related information is recorded, including the detail level bounding sphere and detail level error of the micro-mesh, and the detail level bounding sphere and detail level error of the micro-mesh's parent micro-mesh; Each frame uses the camera, gaze point information, and micro-mesh related information to select micro-meshes at each level of detail; Render all selected micromeshes each frame using the mesh shader.

2. The method for foveated rendering based on mesh detail level according to claim 1, characterized in that: The mesh to be rendered is divided and simplified to obtain micro-meshes at various levels of detail, and the micro-mesh related information is recorded, including: According to the connection relationship between triangles, the triangles in the mesh are clustered to obtain a set of micro-grids at the 0th level of detail. All microgrids in the microgrid set of the current level of detail are clustered according to the connection relationship between triangles to obtain multiple microgrid groups, and all microgrids contained in each microgrid group are merged, simplified, and clustered to obtain a microgrid set of the next level of detail. The process is iterated until only one microgrid is left in the microgrid set to obtain microgrids of each level of detail, and the detail level bounding sphere and detail level error of the microgrid are recorded.

3. The method for foveated rendering based on mesh detail level according to claim 2, characterized in that: All microgrids contained in each microgrid group are merged, simplified, and clustered to obtain a microgrid set at the next level of detail, including: Merge all microgrids contained in each current microgrid group to obtain a merged grid; The merged mesh is simplified with the goal of reducing the number of triangles. During the simplification process, the boundary edges of the micro-mesh group need to be locked to ensure that the simplified mesh has the same boundary as the mesh before simplification. The simplified mesh is obtained, and the error introduced in the simplification process is recorded at the same time. Cluster all triangles in the simplified mesh to get the next level of detail level micro-grid set. Each micro-grid of the next level of detail level is called the parent micro-grid of each micro-grid in the current micro-grid group. The detail level bounding sphere and detail level error of the parent micro-grid are recorded. For each parent micro-grid, its detail level bounding sphere is recorded as the union of the detail level bounding spheres of all its child micro-grids, and its detail level error is the maximum value of the maximum detail level error of all child micro-grids and the error introduced by the simplification process.

4. The method for foveated rendering based on mesh detail level according to claim 3, characterized in that: After segmentation and simplification, the mesh to be rendered is constructed as a tree-shaped directed acyclic graph containing all levels of detail. Each node in the graph corresponds to a microgrid, and all nodes in each layer are all microgrids of a level of detail. The directed edges between nodes represent parent-child relationships. Connections from parent nodes to child nodes indicate that the parent node is simplified from the microgrid group where the child node is located. The leaf nodes of the graph represent all microgrids at the 0th level of detail. The root node of the graph represents the microgrid at the highest level of detail, and microgrid related information is recorded at each node.

5. The method for foveated rendering based on mesh detail level according to claim 1, characterized in that: Each frame uses the camera, gaze point information, and micromesh related information to select micromeshes at each level of detail, including: A compute shader is used to select micro-meshes at each level of detail in parallel. Specifically, the screen error projected onto the screen is calculated based on the camera and gaze point information and the micro-mesh related information. The screen error is compared with the threshold error to select the micro-mesh, and the selected micro-mesh is added to the rendering queue.

6. The method for foveated rendering based on mesh detail level according to claim 5, characterized in that: For each level of detail of the micro-mesh, the screen error projected onto the screen is calculated based on the camera and gaze point information and the micro-mesh related information, and the screen error is compared with the threshold error to select the micro-mesh, including: Calculate the first screen error of the projection of the micro-mesh onto the screen based on the camera and gaze point information and the detail level bounding sphere and detail level error of the micro-mesh itself; Calculate a second screen error of the projection of the micromesh's parent micromesh onto the screen based on the camera and gaze point information and the level-of-detail bounding sphere and level-of-detail error of the parent micromesh; When the second screen error is greater than a threshold error, and when the first screen error is less than or equal to the threshold error, the micro grid is selected.

7. The foveated rendering method based on mesh detail level according to claim 1 or 6, characterized in that: The first screen error of the microgrid to screen projection and the second screen error of the parent microgrid to screen projection are calculated in the same way: The maximum projection error of the detail level error projected onto the screen is calculated according to the detail level bounding sphere and the camera parameters, the screen space position of the gaze point is determined according to the gaze point information, and the maximum projection error is attenuated based on the screen space position of the gaze point to obtain the attenuated error. The attenuated error is converted to pixels to obtain the screen error for comparison with the threshold error.

8. A foveated rendering device based on mesh detail level, characterized in that: include: A detail level division module is used to divide and simplify the mesh to be rendered to obtain micro-grids of various levels of detail levels, and record micro-grid related information, including the detail level bounding sphere and detail level error of the micro-grid, and the detail level bounding sphere and detail level error of the micro-grid's parent micro-grid; A microgrid selection module is used to select microgrids at each level of detail using camera, gaze point information and microgrid related information in each frame; A rendering module that is used to render all selected micromeshes each frame using a mesh shader.

9. A computing device comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the one or more processors execute the executable code, they are used to implement the foveated rendering method based on mesh detail level as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the foveated point rendering method based on grid detail level described in any one of claims 1-7 is implemented.

Images