Method for generating optimum tree for rendering or streaming 3D model, host, and non-temporary storage circuit

Abstract

To provide a method for generating an optimum tree for efficiently and quickly rendering or streaming a 3D model of cross reality (XR) called augmented reality (AR), virtual reality (VR), and mixed reality (MR), a host and a non-temporary storage circuit.SOLUTION: A method includes acquiring a complete tree of a 3D model, acquiring a quality value a node of the complete tree of the 3D model, acquiring a visibility value of the node of the complete tree of the 3D model, determining an optimum tree of the 3D model on the basis of the complete tree, the quality value, and the visibility value, and rendering the 3D model on the basis of the optimum tree.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a method for generating an optimal tree for rendering or streaming a 3D (3-dimensional) model. In particular, the present invention relates to a method for generating an optimal tree for rendering or streaming a 3D model, a host, and a non-transitory memory circuit.

Background Art

[0002] Technologies related to cross-reality (XR) such as Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) are constantly being developed. AR technology enables virtual elements to be brought into the real world for the user. VR technology enables the user to enter a completely new virtual world to experience a different life. MR technology fuses the real world and the virtual world. Furthermore, visual content, audio content, or content of other senses can be provided to the user via the web to provide a complete immersive experience for the user.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The present invention aims at a method for generating an optimal tree for rendering or streaming a 3D model, a host, and a non-transitory memory circuit for efficiently and quickly providing an XR 3D model to a user.

Means for Solving the Problems

[0004] The present invention provides a method for generating an optimal tree for rendering or streaming a 3D model. The method includes obtaining a complete tree of the 3D model, obtaining a quality value of a node of the complete tree of the 3D model, obtaining a visibility value of a node of the complete tree of the 3D model, determining an optimal tree of the 3D model based on the complete tree, the quality value, and the visibility value, and rendering the 3D model based on the optimal tree.

[0005] The present invention provides a host. The host includes a storage circuit and a processor. The storage circuit is configured to store program code. The processor is coupled to the storage circuit and is configured to access the program code to obtain a complete tree of a 3D model, obtain a quality value of a node of the complete tree of the 3D model, obtain a visibility value of a node of the complete tree of the 3D model, determine an optimal tree of the 3D model based on the complete tree, the quality value, and the visibility value, and render the 3D model based on the optimal tree.

[0006] The present invention provides a non-transitory storage circuit. The non-transitory storage circuit is configured to store program code, and the program code is configured to cause a processor to obtain a complete tree of a 3D model, obtain a quality value of a node of the complete tree of the 3D model, obtain a visibility value of a node of the complete tree of the 3D model, determine an optimal tree of the 3D model based on the complete tree, the quality value, and the visibility value, and render the 3D model based on the optimal tree.

Effects of the Invention

[0007] Based on the above, an XR's 3D model is provided to the user efficiently and quickly.

[0008] To make the above-described content easier to understand, some embodiments will be described in detail below with reference to the drawings.

Brief Description of the Drawings

[0009] To provide a further understanding of the present invention, the accompanying drawings are included and incorporated herein to form a part thereof. The drawings represent exemplary embodiments of the present invention and serve to explain the principles of the present invention together with the specification.

[0010]

Figure 1A

Figure 1B

Figure 2A

Figure 2B

Figure 3A

Figure 3B

Figure 4A

Figure 4B

Figure 4C

Figure 5

Figure 6

Figure 7

MODE FOR CARRYING OUT THE INVENTION

[0011] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals will be used throughout the drawings and the specification to refer to the same or like parts.

[0012] Throughout the specification and the appended claims, specific terms are used to refer to specific components. Those skilled in the art should understand that electronic device manufacturers may refer to the same components by different names. Here, it is not intended to distinguish components with different names but the same function. In the following description and claims, terms such as "comprising" and "including" are open-ended terms and should be interpreted as "including but not limited to...".

[0013] To provide an immersive experience for users, technologies related to cross-reality (XR), such as augmented reality (AR), virtual reality (VR), and mixed reality (MR), are constantly being developed. AR technology enables users to bring virtual elements into the real world. VR technology enables users to enter a completely new virtual world to experience different lives. MR technology combines the real world and the virtual world. Furthermore, to provide a complete immersive experience for users, visual content, audio content, or other sensory content can be provided to the user's terminal device (e.g., a smart device or a computing device) via the web, or can be read locally from the memory of the user device.

[0014] XR content such as 3D geometry data of a 3D model is displayed to a user (e.g., by streaming 3D geometry data on the web or by locally loading 3D geometry data), and it is important that the user does not have to wait a long time until it is placed in the virtual world. That is, it is important that the size of the 3D geometry data is preferably as small as possible to enable faster data transmission or loading. In other words, it is important that the process for displaying 3D geometry data is sufficiently performant so as not to cause a noticeable delay to the user while the user device is acquiring the data.

[0015] For example, while the user is viewing 3D geometry data via a user device, a conventional method of loading a virtual scenario in the virtual world reads all the data at startup (resulting in a long loading time) or is often loaded in the data when the user's user representative object (e.g., an avatar) approaches a virtual object. For this reason, the user experience may be interrupted by the waiting time for loading. In view of the above, those skilled in the art are seeking to provide an efficient and rapid method for providing 3D geometry data to a user device.

[0016] The present invention provides a novel method for rendering or streaming a 3D model. An optimization tree is determined based on the quality value and visibility value of each node of the 3D model. The 3D model can be rendered based on the optimization tree rather than the complete tree of the 3D model, and the amount of 3D geometry data can be reduced. For this reason, even if there are limitations in the Internet connection speed or the computing power of the user device, the 3D model can be displayed to the user without or with almost no loading waiting time, thereby improving the user experience. Details of the method will be described below with reference to the accompanying drawings.

[0017] FIG. 1A is a schematic diagram of a tree structure of a 3D model according to one embodiment of the present invention. FIG. 1B is a schematic diagram of a rendering result of a 3D model according to one embodiment of the present invention. Note that FIGS. 1A and 1B are simplified for the clarification and better understanding of the inventive concept of the present invention. That is, although it is illustrated that the tree structure of the 3D model includes two levels and each level of the tree structure includes one node or two nodes, the tree structure may include more levels and each level of the tree structure may include more nodes. However, the present invention is not limited thereto.

[0018] First, referring to FIG. 1A, in one embodiment, a tree structure converted into a tree structure of some geometries by a converter as shown in FIG. 1A is provided, and each level in the tree structure represents a complete model of the 3D model with different qualities by one or more nodes. Each node may represent some geometries (e.g., meshes) of the 3D model. At the maximum width point in the tree structure (i.e., the bottom in FIG. 1A), level 2 of the tree structure may include four nodes (i.e., Node0_0, Node0_1, Node1_0, Node1_1). The original quality of the 3D model may be represented by the geometries formed by these four nodes. On the other hand, at the root of the tree structure (i.e., the top in FIG. 1A), level 0 of the tree structure includes only one node, which is called the root node. The 3D model may be represented by one simplified geometry formed by the root node.

[0019] Reference is also made here to FIG. 1B. Referring to FIGS. 1A and 1B, in one embodiment, the tree structure of FIG. 1A corresponds to the rendering results at each level of FIG. 1B. For example, the rendering result by rendering the 3D model at the root node at level 0 of the tree structure may be as shown at the top of FIG. 1B, which has only the simplified features of the 3D model. Further, the rendering result by rendering the 3D model at two nodes (i.e., node 0, node 1) at level 1 of the tree structure may be as shown in the middle of FIG. 1B, which has more features than the rendering result at level 0. Further, the rendering result by rendering the 3D model at four nodes at level 2 of the tree structure may be as shown at the bottom of FIG. 1B, which has the original quality of the 3D model. In addition, by displaying the rendering result via the user device, the 3D model in the virtual world can be displayed to the user with different qualities.

[0020] To provide a more preferable user experience, when the requirement for 3D model display is determined, the 3D model may be provided starting from the root node so as to be immediately rendered, and the details of the 3D model may be gradually improved by rendering more nodes from higher levels. Further, different parts of the tree structure may be loaded depending on the position where the 3D model is viewed. For example, for the piece of the 3D model that is close to the position or clearly visible in the field of view, the high-quality part of the tree structure may be loaded. Conversely, for the piece of the 3D model that is far from the position or difficult to see, the low-quality part of the tree structure may be loaded.

[0021] In this way, it is possible to reduce the amount of network bandwidth required to stream the 3D model, reduce the amount of geometry required for the rendering engine to create the final rendering, and at the same time optimize the apparent quality of the 3D rendering.

[0022] Figures 2A and 2B are schematic diagrams of display scenarios according to some embodiments of the present invention. Figure 2A shows a display scenario of two 3D models in a virtual world when two 3D models such as model M1 and model M2 have just been loaded during the initial loading process at startup. Figure 2B shows a display scenario of the 3D models after the optimization process after being loaded based on the position of the viewer point VP. In one embodiment, the viewer point VP may be a virtual camera in the virtual world or an avatar in the virtual world. In addition, the distance from the viewer point VP to the 3D model may be determined based on the position of the viewer point VP and the viewing angle from the viewer point VP. However, the present invention is not limited thereto.

[0023] Referring to Figure 2A, when models M1 and M2 have just been loaded during the initial loading process at startup, both models M1 and M2 are rendered at the lowest detail, for example, level 0. Note in Figure 2A that the viewer point VP is on the right side. That is, model M2 is close to the viewer point VP and model M1 is far from the viewer point VP. Since the distance from the viewer point VP to model M1 is relatively long, the user probably does not pay much attention to the detail of model M1. That is, even if model M1 is rendered at the lowest detail, it probably will not have a negative impact on the user experience. In other words, model M1 is already in an optimal state.

[0024] On the other hand, since the distance from the viewer point VP to model M2 is relatively short, the user probably pays more attention to the detail of model M2. That is, since model M2 is rendered at the lowest detail, it probably will have a negative impact on the user experience. In other words, model M1 is not yet in an optimal state. Therefore, in order to reduce the negative impact brought by model M2, the detail of model M2 may be improved. That is, when the initial loading process of model M2 is completed, model M2 may continue to be rendered for more detail.

[0025] Referring to FIG. 2B here, when the initial loading processes of model M1 and model M2 are completed, model M1 may remain at a low level of detail, for example, level 0, and model M2 may continue to be rendered with more detail, for example, level 2. However, the present invention is not limited thereto. That is, in order to provide a more preferable viewing experience for the user, model M2 closer to the view point VP may be displayed with more detail. On the other hand, in order to reduce the waiting time for loading, model M1 farther from the view point VP may be displayed with less detail. In other words, both model M1 and model M2 are in an optimal state. In this way, a good viewing experience can be provided without or almost without loading waiting time, thereby improving the user experience.

[0026] FIGS. 3A and 3B are schematic diagrams of a display scenario according to one embodiment of the present invention. FIG. 3A shows a display scenario of two 3D models when the position of the view point VP changes during runtime. FIG. 3B shows a display scenario of two 3D models after an optimization process during runtime based on the updated position of the view point VP.

[0027] Referring to FIG. 3A, after startup, during runtime, the view point VP may change from the right side to the left side. That is, model M1 gets closer to the view point VP and model M2 gets farther from the view point VP. Since the distance from the view point VP to model M2 is relatively long, the user may probably not pay much attention to the detail of model M2. However, model M2 is rendered with relatively more detail. That is, there may be a waste of computing power for continuing to display model M2 and the like because the detail is too much. In other words, model M2 is no longer in an optimal state.

[0028] On the one hand, since the distance from the viewer point VP to the model M1 is relatively short, the user will probably pay more attention to the details of the model M1. However, the model M1 is rendered with relatively few details. That is, the details are not sufficient and may have a negative impact on the user experience. In other words, the model M1 is not yet in an optimal state either.

[0029] Briefly, the position of the viewer point VP is changing, and the optimal states of the models M1 and M2 may change accordingly. Therefore, in order to provide a more preferable user experience, the updated optimal states of the models M1 and M2 may be determined based on the updated position of the viewer point VP.

[0030] Referring to FIG. 3B, during runtime, after the position of the viewer point VP has changed, the models M1 and M2 may be rendered again based on the updated position of the viewer point VP. For example, the model M1 may be rendered with more details this time instead of fewer details. On the other hand, the model M2 may be rendered with fewer details this time instead of more details. That is, both the models M1 and M2 can be made optimal again. Therefore, the user experience can be improved.

[0031] Note that in order to clarify the inventive concept of the present invention, the models M1 and M2 are illustrated as two separate 3D models. However, in one aspect of the present invention, the models M1 and M2 may be regarded as different parts of a single 3D model. In one embodiment, one of the models M1 and M2 is visible from the viewer point VP (for example, facing the viewer point VP), and the other of the models M1 and M2 is not visible from the viewer point VP. That is, even for the same 3D model, each part of the 3D model may be rendered with different levels of detail based on the position of the viewer point VP, thereby improving the user experience.

[0032] As a point to be noted, if the Internet connection speed on the web or the computing power of the user device were infinite, it could be said that it would be preferable to render all 3D models in the virtual world with as much detail as possible. However, in most cases, there are limitations to the Internet connection speed or the computing power of the user device. Therefore, in order to ensure a good user experience, restrictions may be imposed on the 3D model accordingly.

[0033] For example, as described above, in order to save time under the limitation of the Internet connection speed or computing power, 3D models far from the viewer point VP may be rendered with less detail. Furthermore, in reality, distant objects are usually unclear to the user. Therefore, rendering distant models with less detail can not only save time under the limitation of the Internet connection speed or computing power, but also make the user experience more realistic.

[0034] In addition, in order to provide a more preferable user experience, the Internet connection speed or computing power may also be considered during the rendering process of the 3D model. For example, in order to ensure that the loading waiting time of the 3D model is lower than a predetermined value or close to zero, the maximum size of the 3D geometry data may be determined based on the Internet connection speed or computing power. That is, the 3D model may be rendered starting from the root node, and the detail of the 3D model may be gradually improved by rendering more nodes from a higher level until the size of the 3D geometry data does not exceed the maximum size but approaches the maximum size. In other words, the 3D model can provide a good viewing experience without or almost without waiting time, and the state of the 3D model can be called the optimal state.

[0035] In one embodiment, referring to FIGS. 1A to 3B, the optimal state of model M1 or model M2 can be represented by a tree structure of a 3D model, which may be referred to as an optimal tree. The optimal tree may be obtained by simplifying the complete tree structure (which may be referred to as a complete tree) of the 3D model based on the relationship between the viewer point VP and model M1 or model M2. However, the present invention is not limited thereto. Also, the tree structure of model M1 or model M2 with the lowest detail when model M1 or model M2 is just loaded may be referred to as an initial tree.

[0036] FIG. 4A is a schematic diagram of an optimal tree of a 3D model according to one embodiment of the present invention. In FIG. 4A, when the 3D model is in an optimal state, the tree structure of the 3D model may be referred to as an optimal tree OT. The optimal tree OT may be obtained by simplifying the complete tree of the 3D model.

[0037] For example, the complete tree of the 3D model may include five levels, and each node from level 0 to level 3 may include four nodes. That is, the complete tree may include 1 + 4 + 4×4 + 4×4×4 + 4×4×4×4 + 4×4×4×4×4 = 1365 nodes. The 1365 nodes of the complete tree may be simplified based on the current position of the viewer point VP to obtain the nodes of the optimal tree OT. As shown in FIG. 4A, the optimal tree OT may include 1 + 4 + 4×2 + 4×2 = 21 nodes instead of 1365 nodes. However, the present invention is not limited thereto.

[0038] In one embodiment, the optimal tree OT may include nodes 0, 1, 2, and 3 at level 1. Nodes 0 and 2 may represent pieces that are far from the viewer point VP. Therefore, in order to reduce the size of the 3D geometry data of the 3D model, no further details need to be rendered for nodes 0 and 2. On the other hand, nodes 1 and 3 may represent pieces that are close to the viewer point VP. Therefore, in order to ensure a more favorable viewing experience, more details may be rendered for nodes 1 and 3.

[0039] Similarly, since nodes 3_1 and 3_3 may represent pieces that are even closer to the viewer point VP, more details may be rendered for nodes 3_1 and 3_3. On the other hand, since nodes 1_0, 1_1, 1_2, 1_3, 3_0, and 3_2 may represent pieces that are not close to the viewer point VP, no further details need to be rendered for nodes 1_0, 1_1, 1_2, 1_3, 3_0, and 3_2.

[0040] In one embodiment, the 3D geometry data of the 3D model may be stored remotely on a server or locally in memory. At the start of loading the 3D model, the user device may receive only the root node from the server or memory. After the optimal tree OT is determined, the user device may be configured to request the optimal data regarding the optimal tree OT from the server or memory. Further, the user device may receive the optimal data from the server or memory and be configured to render the 3D model based on the optimal tree and the optimal data. In this way, only the necessary data is transmitted over the web or internally, thereby improving the user experience.

[0041] Figures 4B and 4C are schematic diagrams of the optimization process of the tree structure of a 3D model according to some embodiments of the present invention. Figure 4B may correspond to the conversion process from Figure 2A to Figure 2B when models M1 and M2 have just been loaded during the loading process at startup. Figure 4C may correspond to the conversion process from Figure 3A to Figure 3B when the position of the viewer point VP changes during runtime.

[0042] Referring to Figure 4B, when the 3D model has just been loaded, the 3D model may be rendered with relatively low detail (e.g., including only two levels of nodes) to provide an immediate result, and the tree structure representing the 3D model during that time may be called the initial tree IT (or may be called the rendered tree). To make the tree structure representing the 3D model the optimal tree OT based on the position of the viewer point VP, the 3D model may be rendered with more detail (e.g., including four levels of nodes). That is, the tree structure representing the 3D model may be converted from the initial tree IT to the optimal tree OT. In this way, after the conversion, since the tree structure representing the 3D model is in an optimal state, the user experience can be improved.

[0043] Now referring to Figure 4C, during runtime, when the position of the viewer point VP changes, the optimal tree OT may also need to change accordingly. That is, due to the change in the relationship between the viewer point VP and the 3D model, the tree structure representing the 3D model may be converted from the first optimal tree OT1 (also called the rendered optimal tree) to the second optimal tree OT2 (also called the current optimal tree). In this way, after the conversion, since the tree structure representing the 3D model is in an optimal state, the user experience can be improved.

[0044] In one embodiment, the 3D model may include a plurality of meshes, and each mesh may include a plurality of triangles. In the tree structure of the 3D model, the nodes of the tree structure represent the meshes of the 3D model. That is, each node of the tree structure represents a plurality of triangles. Therefore, in one embodiment, the hard limit on the amount of triangles to be loaded and displayed for displaying the 3D model may be referred to as the "triangle budget". In one embodiment, there may be limitations on the Internet connection speed or computing power. Therefore, when there are limitations on the Internet connection speed or computing power, the triangle budget may be used as a limitation on the 3D model. In other words, the value of the triangle budget may be determined based on the Internet connection speed or computing power. However, the present invention is not limited thereto.

[0045] To optimally allocate the triangle budget to each node of the 3D model, it is essential to establish an evaluation mechanism for each part of the 3D model. For example, referring to FIGS. 4A, 4B, and 4C, each node of the optimal tree OT or the initial node IT may have a score. A low score indicates that the node has less detail, and a high score indicates that the node has more detail. That is, the score can be used to determine whether the node of the tree structure is in an optimal state. Further, when comparing the scores of the nodes in the tree structure, the score may be used to determine a group of nodes in the tree structure that optimizes the visual quality while being restricted by the triangle budget.

[0046] In one embodiment, the score of a node may be defined as the quality of the node divided by the visibility of the node with respect to the viewer point VP. However, the present invention is not limited thereto. That is, the score of the node may be expressed by the following formula. Score = Quality / Visibility

[0047] First, before finding the score of a node in the tree structure, it is important to define an indicator for the quality of the node in the tree structure with respect to the original quality of the 3D model (e.g., divided by it). For example, it is acceptable to define a quality value for the node, and the quality value may linearly correspond to the quality of the node. In one example, a quality value of 1 may represent the complete original quality of the 3D model, and a quality value of 0.5 may represent half of that quality. However, the present invention is not limited thereto. In other words, the quality value of the node may be used to measure the quality of the decimated geometry with respect to the original quality, and the quality value of the node may be used to compare the quality of different nodes with each other.

[0048] In one embodiment, one method used to determine the quality value of a node may be to use the number of triangles of the node with respect to (e.g., divided by) the number of triangles in the original mesh (i.e., the complete mesh) represented by the node. In another embodiment, another method used to determine the quality value may be to use the amount of texels in the mesh compared to (e.g., divided by) the amount of texels in the original mesh. In yet another embodiment, these two methods may be combined, or other indicators may be used to determine the quality value. That is, the present invention does not limit how the quality value is determined as long as the quality value of the node is used to compare the quality of different nodes with each other.

[0049] Next, another important element of the formula is visibility. Visibility may be regarded as how visible a node is. In one embodiment, the visibility value of visibility may be derived from the surface area of the mesh (used as an approximation of the size of the node) divided by the distance from the viewer point VP to the node. That is, for two given nodes with the same distance, the node with a larger surface area will obtain a higher visibility value. Similarly, when the surface areas of both nodes are the same, the closer node will obtain a higher visibility value. In other words, the visibility value may be inversely proportional to the distance. However, the present invention is not limited thereto.

[0050] Next, based on these two values, the score of each node in the tree may be calculated and used to determine the optimal tree OT of the 3D model. For example, as the quality of the node increases, the score increases. Alternatively, as the node becomes smaller (lower visibility) on the screen of the user device, the score also increases. This is reasonable as the score represents the quality of the 3D model on the screen. When the 3D model moves far from the view point VP, the same 3D model may be represented by fewer pixels, and thus the quality is higher. In some embodiments, to fine-tune the behavior of the above-described algorithm (which may be referred to as the "scoring algorithm") used to determine the score, several additional parameters that affect the additional scaling on the distance element (e.g., from the occlusion culling algorithm or ray intersection tests using Halton sequences) may be used. However, the present invention is not limited thereto.

[0051] At this point, each node has a score, which is proportional to the quality obtained by the node from the viewer point VP. Based on the scores of the nodes in the tree structure, an optimal group of nodes for display starting from the root node may be determined. Further, when the amount of triangles to be rendered is still within the triangle budget, the quality of the node with the lowest score may be continuously improved. That is, the node with the lowest score may be selected for improvement. Thus, the appearance of the 3D model will be the best, and a balance can be achieved in the visual quality with respect to the distance of the nodes. For example, as shown in FIGS. 4A, 4B, and 4C, when the total number of triangles in the tree structure of the 3D model is still within the triangle budget, nodes with relatively low scores may be continuously rendered to improve the viewing experience. In one embodiment, different parts of the geometry represented by triangles may be assigned triangles. That is, the triangles may be evenly distributed in the tree structure so that the viewer can easily recognize the geometry. In this way, the overall comprehensive highest score for all virtual scenarios in the virtual world can be achieved efficiently and quickly, thereby improving the user experience.

[0052] In one embodiment, to further improve the above process, an occlusion culling algorithm may be used. The occlusion culling algorithm is a method of accelerating rendering by not rendering objects that are not visible from the viewer point VP. Unity and PlayCanvas are different game engines that both use the occlusion culling algorithm. Therefore, in both game engines, it is possible to implement an occlusion culling algorithm that effectively uses the method proposed here.

[0053] First, in the Unity engine, to implement the occlusion culling algorithm using the method proposed here, a Halton sequence may be adopted to generate a ray from the viewer point VP to the farthest clip plane. To manage the computational load, the ray casting task is split into frames, and Halton sequence points are distributed across the frames. For example, 15,360 points are divided into 60 frames, each having 256 rays. When a ray intersects an object, the hit is gradually displayed.

[0054] In occlusion culling, an important adjustment involves prioritizing the visibility of 3D models based on these hits. When a 3D model is hit, the 3D model is prioritized in rendering. That is, when a node of the 3D model is hit by a ray from the viewer point VP, the visibility value of the node may be increased. Further, after a predetermined time has elapsed without new hits, the mesh renderer of the 3D model is invalidated, and the occluded 3D model can be effectively removed. This adapted approach optimizes occlusion culling by dynamically adjusting the rendering priority based on the real-time visibility queue provided by ray casting guided by the Halton sequence.

[0055] Second, in the PlayCanvas engine, to implement the occlusion culling algorithm using the method proposed here, it is necessary to note that the occlusion culling algorithm of the web player utilizes occlusion queries, which are features of the graphics API including WebGL2. Occlusion queries are performed at intervals to determine whether a node is visible. Individual queries are spread out so that not all individual queries occur on one frame.

[0056] In occlusion culling, the bounding box of the 3D model or the actual mesh of the 3D model is used for the occlusion query. If no pixels of the geometry are rendered, the node is considered occluded and hidden. When a node is hidden, the frequency of the query may be doubled so that the node can reappear more quickly. That is, when a node of the 3D model is hidden, the visibility value of the node may be lowered to zero. Since the occlusion query is "rendered" after the World Opaque layer, there is an existing depth buffer for comparison. The occlusion query does not write to the color or depth buffer, and the hidden nodes are invisible to the user and do not affect the layers to be subsequently rendered.

[0057] FIG. 5 is a flowchart of a method for generating an optimal tree for rendering or streaming a 3D model according to one embodiment of the present invention. Referring to FIG. 5, a method 500 for generating an optimal tree for rendering or streaming a 3D model may include step S510, step S520, step S530, step S540, and step S550.

[0058] In step S510, a complete tree of the 3D model may be obtained, for example, remotely from a server or locally from memory. In step S520, the quality value of the nodes of the complete tree of the 3D model may be obtained, for example, based on the number of triangles of the nodes. In step S530, the visibility value of the nodes of the complete tree of the 3D model may be obtained, for example, based on the position of the viewer point VP. In step S540, an optimal tree OT of the 3D model as shown in FIGS. 4A, 4B, and 4C may be determined based on the complete tree, the quality value, and the visibility value. In step S550, the 3D model may be rendered based on the optimal tree OT. Note that the details of the implementation of method 500 can be referred to the descriptions of FIGS. 4A, 4B, and 4C, and will not be repeated here.

[0059] In this way, it becomes possible to reduce the amount of network bandwidth required to stream the 3D model, reduce the amount of geometry required for the rendering engine to create the final rendering, and at the same time optimize the apparent quality of the 3D rendering.

[0060] FIG. 6 is a flowchart of a method for converting a rendered tree to an optimal tree according to one embodiment of the present invention. Referring to FIG. 6, a method 600 for converting a rendered tree to an optimal tree may include step S610, step S620, step S630, and step S640.

[0061] In step S610, a rendered tree of the 3D model may be obtained. For example, the rendered tree may be the initial tree IT of FIG. 4B or the first optimal tree OT1 of FIG. 4C, but the present invention is not limited thereto. In step S620, an optimal tree OT of the 3D model may be determined. For example, the optimal tree OT may be the optimal tree OT of FIG. 4B or the second optimal tree OT2 of FIG. 4C, but the present invention is not limited thereto. In step S630, the rendered tree of the 3D model may be modified, for example, converted to the optimal tree OT based on the current position of the viewer point VP as in the conversion process shown in FIG. 4B or FIG. 4C. In step S640, the 3D model may be rendered based on the optimal tree OT. Note that the details of the implementation of method 600 can be referred to the descriptions of FIGS. 4B and 4C, and will not be repeated here.

[0062] In this way, even if the relationship between the viewer point VP and the 3D model changes, the 3D model can be provided to the user efficiently and quickly.

[0063] FIG. 7 is a schematic diagram of a host according to one embodiment of the present invention. In various embodiments, host 700 may be any smart device and / or computer device. In some embodiments, host 700 may be any electronic device capable of providing a reality service (e.g., an AR / VR / MR service, etc.). In some embodiments, host 700 may be a computer and / or a server, and host 700 may provide the calculated result (e.g., a rendering result) to another external display device, and the other external display device may display the calculated result to the user. However, the present invention is not limited thereto.

[0064] In FIG. 7, host 700 includes a (non-transitory) memory circuit 710 and a processor 720. The memory circuit 710 may be one or a combination of a fixed or mobile random access memory (RAM), a read only memory (ROM), a flash memory, a hard disk, or other similar devices, and is configured to store a plurality of modules and / or program codes executed by the processor 720.

[0065] The processor 720 is coupled to the memory circuit 710, and the processor 720 may be, for example, a general-purpose processor, a dedicated processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, other types of integrated circuits (ICs), a state machine, etc.

[0066] In one embodiment, the processor 720 may be configured to access modules and / or program code stored in the memory circuit 710 to implement method 500 or method 600, but the present invention is not limited thereto. Additionally, details of the implementation of the host 700 or the memory circuit 710 can be referred to the descriptions of FIGS. 1 to 6, which will not be repeatedly described here.

[0067] In summary, according to method 500, host 700, and memory circuit 710, an optimal tree can be determined based on the quality value and visibility value of each node in the 3D model. The 3D model can be rendered based on the optimal tree rather than the complete tree of the 3D model, and the amount of 3D geometry data can be reduced. Therefore, even if there are limitations in the Internet connection speed or the computing power of the user device, the 3D model can be displayed to the user without or with almost no waiting time, thereby improving the user experience.

[0068] It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed embodiments without departing from the scope or spirit of the present invention. Considering the above, the present invention is intended to embrace modifications and variations as long as they are within the scope of the claims and their equivalents.

Industrial Applicability

[0069] The method, host, and non-transitory memory circuit for generating an optimal tree for rendering or streaming a 3D model of the present invention can be used in applications that require efficiently and quickly providing the 3D model to the user.

Explanation of Reference Numerals

[0070] 500, 600: Method 700: Host 710: Memory Circuit 720: Processor IT: Initial Tree M1, M2: Model OT: Optimal Tree OT1: First Optimal Tree OT2: Second Optimal Tree S510, S520, S530, S540, S550, S610, S620, S630, S640: Steps VP: Viewpoint

Claims

1. A method for generating an optimal tree for rendering or streaming a 3D model, comprising: obtaining a complete tree of the 3D model; obtaining a quality value of a node of the complete tree of the 3D model; obtaining a visibility value of the node of the complete tree of the 3D model; determining an optimal tree of the 3D model based on the complete tree, the quality value, and the visibility value; rendering the 3D model based on the optimal tree. A method.

2. determining the quality value based on the number of triangles or the amount of texels of the node; determining the visibility value based on the position of a viewer point. The method according to claim 1.

3. determining the quality value based on the number of triangles or the amount of texels of the node with respect to the number of triangles or the amount of texels of the original mesh represented by the node. The method according to claim 1.

4. determining the visibility value based on the surface area of the node divided by the distance from the position of the viewer point to the node. The method according to claim 1.

5. determining a score by dividing the quality value by the visibility value; determining the optimal tree based on the score. The method according to claim 1.

6. improving the quality of the node according to whether the total number of triangles of the tree structure of the 3D model is within a triangle budget. The method according to claim 1.

7. requesting optimal data related to the optimal tree from a server or a memory according to the determination of the optimal tree; receiving the optimal data from the server or the memory; rendering the 3D model based on the optimal tree and the optimal data. The method according to claim 1.

8. converting a rendered tree to the optimal tree based on the position of a viewer point. The method according to claim 1.

9. rendering an initial tree during an initial loading process at startup; converting the initial tree to the optimal tree based on the position of a viewer point. The method according to claim 1.

10. ​ ​ ​ ​ ​ ​ ​ ​ ​ During runtime, convert the rendered optimal tree based on the position of the viewer point to the current optimal tree according to the change in the position of the viewer point. Further comprising The method according to claim 1.

11. Raise the visibility value of the node in response to the node of the 3D model being hit by a ray from the viewer point. Further comprising The method according to claim 1.

12. Lower the visibility value of the node to zero in response to the node of the 3D model being hidden. Further comprising The method according to claim 1.

13. A storage circuit configured to store program code, Coupled to the storage circuit, Obtain a complete tree of a 3D model, Obtain the quality value of the nodes of the complete tree of the 3D model, Obtain the visibility value of the nodes of the complete tree of the 3D model, Determine an optimal tree of the 3D model based on the complete tree, the quality value, and the visibility value, Render the 3D model based on the optimal tree A processor configured to access the program code Including Host.

14. The processor Determine the quality value based on the number of triangles or the amount of texels of the node, Determine the visibility value based on the position of the viewer point Further configured as The host according to claim 13.

15. The processor Determine the quality value based on the number of triangles or the amount of texels relative to the number of triangles or the amount of texels of the original mesh represented by the node Further configured as The host according to claim 13.

16. The processor Determine the visibility value based on the surface area of the node divided by the distance from the position of the viewer point to the node Further configured as The host according to claim 13.

17. The processor Determine a score by dividing the quality value by the visibility value, Determine the optimal tree based on the score Further configured as The host according to claim 13.

18. The processor Improve the quality of the node in response to the total number of triangles of the tree structure of the 3D model being within the triangle budget. Further configured as The host according to claim 13.

19. The processor Upon determination of the optimal tree, request optimal data regarding the optimal tree from the server or the memory, Receive the optimal data from the server or the memory, Render the 3D model based on the optimal tree and the optimal data and further configured to be, The host according to claim 13.

20. configured to store program code, wherein the program code causes a processor to, acquire a complete tree of a 3D model, acquire quality values of nodes of the complete tree of the 3D model, acquire visibility values of the nodes of the complete tree of the 3D model, determine an optimal tree of the 3D model based on the complete tree, the quality values, and the visibility values, render the 3D model based on the optimal tree and configured to be, a non-transitory storage circuit.

Images