Method, host and non-transitory storage circuit for optimizing tree of 3D model

Abstract

The invention provides a method, a host and a non-transitory storage circuit for tree optimization of a 3D model. The method is used for generating an optimization tree for rendering or streaming a 3D model, and comprises the following steps: obtaining a complete tree of the 3D model; obtaining quality values of nodes of the complete tree of the 3D model; obtaining visibility values of nodes of the complete tree of the 3D model; determining an optimization tree of the 3D model based on the complete tree, the quality value, and the visibility value; and rendering the 3D model based on the optimization tree.

Description

Technical Field

[0001] The present invention relates to a method for optimizing a tree of a 3D model; specifically, the present disclosure relates to a method, a host, and a non-transitory storage circuit for generating an optimized tree for rendering or streaming a 3D model. Background Art

[0002] In order to bring an immersive experience to users, technologies related to extended reality (XR), such as augmented reality (AR), virtual reality (VR), mixed reality (MR), etc., are constantly being developed. AR technology enables users to bring virtual elements into the real world. VR technology allows users to enter a brand-new virtual world and experience a different life. MR technology combines the real world and the virtual world. In addition, in order to bring a completely immersive experience to users, visual content, audio content, or other sensory content can be provided to users via a network. Summary of the Invention

[0003] The present disclosure provides a method for generating an optimized tree for rendering or streaming a 3D model. The method includes: obtaining a complete tree of the 3D model; obtaining quality values of nodes of the complete tree of the 3D model; obtaining visibility values of nodes of the complete tree of the 3D model; determining an optimized tree of the 3D model based on the complete tree, the quality values, and the visibility values; and rendering the 3D model based on the optimized tree.

[0004] The present disclosure provides a host. The host includes a storage circuit and a processor. The storage circuit is configured to store program code. The processor is coupled to the storage circuit and is configured to access the program code to perform: obtaining a complete tree of the 3D model; obtaining quality values of nodes of the complete tree of the 3D model; obtaining visibility values of nodes of the complete tree of the 3D model; determining an optimized tree of the 3D model based on the complete tree, the quality values, and the visibility values; and rendering the 3D model based on the optimized tree.

[0005] The present disclosure provides a non-transitory storage circuit. The non-transitory storage circuit is configured to store program code, and the program code is configured to cause a processor to perform: obtaining a complete tree of the 3D model; obtaining quality values of nodes of the complete tree of the 3D model; obtaining visibility values of nodes of the complete tree of the 3D model; determining an optimized tree of the 3D model based on the complete tree, the quality values, and the visibility values; and rendering the 3D model based on the optimized tree.

[0006] In summary, 3D models of XR are provided to users in a high-efficiency and fast manner.

[0007] To make the above content clearer and easier to understand, several embodiments will be described in detail below with reference to the accompanying drawings. Description of the Drawings

[0008] Figure 1A is a schematic diagram of the tree structure of a 3D model according to an embodiment of the present invention.

[0009] Figure 1B is a schematic diagram of the rendering result of a 3D model according to an embodiment of the present invention.

[0010] Figure 2A and Figure 2B is a schematic diagram of a display scene according to an embodiment of the present invention.

[0011] Figure 3A and Figure 3B is a schematic diagram of a display scene according to an embodiment of the present invention.

[0012] Figure 4A is a schematic diagram of the optimized tree of a 3D model according to an embodiment of the present invention.

[0013] Figure 4B and Figure 4C is a schematic diagram of the optimization process of the tree structure of a 3D model according to some embodiments of the present invention.

[0014] Figure 5 is a schematic flowchart of a method for generating an optimized tree for rendering or streaming a 3D model according to an embodiment of the present invention.

[0015] Figure 6 is a schematic flowchart of a method for converting a rendering tree into an optimized tree according to an embodiment of the present invention.

[0016] Figure 7 is a schematic diagram of a host according to an embodiment of the present invention. Detailed Description of the Embodiments

[0017] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numerals will be used in the drawings and the description to refer to the same or like parts.

[0018] In the present disclosure, certain terms are used throughout the specification and the appended claims to refer to particular components. Those skilled in the art will understand that electronic device manufacturers may use different names to refer to the same parts. This document is not intended to distinguish between components that perform the same function but have different names. In the following description and claims, words such as "comprising" and "including" are open-ended terms and should be interpreted as "including but not limited to...".

[0019] To bring an immersive experience to users, technologies related to extended reality (XR), such as augmented reality (AR), virtual reality (VR), mixed reality (MR), etc., are constantly being developed. AR technology enables users to bring virtual elements into the real world. VR technology allows users to enter a brand-new virtual world and experience a different life. MR technology combines the real world and the virtual world. In addition, to bring a completely immersive experience to users, visual content, audio content, or content of other senses can be provided to the user device (e.g., any smart device or computing device) of the user via the network or can be obtained by loading from the memory of the user device locally.

[0020] When XR content (e.g., data of the 3D geometry of a 3D model) is displayed to the user (e.g., by transmitting the data of the 3D geometry or the data of the 3D geometry locally over the network or by loading), it is important that the user does not have to wait for a long time to enter the virtual world. That is to say, it is important that the size of the data of the 3D geometry is preferably as small as possible to achieve faster data transmission or loading. In other words, it is important that the process of displaying the data of the 3D geometry must have sufficient efficiency so that there is no obvious delay to the user when the user device obtains the data.

[0021] For example, when the user views the data of the 3D geometry via the user device, the traditional method of loading the virtual scene of the virtual world often loads all the data at once at startup (resulting in a longer loading time), or loads the data when the user's user representative object (e.g., avatar) approaches the virtual object. Therefore, the user experience may be interrupted due to the waiting time for loading. In view of this, how to provide an efficient and fast way to provide the data of the 3D geometry to the user device is the pursuit of those skilled in the art.

[0022] In the present disclosure, a novel method for rendering or streaming a 3D model is proposed. Based on the quality value and visibility value of each node in the 3D model, an optimal tree can be determined. The 3D model can be rendered based on the optimal tree rather than the complete tree of the 3D model, and the amount of data of the 3D geometry can be reduced. Therefore, even if the network connection speed or computing power of the user device is limited, the 3D model can still be displayed to the user with little or no waiting time for loading, thereby enhancing the user experience. Further details of the method will be discussed below with reference to the accompanying drawings.

[0023] Figure 1AIt is a schematic diagram of the tree structure of a 3D model according to an embodiment of the present invention. Figure 1B It is a schematic diagram of the rendering result of a 3D model according to an embodiment of the present invention. It should be noted that, for a clearer and better understanding of the inventive concept of the present disclosure, Figure 1A and Figure 1B have been simplified. That is to say, although the tree structure of the 3D model is shown to include 2 levels and each level of the tree structure includes 1 node or 2 nodes, the tree structure may include more levels and each level of the tree structure may include more nodes. However, the present disclosure is not limited thereto.

[0024] First, refer to Figure 1A . In one embodiment, it is assumed that the 3D model has been converted by a converter into a tree structure of the geometry of multiple pieces, as Figure 1A shown, where each level in the tree structure represents the complete model of the 3D model presented by one or more nodes at different qualities. Each node may represent a piece of geometry (e.g., a mesh) of the 3D model. At the widest part of the tree structure (i.e., the bottom in Figure 1A ), level 2 of the tree structure may include four nodes (i.e., node 0_0, node 0_1, node 1_0, and node 1_1). The original quality of the 3D model may be represented by the geometry formed by these four nodes. On the other hand, at the root of the tree structure (i.e., the top in Figure 1A ), level 0 of the tree structure may contain only one node, which may be called the root node. The 3D model may be represented by the single simplified geometry formed by the root node.

[0025] Now, refer to Figure 1B simultaneously. Refer to Figure 1A and Figure 1B . In one embodiment, Figure 1A the tree structure in Figure 1B may correspond to the rendering results of each level in Figure 1B . For example, by rendering the 3D model using the root node of the tree structure at level 0, the rendering result may be as shown above Figure 1B , which may only have the simplified features of the 3D model. Further, by rendering the 3D model using two nodes (i.e., node 0, node 1) of the tree structure at level 1, the rendering result may be as shown in the middle of Figure 1BAs shown below, it can have the original quality of the 3D model. In addition, by displaying the rendering result via the user device, the 3D model in the virtual world can be displayed to the user with different qualities.

[0026] To provide a better user experience, when the request to view the 3D model is determined, the 3D model can be provided starting from the root node to ensure that the immediate result is rendered, and the details of the 3D model can be gradually refined by rendering more nodes from a higher level. In addition, different parts of the tree structure can be loaded according to the position of viewing the 3D model. For example, for the part of the 3D model that is close to the position or clearly visible in the view, the high-quality part of the tree structure can be loaded. On the other hand, for the part of the 3D model that is far from the position or less visible, the low-quality part of the tree structure can be loaded.

[0027] In this way, the quality of the 3D rendering presentation can be optimized, while also reducing the amount of network bandwidth required to stream the 3D model and reducing the amount of geometry that the rendering engine needs to process when creating the final rendering.

[0028] Figure 2A and Figure 2B are schematic diagrams of a display scene according to some embodiments of the present invention. Figure 2A Shows the display scene of two 3D models (such as model M1 and model M2) in the virtual world when two 3D models are just loaded during the initial loading process at startup. Figure 2B Shows the optimized display scene of the two 3D models loaded according to the position of the viewer point VP. In one embodiment, the viewer point VP can be a virtual camera in the virtual world or an avatar in the virtual world. In addition, based on the viewer point VP and the viewing angle, the distance from the viewer point VP to the 3D model can be determined according to the position of the viewing angle from the viewer point VP. However, the present disclosure is not limited thereto.

[0029] First refer to Figure 2A . When during the initial loading process at startup, after models M1 and M2 are just loaded, models M1 and M2 may both be rendered with the lowest details (for example, at level 0). It is worth noting that in Figure 2A , the viewer point VP is located on the right. That is, model M2 is closer to the viewer point VP, and model M1 is farther from the viewer point VP. Since the distance from the viewer point VP to model M1 is relatively far, the user may not pay much attention to the details of model M1. That is to say, although model M1 may be rendered with the lowest details, it may not have a negative impact on the user experience. In other words, model M1 may already be in an optimized state.

[0030] On the other hand, since the distance from the viewing point VP to the model M2 is relatively small, the user may pay more attention to the details of the model M2. That is to say, since the model M2 may render the lowest details, it may have some negative impacts on the user experience. In other words, the model M1 may not be in an optimized state yet. Therefore, in order to mitigate the negative impacts brought by the model M2, the details of the model M2 may be improved. That is to say, after the initial loading process of the model M2 is completed, the model M2 can be continuously rendered to obtain more details.

[0031] Now refer to Figure 2B . After the initial loading processes of the model M1 and the model M2 are completed, the model M1 can be maintained at a lower detail level (for example, at level 0), and the model M2 can continue to be rendered with a higher detail level (for example, at level 2). However, the present disclosure is not limited to this. That is to say, in order to provide a better viewing experience for the user, the model M2 closer to the viewing point VP can display more details. On the other hand, in order to reduce the waiting time for loading, the model M1 farther from the viewing point VP can be displayed with fewer details. In other words, both the model M1 and the model M2 may be in an optimized state. In this way, a good viewing experience can be provided with little or no loading waiting time, thereby improving the user experience.

[0032] Figure 3A and Figure 3B are schematic diagrams of a display scene according to an embodiment of the present invention. Figure 3A Shows the display scene of two 3D models when the position of the viewing point VP changes during runtime. Figure 3B Shows the display scene of two 3D models after the optimization process based on the updated position of the viewing point VP during runtime.

[0033] First refer to Figure 3A . After startup, during runtime, the viewing point VP may change from the right side to the left side. That is to say, the model M1 may now be closer to the viewing point VP, and the model M2 may now be farther from the viewing point VP. Since the distance from the viewing point VP to the model M2 may be relatively far, the user may not pay much attention to the details of the model M2. However, the model M2 has been rendered with higher details. That is to say, the details may be too much, and continuing to display the model M2 like this may waste computing power. In other words, the model M2 may no longer be in an optimized state.

[0034] On the other hand, since the distance from the viewing point VP to the model M1 may be relatively small, the user may pay more attention to the details of the model M1. However, the rendering details of the model M1 are relatively low. That is to say, the details may be insufficient, which may have a negative impact on the user experience. In other words, the model M1 may no longer be in an optimized state.

[0035] In short, due to the change in the position of the viewing point VP, the optimized states of the model M1 and the model M2 may also change accordingly. Therefore, in order to provide a better user experience, the optimized states of the updated model M1 and model M2 can be determined according to the position of the updated viewing point VP.

[0036] Now refer to Figure 3B . During operation, after the position of the viewing point VP changes, the model M1 and the model M2 can be rendered again according to the position of the updated viewing point VP. For example, the model M1 can now be rendered with higher details instead of lower details. On the other hand, the model M2 may now be rendered with lower details instead of higher details. That is to say, both the model M1 and the model M2 may be in an optimized state again. Therefore, the user experience may be improved.

[0037] It should be noted that, for a better understanding of the inventive concept of the present disclosure, the model M1 and the model M2 are described as two separate 3D models. However, in one aspect of the present disclosure, the model M1 and the model M2 can be regarded as different parts of a single 3D model. In one embodiment, one of the model M1 and the model M2 may be visible to the viewing point VP (e.g., facing the viewing point VP), while the other of the model M1 and the model M2 may be invisible to the viewing point VP. That is to say, even for the same 3D model, each part of the 3D model can be rendered with different levels of detail according to the position of the viewing point VP, thereby improving the user experience.

[0038] It should be pointed out that if the network connection speed of the network or the computing power of the user device is not restricted, it may be better to render all 3D models in the virtual world with the highest possible detail. However, most of the time, the network connection speed or the computing power is restricted. Therefore, corresponding restrictions can be imposed on the 3D models to ensure a good user experience.

[0039] For example, as described above, under the restriction of the network connection speed or the computing power, 3D models far from the viewing point VP can be rendered with less detail to save some time. In addition, in the real world, distant objects usually appear blurry to the user. In this way, rendering distant models with less detail under the restriction of the network connection speed or the computing power can not only save some time but also make the user experience more realistic.

[0040] In addition, to provide a better user experience, during the rendering process of the 3D model, the network connection speed or computing power can also be considered. For example, to ensure that the waiting time for loading the 3D model is lower than a predetermined value or close to zero, the maximum size of the data of the 3D geometry of the 3D model can be determined based on the network connection speed or computing power. That is to say, the 3D model can be rendered starting from the root node, and more nodes can be rendered from higher levels to gradually improve the details of the 3D model until the size of the data of the 3D geometry is close to but not greater than the maximum size. In other words, the 3D model can now provide a good viewing experience with little or no loading waiting time, and the state of the 3D model can be called an optimized state.

[0041] In one embodiment, referring to Figures 1A to 3B , the optimized state of model M1 or model M2 can be represented by the tree structure of the 3D model, and this state can be called an optimization tree. The optimization tree can be obtained by simplifying the complete tree structure (which can also be called the complete tree) of the 3D model according to the relationship between the viewing point VP and model M1 or model M2. However, the present disclosure is not limited thereto. In addition, when model M1 or model M2 is just loaded, the tree structure of model M1 or model M2 with the lowest detail can be called the initial tree.

[0042] Figure 4A is a schematic diagram of the optimization tree of the 3D model according to an embodiment of the present invention. In Figure 4A , when the 3D model is in an optimized state, the tree structure of the 3D model can be called the optimization tree OT. The optimization tree OT can be obtained by simplifying the complete tree of the 3D model.

[0043] For example, the complete tree of the 3D model can include 5 levels, and each node under levels 0 to 5 can include 4 nodes. That is to say, the complete tree can include 1 + 4 + 4×4 + 4×4×4 + 4×4×4×4 + 4×4×4×4×4 = 1365 nodes. The 1365 nodes of the complete tree can be simplified according to the current position of the viewing point VP to obtain the nodes of the optimization tree OT. As Figure 4A shown, the optimization tree OT can include 1 + 4 + 4×2 + 4×2 = 21 nodes instead of 1365 nodes. However, the present disclosure is not limited thereto.

[0044] In one embodiment, for level 1, the optimization tree OT may include node 0, node 1, node 2, and node 3. Nodes 0 and 2 may represent parts that are farther from the viewing point VP. Therefore, more details may not be rendered for nodes 0 and 2 to reduce the size of the 3D geometry data or the 3D model. On the other hand, nodes 1 and 3 may represent parts that are closer to the viewing point VP. Therefore, nodes 1 and 3 may be rendered with more details to ensure a better viewing experience.

[0045] Similarly, nodes 3_1 and 3_3 may represent parts that are closer to the viewing point VP, so more details may be rendered for nodes 3_1 and 3_3. On the other hand, nodes 1_0, 1_1, 1_2, 1_3, 3_0, and 3_2 may represent parts that are relatively less close to the viewing point VP. Therefore, no more details are rendered for nodes 1_0, 1_1, 1_2, 1_3, 3_0, and the viewing point VP.

[0046] In one embodiment, the data of the 3D geometry of the 3D model may be stored remotely in a server or locally in a memory. At the startup of loading the 3D model, the user device may only receive the root node from the server or the memory. After determining the optimization tree OT, the user device may be configured to request the optimization data about the optimization tree OT from the server or the memory. In addition, the user device may be configured to receive the optimization data from the server or the memory and render the 3D model based on the optimization tree and the optimization data. In this way, only the necessary data needs to be transmitted over the network or internally, thereby improving the user experience.

[0047] Figure 4B and Figure 4C are schematic diagrams of the optimization process of the tree structure of the 3D model according to some embodiments of the present invention. Figure 4B may correspond to the conversion process from Figures 2A to 2B At this time, models M1 and M2 have just been loaded during the loading process at startup. Figure 4C may correspond to the transformation process from Figures 3A to 3B when the position of the viewing point VP changes during operation.

[0048] First, refer to Figure 4BWhen the 3D model is first loaded, the 3D model can be rendered with relatively low detail (e.g., including nodes at two levels) to provide immediate results, and the tree structure used to represent the 3D model at that time can be referred to as the initial tree IT (also known as the rendered tree). Based on the position of the viewing point VP, the 3D model can be rendered with more detail (e.g., including nodes at four levels) so that the tree structure representing the 3D model becomes the optimized tree OT. That is to say, the tree structure representing the 3D model can be transformed from the initial tree IT to the optimized tree OT. In this way, after the transformation, since the tree structure representing the 3D model is in an optimized state, the user experience can be improved.

[0049] Now refer to Figure 4C During operation, when the position of the viewing point VP changes, the optimized tree OT may need to change accordingly. That is to say, due to the change in the relationship between the viewing point VP and the 3D model, the tree structure representing the 3D model may be transformed from the first optimized tree OT1 (also known as the rendered optimal tree) to the second optimized tree OT2 (also known as the current optimal tree). In this way, after the transformation, since the tree structure representing the 3D model is in an optimized state, the user experience can be improved.

[0050] In one embodiment, the 3D model can include multiple meshes and each mesh can include multiple triangles. In the tree structure of the 3D model, the nodes of the tree structure can represent the meshes of the 3D model. That is to say, each node of the tree structure can represent multiple triangles. Therefore, in one embodiment, the hard limit on the number of triangles to be loaded and displayed when displaying the 3D model can be referred to as the "triangle budget". In one embodiment, the network connection speed or computing power may be limited. Therefore, when the network connection speed or computing power is limited, the triangle budget can be used as a limit on the 3D model. In other words, the value of the triangle budget can be determined based on the network connection speed or computing power. However, the present disclosure is not limited thereto.

[0051] To allocate the triangle budget to each part of the 3D model in an optimized manner, it is necessary to establish an evaluation mechanism for each part of the 3D model. For example, refer to Figure 4A 、 Figure 4B and Figure 4C, each node in the optimized tree OT or the initial tree IT can have a score. A lower score can represent that the node has less detail, while a higher score can represent that the node has more detail. That is to say, the score can be used to determine whether the nodes of the tree structure are in an optimized state. In addition, when comparing the scores of the nodes in the tree structure, the scores can be used to determine a set of nodes in the tree structure that can be used to optimize the visual quality under the limitation of the triangle budget.

[0052] In one embodiment, the score of a node can be defined as the quality of the node divided by the visibility of the node to the viewing point VP. However, the present disclosure is not limited thereto. That is to say, the score of a node can be represented by the following formula.

[0053] Score = Quality / Visibility

[0054] First, before finding the scores of the nodes in the tree structure, it is important to define a measure of the quality of the nodes in the tree structure relative to (e.g., divided by) the original quality of the 3D model. For example, a quality value of a node can be defined, and the quality value can linearly correspond to the quality of the node. In one case, a quality value of 1 can represent the complete original quality of the 3D model, and a quality value of 0.5 can correspond to half of the quality. However, the present disclosure is not limited thereto. In other words, the quality value of a node can be used to measure the quality of the extracted geometry relative to its original value, and the quality values of different nodes can be used to compare the quality of different nodes with each other.

[0055] In one embodiment, one way to determine the quality value of a node can use the triangle count of the node relative to (e.g., divided by) the triangle count in the original mesh (i.e., the complete mesh) represented by the node. In another embodiment, another way to determine the quality value can use the number of texture pixels (texels, hereinafter referred to as texels) in the mesh compared to (e.g., divided by) the number of texels in the original mesh. In yet another embodiment, these two ways can be combined, or any other metric can be used to determine the quality value. That is to say, the present invention does not limit how the quality value is defined, as long as the quality of different nodes can be compared using the nodes or the quality values.

[0056] Next, another important factor in the formula is visibility. Visibility can be regarded as the degree to which a node is visible. In one embodiment, the visibility value of visibility can be obtained by dividing the surface area of the grid (used as an approximation of the size of the node) by the distance from the viewing point VP to the node. That is, for two nodes at the same distance, the node with the larger surface area will obtain a higher visibility value. Similarly, if two nodes have the same surface area, the closer node will obtain a higher visibility value. In other words, the visibility value can be inversely proportional to the distance. However, the present disclosure is not limited thereto.

[0057] Then, based on these two values, the score of each node in the tree can be calculated and can be used to determine the optimized tree OT of the 3D model. For example, when the quality of the node increases, the score increases. Or, when the node on the user device screen is smaller (visibility decreases), the score also increases. This is quite reasonable because the score can represent the quality of the 3D model on the screen. If the 3D model is farther from the viewing point VP, the same 3D model may be displayed with fewer pixels, so the quality will be higher. In some embodiments, some other parameters of additional scaling of the factor affecting the distance (e.g., from the Occlusion Culling algorithm or the ray intersection test using the Halton sequence) can be used to fine-tune the behavior of the above algorithm (which can be called the "scoring algorithm") for determining the score. However, the present disclosure is not limited thereto.

[0058] Now, each node has a score, and the score is proportional to the score of the node from the viewing point VP to the quality. Based on the scores of the nodes in the tree structure, the nodes of the optimized group can be determined for display starting from the root node. Further, when the number of rendered triangles is still within the triangle budget, the quality in the node with the lowest score can be continuously improved. That is, the node with the lowest score can be selected for improvement. Therefore, the 3D model can look the best, and the visual quality is also balanced with respect to the distance of the nodes. For example, as Figure 4A , Figure 4B , Figure 4C shown, when the total triangle count of the tree structure of the 3D model is still within the triangle budget, the nodes with lower scores can be continuously rendered to improve the viewing experience. In one embodiment, the triangles can be assigned to different parts of the geometry represented by the triangles. That is, the triangles can be evenly distributed in the tree structure to enable the viewer to easily identify the geometry. In this way, the overall total highest score of the entire virtual scene in the virtual world can be obtained efficiently and quickly, thereby improving the user experience.

[0059] In one embodiment, to further enhance the above process, an occlusion culling algorithm can be utilized. The occlusion culling algorithm is a method to speed up rendering by not rendering objects that are not visible from the viewing point VP. Unity and PlayCanvas are two different game engines that both use the occlusion culling algorithm. Therefore, the proposed method can be utilized in the two game engines to effectively implement the occlusion culling algorithm.

[0060] First, in the Unity engine, to implement the occlusion culling algorithm using the proposed method, a Halton sequence can be employed to generate rays from the viewing point VP to the farthest clipping plane. To manage the computational load, the raycasting tasks are divided into multiple frames, thus distributing the Halton sequence points over multiple frames. For example, for 15,360 points, they are divided into 60 frames, with 256 rays per frame. When a ray intersects an object, the hit is gradually revealed.

[0061] For occlusion culling, a key adjustment involves prioritizing the visibility of 3D models based on these hits. When a 3D model is hit, it is rendered first. That is, when a node of a 3D model is hit by a ray from the viewing point VP, the visibility value of the node may increase. Additionally, after a set time without new hits, the mesh renderer of the 3D model can be deactivated, thus effectively culling the occluded 3D model. This customized method dynamically adjusts the rendering priority via the immediate visibility cues provided by guiding raycasting based on the Halton sequence, thereby optimizing occlusion culling.

[0062] Second, in the PlayCanvas engine, to implement the occlusion culling algorithm using the proposed method, it is necessary to note that the occlusion culling algorithm in the web player utilizes occlusion queries, which is a feature of graphics APIs including WebGL2. The occlusion queries are performed at regular intervals to determine whether a node is visible. The individual queries are spread out so that not all of them appear in one frame.

[0063] For occlusion culling, the bounding box of the 3D model or the actual mesh of the 3D model is used in the occlusion queries. If no pixels in the geometry are rendered, the node is considered occluded and hidden. When a node is hidden, the frequency of the query can be doubled so that the node reappears faster. That is, when a node of a 3D model is hidden, the visibility value of the node can be reduced to zero. The occlusion queries are "rendered" after the opaque layer of the world so that there is an existing depth buffer for comparison. The occlusion queries do not write to the color or depth buffer, so the hidden nodes are invisible to the user and do not affect subsequent rendering layers.

[0064] Figure 5 is a schematic flowchart of a method for generating an optimized tree for rendering or streaming a 3D model according to an embodiment of the present invention. Refer to Figure 5 , the method 500 for generating an optimized tree for rendering or streaming a 3D model may include step S510, step S520, step S530, step S540, and step S550.

[0065] In step S510, the complete tree of the 3D model may be obtained from a remote server or a local memory. In step S520, for example, according to the triangle count of the nodes, the quality values of the nodes of the complete tree of the 3D model may be obtained. In step S530, for example, according to the position of the viewing point VP, the visibility values of the complete tree of the 3D model may be obtained. In step S540, the optimized tree OT of the 3D model, as Figure 4A , Figure 4B and Figure 4C shown, may be determined based on the complete tree, the quality values, and the visibility values. In step S550, the 3D model may be rendered based on the optimized tree OT. It should be noted that the implementation details of method 500 may refer to the descriptions of Figure 4A , Figure 4B and Figure 4C , and will not be elaborated here.

[0066] In this way, the quality of the 3D rendering presentation can be optimized, while at the same time reducing the amount of network bandwidth required to stream the 3D model and reducing the amount of geometry that the rendering engine needs to process when creating the final rendering.

[0067] Figure 6 is a schematic flowchart of a method for converting a rendering tree to an optimized tree according to an embodiment of the present invention. Refer to Figure 6 , the method 600 for converting a rendered tree to an optimized tree may include step S610, step S620, step S630, and step S640.

[0068] In step S610, the rendering tree of the 3D model may be obtained. For example, the rendered tree may be the initial tree IT of Figure 4B or the first optimized tree OT1 of Figure 4C , but the present disclosure is not limited thereto. In step S620, the optimized tree OT of the 3D model may be determined. For example, the optimized tree OT may be the optimized tree OT of Figure 4B or the second optimized tree OT2 of Figure 4C , but the present disclosure is not limited thereto. In step S630, the rendering tree of the 3D model may be modified according to the current position of the viewing point, so as to transform to the optimized tree OT (for example, as Figure 4B or Figure 4C(The transformation process shown). In step S640, the 3D model can be rendered based on the optimized tree OT. It should be noted that the implementation details of method 600 can be referred to Figure 4B and Figure 4C for the description, which will not be elaborated here.

[0069] In this way, even if the relationship between the viewing point VP and the 3D model changes, the 3D model can still be efficiently and quickly provided to the user.

[0070] Figure 7 is a schematic diagram of a host according to an embodiment of the present invention. In various embodiments, the host 700 can be any intelligent device and / or computer device. In some embodiments, the host 700 can be any electronic device capable of providing reality services (e.g., AR / VR / MR services, etc.). In some embodiments, the host 700 can be a computer and / or a server, and the host 700 can provide the calculated results (e.g., rendering results) to other external display devices, so that the external display devices can display the calculated results to the user. However, the present disclosure is not limited thereto.

[0071] In Figure 7 , the host 700 includes a (non-transitory) storage circuit 710 and a processor 720. The storage circuit 710 can be one or a combination of a fixed or mobile random access memory (RAM), read-only memory (ROM), flash memory, hard disk, or any other similar device, and it is configured to store multiple modules and / or program codes executable by the processor 720.

[0072] The processor 720 can be coupled to the storage circuit 710, and the processor 720 can be, for example, a general-purpose processor, a dedicated processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, a combination of one or more microprocessors with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc.

[0073] In an embodiment of the present invention, the processor 720 can be used to access the modules and / or program codes stored in the storage circuit 710 to implement method 500 or method 600, but not limited thereto. Additionally, the implementation details of the host 700 or the storage circuit 710 can be referred to FIGS. 1 to Figure 6 for the description, which will not be elaborated here.

[0074] In summary, according to the method 500, the host 700, and the storage circuit 710, an optimized tree can be determined based on the quality value and visibility value of each node in the 3D model. The 3D model can be rendered based on the optimized tree instead of the complete tree of the 3D model, and the amount of data of the 3D geometry can be reduced. Therefore, even if the network connection speed or computing power of the user device is limited, the 3D model can still be displayed to the user with little or no latency, thereby enhancing the user experience.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating an optimized tree for rendering or streaming a 3D model, characterized in that, Comprising: Obtaining a complete tree of the 3D model; Obtaining a quality value of a node of the complete tree of the 3D model; Obtaining a visibility value of the node of the complete tree of the 3D model; Determining an optimized tree of the 3D model based on the complete tree, the quality value, and the visibility value; And Rendering the 3D model based on the optimized tree.

2. The method according to claim 1, wherein Further comprising: Determining the quality value based on the triangle count or the number of texels of the node; And Determining the visibility value according to the position of the viewing point.

3. The method according to claim 1, characterized in that Further comprising: Determining the quality value based on the triangle count or the number of texels of the node relative to the triangle count or the number of texels of the original mesh represented by the node.

4. The method according to claim 1, characterized in that, Further comprising: Determining the visibility value based on the surface area of the node divided by the distance from the position of the viewing point to the node.

5. The method according to claim 1, characterized in that, Further comprising: Dividing the quality value by the visibility value to determine a score; And Determining the optimized tree based on the score.

6. The method according to claim 1, wherein Further comprising: Improving the quality of the node in response to the total triangle count of the tree structure of the 3D model being within the triangle budget.

7. The method according to claim 1, characterized in that, Further comprising: Requesting optimization data about the optimized tree from a server or a memory in response to the optimized tree being determined; Receiving the optimization data from the server or the memory; And Rendering the 3D model based on the optimized tree and the optimization data.

8. The method according to claim 1, characterized in that, Further comprising: Converting the rendering tree to the optimized tree according to the position of the viewing point.

9. The method according to claim 1, wherein Further comprising: Rendering an initial tree during an initial loading process at startup; And Converting the initial tree to the optimized tree according to the position of the viewing point.

10. The method according to claim 1, characterized in that, Further comprising: During operation, in response to a change in the position of the viewing point, converting the rendered optimized tree to the current optimized tree according to the position of the viewing point.

11. The method according to claim 1, characterized in that Further comprising: Increasing the visibility value of the node in response to the node of the 3D model being hit by a ray from the viewing point.

12. The method according to claim 1, wherein Further comprising: Reducing the visibility value of the node to zero in response to the node of the 3D model being hidden.

13. A host, characterized in that, Comprising: A storage circuit configured to store program code; And A processor coupled to the storage circuit and configured to access the program code to execute: Obtaining a complete tree of the 3D model; Obtaining a quality value of a node of the complete tree of the 3D model; Obtaining a visibility value of the node of the complete tree of the 3D model; Determining an optimized tree of the 3D model based on the complete tree, the quality value, and the visibility value; And Rendering the 3D model based on the optimized tree.

14. The host according to claim 13, characterized in that, Wherein the processor is further configured to access the program code to execute: Determining the quality value based on the triangle count or the number of texels of the node; and Determining the visibility value according to the position of the viewing point.

15. The host according to claim 13, characterized in that, Wherein the processor is further configured to access the program code to execute: Determining the quality value based on the triangle count or the number of texels of the node relative to the triangle count or the number of texels of the original mesh represented by the node.

16. The host according to claim 13, wherein Wherein the processor is further configured to access the program code to execute: Determine the visibility value based on the surface area of the node divided by the distance from the position of the viewing point to the node.

17. The host according to claim 13, characterized in that, Wherein the processor is further configured to access the program code to perform: Divide the quality value by the visibility value to determine a score; and Determine the optimized tree according to the score.

18. The host according to claim 13, wherein Wherein the processor is further configured to access the program code to perform: Improve the quality of the node in response to the total triangle count of the tree structure of the 3D model being within the triangle budget.

19. The host according to claim 13, characterized in that, Wherein the processor is further configured to access the program code to perform: In response to the optimized tree being determined, request optimization data about the optimized tree from a server or a memory; Receive the optimization data from the server or the memory; And Render the 3D model based on the optimized tree and the optimization data.

20. A non-transitory storage circuit configured to store program code, characterized in that, The program code is configured to cause the processor to perform: Obtain a complete tree of the 3D model; Obtain the quality value of the nodes of the complete tree of the 3D model; Obtain the visibility value of the nodes of the complete tree of the 3D model; Determine the optimized tree of the 3D model based on the complete tree, the quality value, and the visibility value; And Render the 3D model based on the optimized tree.

Images