Three-dimensional model processing system, method, and storage circuit

Abstract

To provide a 3D model processing system, a method, and a program for efficiently providing a 3D model with low latency to a user.SOLUTION: A 3D model processing system includes a storage circuit and a processor. The storage circuit stores a program code and is combined with the processor. The program code causes a processor to perform steps of: acquiring a plurality of nodes of a 3D model; separating the 3D model into a first original partition containing a first node group and a second original partition containing a second node group, according to position or material; generating a plurality of first-stage partitions of different detail levels for the first original partition according to the first node group; and generating a plurality of second-stage partitions of different detail levels for the second original partition according to the second node group.SELECTED DRAWING: Figure 2

Description

[Background technology]

[0001] The present invention relates to a 3D (3-dimensional) model processing system, and more particularly, the present invention relates to a 3D model processing system, a 3D model processing method, and a memory circuit.

[0002] To provide users with an immersive experience, technologies related to cross reality (XR), such as Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR), are constantly being developed. AR technology allows users to bring virtual elements into the real world. VR technology allows users to enter a whole new virtual world to experience a different life. MR technology merges the real world with the virtual world. Furthermore, visual, audio, or other sensory content can be provided to users via the web to provide a fully immersive experience to users. Summary of the Invention [Problem to be solved by the invention]

[0003] The present invention relates to a 3D model processing system, a 3D model processing method, and a storage circuit that provide an efficient and low-latency method for providing XR content to a user. [Means for solving the problem]

[0004] In the present invention, a 3D model processing system is provided. The 3D model processing system includes a memory circuit and a processor. The memory circuit is configured to store program code. The processor is coupled to the memory circuit and configured to access the program code to obtain a plurality of nodes of the 3D model, divide the 3D model into a first original partition including a first group of nodes and a second original partition including a second group of nodes based on a position or a material, generate a plurality of first graded partitions of different Levels of Detail (LoD) for the first original partition based on the first group of nodes, and generate a plurality of first graded partitions of different Levels of Detail (LoD) for the second original partition based on the second group of nodes.

[0005] In the present invention, a 3D model processing method is provided, which includes: obtaining a plurality of nodes of a 3D model, dividing the 3D model into a first original partition including a first group of nodes and a second original partition including a second group of nodes based on a position or a material, generating a plurality of first stage partitions with different levels of detail for the first original partition based on the first group of nodes, and generating a plurality of first stage partitions with different levels of detail for the second original partition based on the second group of nodes.

[0006] In accordance with the present invention, a storage circuit is provided, the storage circuit being configured to store program code configured to cause a processor to obtain a plurality of nodes of a 3D model, partition the 3D model into a first original partition including a first group of nodes and a second original partition including a second group of nodes based on position or material, generate a plurality of first stage partitions with different levels of detail for the first original partition based on the first group of nodes, and generate a plurality of first stage partitions with different levels of detail for the second original partition based on the second group of nodes. Effect of the Invention

[0007] Based on the above, the 3D model processing system, the 3D model processing method, and the storage circuit can provide the 3D model to the user efficiently and with low delay.

[0008] In order to make the above content easier to understand, several embodiments are described in detail below with reference to the drawings. [Brief description of the drawings]

[0009] The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0010] [Figure 1] FIG. 1 is a schematic diagram of a 3D model processing system according to one embodiment of the present invention. [Diagram 2] FIG. 2 is a flow diagram of a 3D model processing method according to one embodiment of the present invention. [Figure 3A] FIG. 1 is a schematic diagram of a segmentation scenario for a 3D model according to one embodiment of the present invention. [Figure 3B] FIG. 1 is a schematic diagram of a segmentation scenario for a 3D model according to one embodiment of the present invention. [Figure 3C]FIG. 1 is a schematic diagram of a segmentation scenario for a 3D model according to one embodiment of the present invention. [Figure 3D] FIG. 1 is a schematic diagram of a segmentation scenario for a 3D model according to one embodiment of the present invention. [Figure 4] FIG. 1 is a schematic diagram of a baking scenario for a 3D model according to one embodiment of the present invention. [Diagram 5] FIG. 1 is a schematic diagram of a flattening scenario for a 3D model according to one embodiment of the present invention. [Figure 6] FIG. 2 is a schematic diagram of a file format of a 3D model processing system according to one embodiment of the present invention. [Figure 7] FIG. 2 is a flow diagram of a 3D model processing method according to one embodiment of the present invention. [Figure 8] FIG. 2 is a schematic diagram of a viewing scenario for a 3D model according to one embodiment of the present invention. [Figure 9] FIG. 2 is a flow diagram of a 3D model processing method according to one embodiment of the present invention. [Figure 10] FIG. 1 is a schematic diagram of a compression scenario for a 3D model according to one embodiment of the present invention. [Figure 11] FIG. 1 is a schematic diagram of a compression scenario for a 3D model according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Reference will now be made in detail to the exemplary embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers are used in the drawings and the description to refer to the same or like parts.

[0012] Throughout the specification and the appended claims, certain terms are used to refer to particular components. Those skilled in the art will understand that electronics manufacturers may refer to the same components by different names. No distinction is intended herein between components with the same function but different names. In the following description and claims, terms such as "comprising" and "including" are open-ended terms and should be interpreted as "including, but not limited to."

[0013] The term "coupled" as used throughout the specification (including the appended claims) of this application may refer to any direct or indirect connection means. For example, when a first device is described as being coupled to a second device, it should be interpreted that the first device is directly connected to the second device, or that the first device is indirectly connected via another device, or that a particular connection means is connected to the second device. The terms "first", "second" and similar terms mentioned throughout the specification (including the appended claims) of this application are used merely to designate individual elements or to distinguish different embodiments or scopes. Thus, the terms should not be considered as limiting the upper or lower limits of the quantity of elements, nor should they be used to limit the order of arrangement of elements. In addition, wherever possible, elements / components / steps using the same reference numbers in the drawings and embodiments represent the same or similar parts. The same reference numbers or the same terms may be used to mutually refer to related descriptions of elements / components / steps in different embodiments.

[0014] It should be noted that in the following embodiments, the technical features of several different embodiments can be replaced, recombined, and mixed to complete other embodiments without departing from the spirit of the present invention. The features of each embodiment can be used in any combination as long as they do not go against the spirit of the present invention or contradict each other.

[0015] Technologies related to cross reality (XR), such as Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR), are constantly being developed to provide users with immersive experiences. AR technology allows users to bring virtual elements into the real world. VR technology allows users to enter a whole new virtual world to experience a different life. MR technology merges the real world with the virtual world. Furthermore, visual, audio, or other sensory content can be provided to users via the web to provide a fully immersive experience to users.

[0016] When XR content (e.g., 3D geometry data) is streamed to users over the internet, it is important that the user does not have to wait a long time before being placed in virtual reality. Therefore, the size and composition of the XR content is important during the creation stage of the XR content.

[0017] Furthermore, when transmitting XR content (e.g., 3D geometry data) over the web, it is important that the mesh of the XR content's set of triangles is stored in as few bytes as possible to make data transmission faster. At the same time, when the system fetches the data, decoding the data must be performant enough that it does not cause noticeable delays to the user.

[0018] Furthermore, while a user is viewing XR content through a user device (e.g., a computer, a smartphone, etc.), the conventional method of loading a virtual world over a network by the user device often loads all data at startup (long loading time) or when the user's user representative object (e.g., an avatar) goes near it (an object that appears and disappears in the scene). This may interrupt the user experience due to loading latency or loading delay. In view of the above, those skilled in the art have sought to provide an efficient and low-latency method for providing XR content to a user.

[0019] In order to provide XR content such as a 3D model to a user in an efficient and low-latency manner, three aspects of a 3D model are described below with some embodiments: first, how the 3D model is prepared, created, and partitioned; second, how the 3D model is compressed; and third, how the 3D model is loaded from a user end (e.g., a user device of a user). These three aspects may be used individually or together to provide XR content to a user, and the present invention is not limited thereto.

[0020] 1 is a schematic diagram of a 3D model processing system according to one embodiment of the present invention. Referring to FIG. 1, a 3D model processing system 100 is illustrated. In different embodiments, the 3D model processing system 100 may be implemented as, but is not limited to, various smart devices and / or computing devices.

[0021] 1, the 3D model processing system 100 includes a (non-transitory) memory circuit 102 and a processor 104. The memory circuit 102 may be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, a hard disk, other similar devices, or a combination of these devices, and may be used to store multiple program codes or modules.

[0022] The processor 104 is coupled to the memory circuitry 102 and may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor, multiple microprocessors, one or more microprocessors combined with a digital signal processor, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), other types of integrated circuits, a state machine, an Advanced RISC Machine (ARM) based processor, or the like.

[0023] In one embodiment of the present invention, the processor 104 may access modules and program codes stored in the memory circuitry 102 to implement the 3D model processing proposed by the present invention, which will be described in detail below.

[0024] Fig. 2 is a flow diagram of a 3D model processing method according to one embodiment of the present invention. With reference to Fig. 1 and Fig. 2, a 3D model processing method 200 of this embodiment may be performed by the 3D model processing system 100 in Fig. 1, and the details of each step in Fig. 2 will be described below together with the components shown in Fig. 1.

[0025] First, in step S210, a plurality of nodes of a 3D model may be obtained. In one embodiment, the node is a basic component of the 3D model. That is, the plurality of nodes may form a geometry (e.g., a triangle), and the plurality of geometries may form the 3D model. Then, in step S220, the 3D model may be divided into a first original partition and a second original partition based on a position or a material. Furthermore, the first original partition may include a first node group, and the second original partition may include a second node group. That is, the 3D model may include a plurality of positions and a plurality of materials, and the 3D model may be partitioned into a plurality of blocks (e.g., a first original partition, a second original partition, etc.) based on a position or a material of each region of the 3D model. Then, in step S230, the plurality of nodes may be partitioned into a first node group and a second node group based on the first original partition and the second original partition. It is to be mentioned that step S220 and step S230 may be performed simultaneously, step S220 may define the shape of the piece of the 3D model, and step S230 may define which nodes belong to the piece of the 3D model, but the present invention is not limited thereto. Furthermore, in step S240, a plurality of first graded partitions with different levels of detail for the first original partition may be generated based on the first node group. Furthermore, in step S250, a plurality of second graded partitions with different levels of detail for the second original partition may be generated based on the second node group.

[0026] Although the 3D model is described as including a first original partition and a second original partition for convenience of explanation, the 3D model may include multiple original partitions. Furthermore, multiple stage partitions correspond to multiple original partitions. However, the present invention is not limited thereto.

[0027] Based on the above, by dividing the 3D model based on material or position, the size of each partition of the 3D model can be reduced, so that based on a request from a user device at a user end, instead of loading the entire original model at once, only a part of the original model can be loaded on demand, thereby reducing waiting time and improving user experience.

[0028] In addition, details of the implementation of the 3D model processing system 100 and the 3D model processing method 200 may be referred to the following description of Figures 3A-6 for sufficient teachings, suggestions, and embodiments. In some embodiments, the functions of the 3D model processing system 100 and the 3D model processing method 200 may be performed by the processor 104, although the invention is not limited thereto.

[0029] FIG 3A is a schematic diagram of a partitioning scenario of a 3D model according to one embodiment of the present invention. FIG 3B is another schematic diagram of a partitioning scenario of a 3D model according to one embodiment of the present invention. Referring to FIG 3A and FIG 3B, partitioning scenario 300A includes an original model 310 and multiple original partitions 320, and partitioning scenario 300B includes an original model 310 and multiple original partitions 330.

[0030] In the preparation stage of the 3D model, how to cut the 3D model into pieces affects not only the efficiency of streaming the 3D model over the Internet, but also the speed of loading the 3D model at the user end. That is, in order to improve the speed and performance of loading and rendering the 3D model, how to cut the 3D model into pieces is important. In one embodiment, the smaller pieces of the 3D model may be called partitions.

[0031] Referring to FIG. 3A, in one embodiment, the 3D model may be partitioned into smaller pieces (i.e., partitions) based on the material or location of each piece or region of the 3D model. For example, the original model 310 of the robot may include different parts, such as a head, a body, arms, wheels, etc. These parts may be designed to be made of different materials, such as metal, plastic, rubber, etc. The original model 310 of the robot may be classified into different materials and may be partitioned into pieces based on the materials (e.g., the material index of each piece), as shown in FIG. 3A. In one embodiment, the material index of each piece may be predefined according to design requirements. For example, a piece of material with a material index of 1 may be metal. The material index of every piece may be associated with a UV map corresponding to the 3D model. However, the present invention is not limited thereto. That is, partitions of the same material may be grouped together. In other words, a first original partition may belong to a first material, and a second original partition may belong to a second material. The mode of processing that divides the original model 310 into pieces based on the material of each piece may be referred to as the “material mode,” and these partitioned pieces may be referred to as multiple original partitions 320 .

[0032] With reference to FIG. 3B, in one embodiment, the 3D model may be partitioned into smaller pieces based on the location of each piece of the 3D model. For example, the original model 310 of the robot may include a top portion, a bottom portion, a left portion, a right portion, a front portion, a rear portion, etc. The original model 310 of the robot may be divided into different locations or quadrants in space and partitioned into pieces based on location, as shown in FIG. 3B. That is, partitions that are close to each other may be grouped together. In other words, a first original partition may belong to a first location, and a second original partition may belong to a second location. The mode of processing that partitions the original model 310 into pieces based on location may be referred to as a "spatial mode," and the partitioned pieces may be referred to as a plurality of original partitions 330.

[0033] 3C is a schematic diagram of a splitting scenario of a 3D model according to one embodiment of the present invention. Referring to FIG. 3C, splitting scenario 300C includes splitting scenario 301C and splitting scenario 302C. Splitting scenario 301C illustrates an original model 310 split in material mode, and splitting scenario 302C illustrates an original model 310 split in spatial mode.

[0034] In one embodiment, the original model 310 may be divided into pieces using either a material mode to create multiple original partitions 320 or a spatial mode to create multiple original partitions 330, but the invention is not limited thereto. That is, the mode of division may be determined in advance. Then, based on the mode of division, the original model 310 may be divided into multiple partitions. In one embodiment, but is not limited thereto, bounding boxes may represent the partitions of the original model 310. For example, as shown in FIG. 3C, each of the division scenarios 301C and 302C may include multiple bounding boxes, and the multiple bounding boxes may represent the multiple original partitions 320 or the multiple original partitions 330, respectively.

[0035] Note that larger partitions may be split into smaller partitions, i.e., the original model 310 may be split into smaller partitions and parts of the original model 310 may vary in visual quality independently of each other. The splitting process may be done with a recursive algorithm, i.e., the splitting process may be performed repeatedly until a certain condition is met. Thus, the original model 310 may be stored as individual triangles, allowing the original model 310 to be analyzed triangle by triangle without needing to have the entire original model 310 in memory at once. In one embodiment, the triangles may be called or stored as a soup file, which may be a file of a collection of individual triangles.

[0036] For example, after the splitting mode is determined and the splitting is performed, the bounding box of each partition may be split into multiple parts, and then a new soup file of triangles contained in the new bounding box may be created. This splitting process may be performed recursively. For example, in a first iteration of the splitting process, all triangles of the original partitions 320 or the original partitions 330 may be split into two or more groups (also known as children). Then, the splitting process may be performed again to create two or more children of each group. This splitting process may be repeated until a certain condition is met, for example, each of all children is below the triangle threshold. The triangle threshold may be adjusted according to design requirements. In one embodiment, the triangle threshold may be determined based on the maximum computing power or connection speed of the user device. That is, after the last iteration of the splitting process, the number of triangles in each group of triangles is less than the triangle threshold. After the splitting process, the partitions created together may form a tree or hierarchy of the original model 310. That is, instead of loading the 3D model all at once, the 3D model may be loaded partition by partition. In this manner, by dividing the 3D model based on material or position, the size of each partition of the 3D model can be reduced. Thus, based on a request from a user device, instead of loading the entire original model 310 all at once, only a portion of the original model 310 can be loaded as needed, thereby reducing waiting time and improving the user experience.

[0037] 3D is a schematic diagram of a partitioning scenario for a 3D model according to one embodiment of the present invention. Referring to FIG. 3D, partitioning scenario 300D includes staged partitions 340.

[0038] In one embodiment, after splitting the original model 310, the original model 310 may be divided into pieces, such as original partition 320 or original partition 330. That is, while the original model 310 is loaded by the user device, the original model 310 is loaded partition by partition, and the size of each partition may be determined based on the computer capability of the user end. However, in certain situations, it is necessary to load the 3D model all at once, rather than loading the 3D model partition by partition. That is, the total number of triangles in the original model 310 needs to be less than a triangle threshold for the user device to process all the triangles at once. For example, the triangle threshold may be determined based on the maximum computing capability of the user device. In one embodiment, a decimation algorithm (selected from existing techniques according to design requirements) may be used to remove triangles until the total number of triangles is less than the triangle threshold.

[0039] At the same time, when a user is viewing a portion of the 3D model instead of the entire 3D model, the portion of the 3D model should provide the user with an optimal visual experience. That is, when a user is viewing a portion (e.g., a partition) of the original model 310 instead of the entire original model 310, the number of triangles in the portion should also meet the triangle threshold. In other words, some triangles containing important features (e.g., visually salient features) may be returned to the portion to increase the number of triangles in the portion to provide an optimal visual experience.

[0040] On the other hand, when the user is looking at a portion of the portion of the 3D model, the portion of the portion should also provide the user with an optimal visual experience. That is, when the user is looking at a portion of the portion of the original model 310 (e.g., a portion of a partition), the number of triangles of the portion should also meet the triangle threshold. Similarly, some triangles containing important features may be returned to the portion to increase the number of triangles of the portion to provide an optimal visual experience. In one embodiment, the partition or the portion of the partition may be defined by a segmentation process of the original model 310 in a material mode or a spatial mode. However, the present invention is not limited thereto.

[0041] In summary, original model 310 may include multiple partitions (e.g., original partition 320 or original partition 330), and each partition may be processed to include different amounts of triangles for different display situations to provide an optimal visual experience. That is, each partition may be processed to generate multiple tiered partitions to be displayed separately. Each tiered partition may belong to one level of division. That is, each partition of original model 310 may belong to a different level of division of the tiered partitions.

[0042] In one embodiment, each partition may include three levels of division, such as level 0, level 1, and level 2. The level of division may define how many parts or groups the 3D model or partition of the 3D model is divided into, with higher levels of division representing more groups. That is, the higher the level of division, the more groups of triangles are included in the partition of the original model 310. For example, as shown in FIG. 3D , in the level 0 division, the triangles of the partition of the original model 310 (e.g., the monkey's head) or the triangles of the original model 310 belong to the same group. More specifically, one group of triangles may include 50,000 triangles, and the total number of triangles of the partition of the level 0 division or the original model 310 may be 50,000. Furthermore, in the level 1 division, the triangles of the partition or the original model 310 may belong to two groups to be displayed separately. More specifically, each group of triangles may include 50,000 triangles, and the total number of triangles of the partition of the level 1 division or the original model 310 is 100,000. Furthermore, in a level 2 division, the triangles of the partition or the triangles of the original model 310 may belong to four groups to be displayed separately. More specifically, each group of triangles may include 50,000 triangles, and the total number of triangles of the partition or the original model 310 in the level 2 division may be 200,000. However, the present invention is not limited thereto.

[0043] Referring back to FIG. 3D, in one embodiment, for each level of division, a different level of detail (LoD) may be created. That is, each original partition 320 or each original partition 330 may include a different LoD. The purpose is to ensure that the triangles of each stage partition are smaller than a different triangle threshold. For example, in different types of user devices at the user end, the computing power of each user device may be different. Therefore, different triangle thresholds are applied to different user devices. A decimation algorithm (selected from existing techniques according to design requirements) may be used to remove triangles until the number of triangles is below a certain limit to generate different levels of detail for each stage partition. This limit may be determined based on the computing power of the user device. In one embodiment, each level of division may include three levels of detail (LoD), such as LoD0, LoD1, and LoD2. That is, the stage partition may include at least a partition with a level of detail 0, a partition with a level of detail 1, and a partition with a level of detail 2. The LoD defines how many triangles each triangle group should contain, with a lower LoD representing more triangles, i.e. the lower the level of detail, the more triangles may be included in each triangle group, in other words the higher the level of detail the higher the compression ratio.

[0044] For example, the triangle number limit in LoD0 may be 50,000, i.e., one triangle group may contain 50,000 triangles. The triangle number limit in LoD1 may be 12,500, i.e., one triangle group may contain 12,500 triangles. The triangle number limit in LoD2 may be 4,000, i.e., one triangle group may contain 4,000 triangles. The triangle number limit may be determined according to design requirements or based on the computing capabilities of a user device. However, the present invention is not limited thereto. It should be mentioned that the groups (i.e., partitions) of each level of division may together form the original model 310 or a partition of the original model 310. That is, the original model 310 or a partition of the original model 310 with level 0 decomposition may include 50,000 triangles at LoD0, the original model 310 or a partition of the original model 310 with level 1 decomposition may include 100,000 triangles at LoD0, and the original model 310 or a partition of the original model 310 with level 2 decomposition may include 200,000 triangles at LoD0.

[0045] It should be noted that one purpose of the different detail levels is to provide the user with a choice while viewing the model, so that the ideal visual result is provided to the user without providing some unimportant details completely. That is, each partition of the original model 310 may be rendered with higher or lower quality when necessary. For example, only the parts of the original model 310 that are closer to the viewer may be rendered with higher details, while other parts of the original model 310 that are farther from the viewer may be rendered with lower details. These different detail levels of each partition help improve the rendering performance while the original model 310 can provide the ideal visual result to the user, since not all details of the 3D model need to be rendered all at once. In this way, the 3D model processing system 100 or the 3D model processing method 200 can provide a powerful method for improving the speed and performance of loading and rendering of 3D models.

[0046] Figure 4 is a schematic diagram of a baking scenario for a 3D model according to one embodiment of the present invention. Referring to Figure 4, baking scenario 400 illustrates how textures are applied to a mesh (e.g., triangles) of a 3D model using baking.

[0047] In one embodiment, if a partition of a 3D model contains a large texture, not only may the mesh (e.g., triangles) contain several levels of detail, but multiple levels of detail may be generated for the texture. For example, if a partition has a texture of 2048x2048 pixels, the partition may be partitioned into more pieces, e.g., four pieces. That is, four partitions, each having 1024x1024 pixels, may be created. In this way, instead of loading the entire texture of the partition at once, only a portion of the texture may be loaded from the user device on demand, thereby reducing latency and improving the user experience.

[0048] In one embodiment, baking is the process of transferring detailed information from a high-resolution 3D model to a low-resolution 3D model. This is typically done to improve performance or reduce file size. Baking involves creating texture maps that capture the details of the high-resolution model, such as lighting, shadows, and surface imperfections. These texture maps are then applied to the low-resolution model, giving it the appearance of the high-resolution model without the computational cost of rendering every polygon.

[0049] It is worth mentioning that if the texture coordinates of a texture map for a 3D model take up too much space in the texture image, the texture coordinates may be recalculated and the texture may be baked again as shown in FIG. 4. For example, the texture of an original model 410 may be obtained and stored as multiple texture maps 420 in a UV coordinate system. For example, as shown in FIG. 4, the original model 410 corresponds to four texture maps 420. The texture maps 420 may be compressed to generate multiple compressed texture maps 422. For example, as shown in FIG. 4, the four texture maps 420 may be compressed to a quarter of their original size to form the four compressed texture maps 422. Then, a baking process may be performed on each compressed texture map to generate multiple baked texture maps 424. Then, each of the multiple baked texture maps 424 may be applied to a corresponding mesh of the original model 410 to form a baked partition 430. That is, the 3D model may be unwrapped onto a 2D surface (e.g., the texture maps 420). Further, a baked texture may be created (baked) by converting a high-resolution model of the 3D model's texture into a low-resolution model. In other words, the baked texture may contain key features (e.g., visually significant features) of the high-resolution model of the texture. Further, the baked texture may be applied to the mesh of the 3D model. In this way, baking can be more efficient for storing and rendering the original model 410.

[0050] In one embodiment, if the mesh can be stored more efficiently by baking the texture, baking may be performed to recalculate the texture coordinates and bake the texture again. If the texture coordinates take up too much space, the total area of ​​the texture coordinates may be divided by the total area of ​​the texture. If the result of the division is lower than a certain threshold, the texture coordinates may be recalculated and the texture may be baked again. The threshold may be a value set by the user. The lower the threshold, the more likely the texture coordinates will be recalculated.

[0051] 5 is a schematic diagram of a flattening scenario for a 3D model according to one embodiment of the present invention. Referring to FIG. 5, flattening scenario 500 may include a non-flattened model hierarchy 510 and a flattened model hierarchy 520.

[0052] In one embodiment, a 3D model may be made up of many different parts, called layers. These layers may be nested, with one part containing another. Note that when a 3D model is flattened, all layers are removed and the 3D model is converted into a single, flat array of triangles. For example, by performing a flattening process, the non-flattened model layer 510 may be made into a flattened model layer 520 by removing all layers in the structure of the 3D model to ensure that only a flat array of triangles remains. This does not change the appearance of the 3D model, but makes it easier to process. That is, there is no visual difference between the non-flattened model corresponding to the non-flattened model layer 510 and the flattened model corresponding to the flattened model layer 520. These files that store the flat array of triangles may be called soup files, or stored as soup files, which are files of a collection of independent triangles.

[0053] In one embodiment, each triangle in the flat array may store three pieces of information: the positions of the vertices that make up the triangle, the texture coordinates of the triangle, and the material index of the triangle. The texture coordinates may tell the renderer of the 3D model where to find the texture image that should be used to render the triangle. The material index may tell the renderer which material to use to render the triangle. It should be noted that storing all data for a 3D model in a flat array of triangles can be easier to process, especially for large 3D models, because the renderer can quickly access the data it needs without having to traverse the hierarchy. In this way, it is possible to process 3D models that are larger than the amount of available memory on the computer.

[0054] 6 is a schematic diagram of a file format for a 3D model processing system in accordance with one embodiment of the present invention. With reference to FIG. 6, file format 600 illustrates an exemplary content of a 3D model, although the present invention is not limited thereto.

[0055] In one embodiment, when all of the meshes and textures for all levels of detail have been generated, the meshes and textures may be stored in a file format as shown in Figure 6. File format 600 may be an Extended Reality Geometry (XRG) format or an Extended Reality Geometry Compressed (XRGC) file format. An XRG or XRGC file may be a data container used to store other files, along with common files such as JPG (Joint Photographic Experts Group) and PNG (Portable Network Graphics), although the invention is not limited thereto.

[0056] In one embodiment, the file format 600 may include a header file, a JavaScript Object Notation (JSON) file, a mesh (e.g., triangles) of the 3D model, textures of the 3D model, etc. The header file may be configured to hold general information of the files in the container file. For example, the header file may describe some general information and where the JSON file is located. Furthermore, the JSON file may be configured to describe some general information such as the complete layout and hierarchy of the 3D model and where the JSON file is located. This JSON file may be used by the client application to read the model description and execute an optimized query for the required geometry. The query may be optimized based on different factors such as the location of the user representative object of the user in the 3D space (e.g., the virtual world in XR) and the distance between the user representative object and the 3D model. It should be mentioned that the user representative object may be, for example, the user's avatar, a virtual object representing the user, a virtual object representing a head mounted display (HMD) device worn by the user, or other similar objects. However, the present invention is not limited thereto. In this way, based on the file format 600, the data of the 3D model can be stored or compressed in an efficient manner, which is important for XR applications, allowing the data of the 3D model to be streamed in real time as needed, thereby improving the user experience.

[0057] Fig. 7 is a flow diagram of a 3D model processing method according to one embodiment of the present invention. With reference to Fig. 1 and Fig. 7, the 3D model processing method 700 of this embodiment may be performed by the 3D model processing system 100 in Fig. 1, and the details of each step in Fig. 7 will be described below together with the components shown in Fig. 1.

[0058] First, in step S710, a request to view a 3D model from a user device may be determined. For example, a user may want to display a 3D model in a virtual world while the user's user representative object is located at a certain distance from the 3D object in the virtual world. Furthermore, the 3D model may be partitioned into two parts, a first part being close to the user representative object and a second part being significantly far from the avatar. Furthermore, the first part may belong to a first original partition, and the second part may belong to a second original partition. Also, as shown in FIG. 3D, a plurality of first stage partitions of different LoDs may be generated for the first original partition, and a plurality of second stage partitions of different LoDs may be generated for the second original partition. Next, in step S720, a first level of detail for the plurality of first stage partitions may be determined based on a state of the user device of the user. For example, since the first part is close to the user representative object (compared to the second part), the first LoD may be determined to be LoD0 to provide more details to the user device. Then, in step S730, a second level of detail for the plurality of second tier partitions may be determined based on the state of the user device. For example, since the second portion is farther from the user representative object (compared to the first portion), the second LoD may be determined as LoD2 to provide less detail to the user device. Furthermore, in step S740, one of the plurality of first tier partitions may be provided to the user device based on the first level of detail. For example, when the computer capacity of the user device is high, a tier partition of a level 2 division of the plurality of first tier partitions may be provided to the user device. Meanwhile, when the computer capacity is low, a tier partition of a level 0 division of the plurality of first tier partitions may be provided to the user device. Also, in step S750, one of the plurality of second tier partitions may be provided based on the second level of detail.For example, when the computer capacity of the user device is high, the staging partitions of the level 2 division of the plurality of second staging partitions may be provided to the user device, whereas when the computer capacity is low, the staging partitions of the level 0 division of the plurality of second staging partitions may be provided to the user device, although the present invention is not limited thereto.

[0059] Based on the above, an appropriate level of detail can be determined based on the state of the user device and the corresponding content of the 3D model can be provided to the user device with low latency.

[0060] In addition, for details of the implementation of the 3D model processing system 100 and the 3D model processing method 700, reference may be made to the following description of Fig. 8 to obtain full teachings, suggestions, and embodiments. In some embodiments, the functions of the 3D model processing system 100 and the 3D model processing method 700 may be performed by the processor 104, although the invention is not limited thereto.

[0061] 8 is a schematic diagram of a viewing scenario of a 3D model according to one embodiment of the present invention. Referring to FIG. 8, a viewing scenario 800 illustrates how a 3D model is loaded depending on the state of a user device.

[0062] In one embodiment, during the model creation stage, the 3D model may be partitioned and the partitions may be stored with different levels of detail. Then, while the user is viewing the 3D model, the 3D model may be rendered based on the state of the user device, such as the distance between the user representative object and the 3D model, the user's eyesight, the user's line of sight, the (internet) connection speed of the user device, the computing power of the user device, etc. For example, parts of the 3D model that are closer to the user representative object are rendered with higher details, and parts that are farther from the user representative object are rendered with lower details. These different levels of detail of each partition help to improve the rendering performance while the 3D model still provides an ideal visual result for the user, since not all details of the 3D model need to be rendered at once. That is, similar to how the user feels in the real world, what is closer to the user is clearer and what is not closer to the user is blurrier. This allows the computing power to be saved for rendering further parts of the 3D model, which improves the performance of rendering the 3D model.

[0063] In summary, the client-side logic algorithm may be considered based on the location of the user representative object in the virtual world. More specifically, the client-side logic algorithm may be configured to determine which partitions of which level of detail to provide to the user device to provide the best visual result. The client-side logic algorithm may be continuously updated in real-time as the user representative object walks around the virtual world and the user's view (i.e., field of view or line of sight) changes accordingly. Furthermore, the user's field of view may be represented as a field of view (FOV) of a camera or viewer in the virtual world. Furthermore, in some embodiments, a portion of the 3D model in or close to the FOV may be defined as a visible partition of the 3D model, and other portions of the 3D model may be defined as invisible partitions of the 3D model. Only when a partition of the 3D model becomes a visible partition, the partition of the 3D model may be provided to the user device. However, the invention is not limited thereto.

[0064] Specifically, when a user device wants to start presenting (i.e., showing) a 3D model, the user device may first download metadata about the 3D model. This metadata may include the layout of the virtual world and the hierarchy of the 3D model. The structure of the 3D model may then be replicated in memory as a tree of nodes. Each node in the tree may include one or more levels of detail. Based on the tree of nodes and the state of the user device, a client-side logic algorithm is used to find the optimal tree of nodes to make visible. The algorithm may take into account the position and view of the user representative object to determine which nodes should be made visible to provide the best visual result.

[0065] It should be noted that in the previous method, there may be 3D models at multiple levels of detail. However, meshes are always loaded from the entire 3D model each time a different level of detail is rendered, but fewer triangles are used for rendering. In this embodiment, the 3D model is divided into a hierarchy, and elements in the hierarchy may be switched between levels of detail independently of each other. That is, meshes may be loaded from a portion of the entire 3D model each time a different level of detail is rendered, rather than the entire 3D model. For example, as described in the 3D model processing method 200 of FIG. 2, the 3D model may include a first original partition and a second original partition, and a plurality of first stage partitions and a plurality of second stage partitions may be generated based on the first original partition and the second original partition. Furthermore, as described in the 3D model processing method 700 of FIG. 7, the state of the user device may be used to determine the level of detail of the first stage partition and the level of detail of the plurality of second stage partitions. That is, instead of loading the entire 3D model at once, only a portion of the 3D model may be loaded by the user device, thereby reducing waiting time and improving the user experience.

[0066] In addition, there are several strategies that can be used by the algorithm to find the optimal tree of nodes to display the 3D model. One strategy is to determine the level of detail of each partition or node to be visualized based on a threshold of the distance of the partition from the user representative object. Another strategy is to start from the root node with the lowest level of detail to visualize the node and gradually improve the tree of nodes.

[0067] In one embodiment, the threshold of the distance policy may calculate the size of the partition in the view of the user device and attempt to provide the partition with the highest detail to the user device. If the size of the partition is larger than a certain threshold, the 3D model processing system 100 may simplify the level of detail to reduce the size of the partition. If the size of the partition is smaller than a certain threshold, the system may refine the level of detail to increase the size of the partition. That is, the 3D model processing system 100 may attempt to provide a 3D model as detailed (large) as possible as long as the computing power of the user device is sufficient. In addition, the view of the user device may include the distance between the user representative object and the 3D model, or the angle at which the user views the 3D model, but the present invention is not limited thereto. As shown in FIG. 8, if a partition is close to the user representative object, the 3D model processing system 100 may render the partition with higher detail (e.g., LoD0 in FIG. 3D), and partitions far from the user representative object may be rendered with lower detail (e.g., LoD2 in FIG. 3D). In other words, in the 3D model processing system 700, as the distance between the user representative object and the 3D model becomes closer, the first or second level of detail may be upgraded to provide more details of the 3D model (e.g., from LoD2 to LoD0 in FIG. 3D ), whereas as the distance between the user representative object and the 3D model becomes farther, the first or second level of detail may be downgraded to provide less details of the 3D model (e.g., from LoD0 to LoD2 in FIG. 3D ).

[0068] In one embodiment, the root node strategy may start with the lowest level of detail (e.g., LoD2 in FIG. 3D) for providing visual results to the user device. Note that the closer a partition of the 3D model is to the root node, the higher the visual distinctiveness (e.g., important feature) of the partition for the 3D model. The system then replaces nodes for their children and gradually improves the tree by gradually increasing the level of detail of the nodes. This strategy allows the system to comply with the limit of the amount of triangles loaded at one time. That is, compared to the 3D model processing system 700, instead of determining the level of detail of the partition to be provided to the user device based on the state of the user device, the level of detail of the partition to be provided to the user device may start from the lowest level of detail. In other words, in response to determining a request from the user device to view the 3D model, the lowest level of detail (e.g., LoD2 in FIG. 3D) may be determined as the first level of detail of the first stage partition and / or the second level of detail of the second stage partition. A lowest level of detail of the first tier partition and / or a lowest level of detail of the second tier partition may then be provided to the user device based on the first level of detail and / or the second level of detail.

[0069] In one embodiment, the connection speed of the user device or the computing power of the user device may be used to determine the level of detail of the partition to provide to the user device. That is, in response to the connection speed of the user device being higher than a speed threshold or the computing power of the user device being higher than a computing threshold, the first level of detail or the second level of detail may be upgraded to more detail of the 3D model. Conversely, in response to the connection speed of the user device being lower than a speed threshold or the computing power of the user device being lower than a computing threshold, the first level of detail or the second level of detail may be downgraded to less detail of the 3D model.

[0070] Based on the above, at least one of these strategies may be used to determine when to refine or simplify the level of detail of the 3D model. It should be noted that the best strategy to use depends on the specific application or design requirements, and the present invention is not limited thereto. Thus, by utilizing at least one of these strategies, the viewer logic (e.g., the user's distance, field of view, or line of sight) may be taken into account when displaying the 3D model in order to provide a better visual result with lower latency on the user device.

[0071] Fig. 9 is a flow diagram of a 3D model processing method according to one embodiment of the present invention. With reference to Fig. 1 and Fig. 9, the 3D model processing method 900 of this embodiment may be performed by the 3D model processing system 100 in Fig. 1, and the details of each step in Fig. 9 will be described below together with the components shown in Fig. 1.

[0072] First, in step S910, a first original spatial value and a second original spatial value of a 3D model are obtained. Next, in step S920, a difference value between the first original spatial value and the second original spatial value is obtained. After that, in step S930, a compressed spatial value corresponding to the second original spatial value based on the difference value is generated.

[0073] Based on the above, 3D model data can be compressed in an efficient manner, enabling real-time streaming of 3D model data, which is important for XR applications, thereby improving the user experience.

[0074] In addition, details of the implementation of the 3D model processing system 100 and the 3D model processing method 900 may be referred to the following description of Figures 10-11 for sufficient teachings, suggestions, and embodiments. In some embodiments, the functions of the 3D model processing system 100 and the 3D model processing method 900 may be performed by the processor 104, although the invention is not limited thereto.

[0075] 10 is a schematic diagram of a compression scenario for a 3D model according to one embodiment of the present invention. Referring to FIG.

[0076] In some embodiments, after the partitions of the 3D model are generated, each partition may be compressed to reduce the size of the data, thereby increasing the transmission and loading speed. The compression is specifically designed to decompress quickly, allowing the system to quickly load the required partition of the 3D model. For example, the XRGC format is a compressed data format designed for streaming 3D models, which can produce a good balance between decoding speed and file size. It is important to decode the XRGC data quickly in order to render the 3D model as quickly as possible. This is especially important for real-time applications such as games and virtual reality.

[0077] 10, a compression algorithm 1010 may be adapted to reduce the size of the data. That is, by performing the compression algorithm 1010 on the original data, compressed data may be generated, and the size of the compressed data may be smaller than the original data. The compression algorithm may include, for example, smart grouping of triangles, baking textures, flattening hierarchies, compressing position data (e.g., coordinates), compressing triangle data (e.g., vertex indices), compressing vectors, etc.

[0078] In one embodiment, the 3D model may include multiple spatial values ​​such as coordinates, vertex indices, etc. It is worth mentioning that the spatial values ​​are points such as mesh coordinates, but instead of storing multiple coordinates (e.g., data points) individually, the difference between consecutive coordinates may be stored. For example, a sequence of 2, 5, 4, 6, 8 may be stored by converting it into a sequence of 2, 3, -1, 2, 2. That is, the relative values ​​(i.e., the difference / distance) between each coordinate may be stored instead of the absolute value of each coordinate. By using this approach, the compression ratio can be greatly improved. Furthermore, the approach can be applied to the 3D model processing method 900, where the first original spatial value may be a first coordinate of the 3D model, and the second original spatial value may be a second coordinate of the 3D model. The first coordinate and the second coordinate may belong to the same mesh, but the invention is not limited thereto. That is, a compressed spatial value for the second original spatial value may be generated by replacing the second original spatial value with the difference between the first original spatial value and the second original spatial value.

[0079] In another embodiment, when the spatial value is a vertex index of a triangle, instead of storing the multiple vertex indexes individually, no value needs to be stored or a specific value (e.g., 0) may be stored when the multiple vertex indexes belong to a consecutive sequence. Meanwhile, the next vertex index needs to be stored only when the next vertex index is not consecutive with the current vertex index. For example, a sequence of vertex indexes V1, V2, V3, V4, V6 may be converted and stored as a sequence of 0, 0, 0, 0, V6. By using this measure, the compression ratio can be greatly improved. Furthermore, the measure can be applied to the 3D model processing method 900, in which the first original spatial value can be a first vertex index of a triangle of the 3D model, and the second original spatial value can be a second vertex index of the triangle of the 3D model. That is, a compressed spatial value for the second original spatial value can be generated by replacing the second original spatial value with a specific value (e.g., 0) according to whether the second vertex index is consecutive with the first vertex index.

[0080] Based on the above, whether the spatial values ​​are mesh coordinates or triangle vertex indices, the spatial values ​​can be compressed to reduce file size, although the compression algorithm 1010 may include these strategies as well as others described below.

[0081] In one embodiment, the position data (e.g., coordinates) may be stored typically as Cartesian coordinates. For example, a list of three values ​​for each point X, Y, Z may be stored. The data is arranged in an array such as [{x,y,z},{x,y,z},{x,y,z}]. In one embodiment, each channel (e.g., X, Y, or Z) may be processed separately. In other words, there is a separate list for each channel. However, the resulting compressed streams may still be interleaved with each other.

[0082] In one embodiment, the position data may include vertex indices of texture coordinates. The vertex positions are the coordinates of the vertices in the 3D model. The texture coordinates are the coordinates of the texture pixels that map to each vertex. The triangle indices may be stored separately, and the triangle indices are identification numbers that identify each triangle. Compressing the position data involves two steps: (1) storing the numbers as integers with N-bit precision, and (2) compressing the sequence of integers.

[0083] To store numbers as integers with N-bit precision, in one embodiment, consider a sequence of numbers with a minimum value MIN and a maximum value MAX. The range of values ​​of the numbers between MIN and MAX is 2 Npoints, where N is an integer indicating the number of bits of precision used. For each number in the sequence, the number may be stored as an index of the nearest discrete point. For example, when N is 11, the range of values ​​between a minimum (MIN) and a maximum (MAX) value of the sequence of numbers may be partitioned into 2048 points. That is, MIN may be stored as 0 and MAX may be stored as 2047, as shown in the table below. However, the invention is not so limited.

[0084] [Table 1]

[0085] Note that storing numbers in this manner may result in some loss of precision. For example, if the original numbers range from 0 to 8000, a range containing 8001 numbers can only be represented by 2048 discrete points (numbers). Thus, if a number in the sequence is 4001, it may be stored as the index of the 1024th discrete point even though it is actually closer to the 1023rd discrete point. The amount of precision lost depends on the value of N. The higher the value of N, the more precision is maintained. However, the higher the value of N, the more space is required to store the data. For this reason, it is important to select the value of N depending on the quality requirements. If the quality requirements are high, a higher value of N may be used. Conversely, if the quality requirements are not so high, a lower value of N may be used to save space.

[0086] To compress a sequence of integers, there may be three steps: (1) storing the decompressed numbers with full precision, (2) calculating the offset (difference value) to the next number, and (3) checking the range of the offset. For example, if the sequence of numbers is 2, 5, 4, 6, 8, these numbers may be stored directly in memory. Furthermore, the sequence of numbers may be converted to the sequence 2, 3, -1, 2, 2 by calculating the offset between each number. If there are more numbers in the sequence, step (2) may be performed again. Furthermore, after the offset or all offsets are calculated, the value / range of the offset may be checked if it is within a predefined range. For example, if the offset is within -127 to 127, 127 may be added to the offset and the result may be stored as an unsigned byte (i.e., the sign bit is removed). The purpose is to remove the negative sign. That is, in the 3D model processing method 900, in response to the difference value between the first spatial value and the second spatial value being negative, a compensation value may be added to ensure that the difference value between the first spatial value and the second spatial value is positive. For this reason, the sequence of numerical values ​​may be represented as an unsigned byte, which allows for a reduction of one bit (i.e., representing the sign) when sorting the data. Conversely, if the offset is outside the predetermined range (i.e., -127 to 127), a binary value of 255 may be stored to indicate that the numerical value will then be stored with full precision. However, the present invention is not limited thereto.

[0087] Note that the position data strategy takes advantage of the fact that geometry data is often stored in a very localized way: vertex positions are usually close to their neighbors, and in most cases only the offsets need to be stored. If the data is constantly jumping from MIN to MAX, the compression will be worse than storing the uncompressed numbers. However, in most cases the compression ratio is expected to be good, thus reducing the size of the 3D model file.

[0088] In one embodiment, the vertex indices of triangles in a 3D model are often stored in order. For example, a first triangle may have vertex indices V1, V2, V3, a second triangle may have vertex indices V4, V5, V6, etc. To take advantage of this, the system may store only a single bit to indicate that the index of the next vertex is also the next consecutive index. Specifically, as shown in Table 2 below, if a triangle T1 is stored with vertices V1, V2, V3, the index of the current vertex is V3 (i.e., the most recently stored vertex) and the index of the next vertex of the next triangle T2 is V4. The single bit of the index of the next vertex (i.e., V4) may be stored as 0 to indicate that the next index is the next consecutive index (i.e., 4, which immediately follows 3). This can save space since it only takes a single bit to store the next index of the next vertex rather than storing the entire data for the next vertex.

[0089] [Table 2]

[0090] On the other hand, if the next vertex index is not the next consecutive index but is one of the three previously used indexes, the system may store a 1 as the first bit of the next vertex index. The next two bits of the vertex next index may be used to store the three previously used indexes. That is, three bits may be used to represent the vertex index instead of a single bit. For example, if the current index of the vertex is V4 and the next index of the vertex is V1 (i.e., the first one of the previously used indexes), the next vertex index may be stored as 100. Similarly, if the next index of the vertex is V2 (i.e., the second one of the previously used indexes), the next vertex index may be stored as 101 or 110. However, the invention is not so limited. Additionally, if the next vertex index is not one of the three previously used indexes, the next vertex index may be stored as 111 and the actual index may be stored as the subsequent bits of the next vertex index. For example, the third vertex of triangle T2 may be stored as 111 and V6.

[0091] This allows the triangle data to be stored efficiently since only a single bit is stored to indicate when the next index is the next consecutive index. Furthermore, when the next index is not the next consecutive index but is one of the three previously used indexes, only three bits are stored. In this way, the next index can be stored in a relatively small amount of space, thereby reducing the loading time of the 3D model.

[0092] In one embodiment, in addition to the position data and triangle data, normals may be vectors that describe the angle and curvature of the mesh at each vertex position. To improve compression, the normals may be encoded or compressed using similar algorithms, the details of which will not be repeated here.

[0093] Based on the above, the compression algorithm 1010 may include different strategies for different components in the 3D model, so that by performing the compression algorithm 1010 on the original data of the 3D model, compressed data of the 3D model can be generated, and the size of the compressed data can be smaller than the original data, thereby reducing the loading time of the 3D model.

[0094] Figure 11 is a schematic diagram of a compression scenario for a 3D model according to one embodiment of the present invention. Referring to Figure 11, a compression scenario 1100 illustrates how triangles of a 3D model can be grouped to reduce the size of the 3D model.

[0095] In one embodiment, to improve the predictability of the mesh data (i.e., to increase the compression ratio), the mesh of the 3D model may be subdivided into groups of triangles. That is, in the 3D model processing method 900, before step S910, the mesh of nearby 3D models may be grouped into a plurality of groups of triangles. Note that when forming the groups, the triangles may be selected based on being close to each other and sharing as many vertices and edges as possible. For example, as shown in FIG. 11, to reduce the amount of data, nearby triangles (e.g., triangles that share an edge or adjacent triangles) are grouped together. This helps improve the compression ratio because the data of each group is more similar than the data of the entire mesh.

[0096] Furthermore, once the triangle groups are formed, a compression algorithm 1010 may be applied to the 3D model to further compress the data of each triangle group individually. For example, position data, triangle indices, triangle vertex indices, or normals may be compressed to reduce the file size of the 3D model, thereby reducing the loading time of the 3D model and achieving streaming of the 3D model in real time.

[0097] In summary, according to the 3D model processing system 100, the 3D model processing method 200, the 3D model processing method 700, and the 3D model processing method 900, the 3D model is formed, compressed, or loaded while taking into account the loading time from the user end, so that the 3D model can be streamed to the user device in real time, thereby improving the user experience.

[0098] It will be apparent to one skilled in the art that various modifications and variations can be made to the disclosed embodiments without departing from the scope or spirit of the invention. In view of the above, it is intended that the present invention cover modifications and variations provided they come within the scope of the appended claims and their equivalents. [Industrial Applicability]

[0099] The 3D model processing system, the 3D model processing method, and the storage circuit of the present invention can be used in applications that require high speed and low latency in 3D model rendering. [Explanation of symbols]

[0100] 100: 3D model processing system 102:Memory circuit 104: Processor 200, 700, 900: 3D model processing method S210, S220, S230, S240, S250, S260, S710, S720, S730, S740, S750, S910, S920, S930: Step 300A, 300B, 300C, 300D, 301C, 302C: Split scenario 310, 410: Original models 320, 330: Original partition 340: Stage partition 400: Bake scenario 420: Texture map 422: Compression map 424: Baked Texture Map 430: Baked partition 500: The Flattening Scenario 510: Non-flattened model hierarchy 520: Flattened Model Hierarchy 600: File format 800: View Scenario 1000, 1100: Compression scenario 1010: Compression algorithm

Claims

1. A memory circuit for storing program code; coupled to the memory circuit; Obtaining multiple nodes of a 3D model; partitioning the 3D model into a first original partition including a first group of nodes and a second original partition including a second group of nodes based on position or material; generating a plurality of first stage partitions at different levels of detail for the first original partition based on the first group of nodes; generating a plurality of second stage partitions at different levels of detail for the second original partition based on the second group of nodes; a processor accessing said program code for Including, 3D model processing system.

2. The processor further comprises: determining a request to view the 3D model; determining a first level of detail for the plurality of first stage partitions based on a state of a user device; determining a second level of detail for the plurality of second tier partitions based on the state of the user device; providing one of the plurality of first tier partitions to the user device based on the first level of detail; providing one of the plurality of second tier partitions to the user device based on the second level of detail. The 3D model processing system according to claim 1 .

3. The state of the user device includes a distance between a user representative object and the 3D model, a user's field of view, a line of sight of the user, a connection speed of the user device, or a computing power of the user device. The 3D model processing system according to claim 2 .

4. The processor further comprises: upgrading the first level of detail or the second level of detail to provide more details of the 3D model in response to a closer distance between the user representative object and the 3D model; The 3D model processing system according to claim 2 .

5. The processor further comprises: downgrading the first level of detail or the second level of detail to render less detail of the 3D model in response to an increased distance between the user representative object and the 3D model. The 3D model processing system according to claim 2 .

6. The processor further comprises: determining a request to view the 3D model; determining a lowest level of detail as a first level of detail for the plurality of first stage partitions; determining the lowest level of detail as a second level of detail for the plurality of second tier partitions; providing one of the plurality of first tier partitions to a user device based on the first level of detail; providing one of the plurality of second tier partitions to the user device based on the second level of detail. The 3D model processing system according to claim 1 .

7. the first original partition belongs to a first material and the second original partition belongs to a second material; The 3D model processing system according to claim 1 .

8. the first original partition belongs to a first location and the second original partition belongs to a second location; The 3D model processing system according to claim 1 .

9. The processor further comprises: The 3D model is converted into a texture map. Baking the texture by converting a high-resolution model of the texture of the 3D model into a low-resolution model; applying the baked texture to a mesh of the 3D model; The 3D model processing system according to claim 1 .

10. The processor further comprises: performing a flattening process to remove all layers in the 3D model; The 3D model processing system according to claim 1 .

11. Obtaining a plurality of nodes of a 3D model; partitioning the 3D model based on location or material into a first original partition including a first group of nodes and a second original partition including a second group of nodes; generating a plurality of first stage partitions at different levels of detail for the first original partition based on the first group of nodes; generating a plurality of second tier partitions at different levels of detail for the second original partition based on the second group of nodes; Including, 3D model processing method.

12. determining a request to view the 3D model; determining a first level of detail for the plurality of first tier partitions based on a state of a user device; determining a second level of detail for the plurality of second tier partitions based on the state of the user device; providing one of the plurality of first tier partitions to the user device based on the first level of detail; providing one of the plurality of second tier partitions to the user device based on the second level of detail; Further comprising: The method for processing a 3D model according to claim 11.

13. The state of the user device includes a distance between a user representative object and the 3D model, a user's field of view, a line of sight of the user, a connection speed of the user device, or a computing power of the user device.

13. A method for processing a 3D model according to claim 12.

14. upgrading the first level of detail or the second level of detail to provide more detail of the 3D model in response to a closer distance between the user representative object and the 3D model; Further comprising:

13. A method for processing a 3D model according to claim 12.

15. downgrading the first level of detail or the second level of detail to render less detail of the 3D model in response to an increased distance between the user representative object and the 3D model. Further comprising:

13. A method for processing a 3D model according to claim 12.

16. determining a request to view the 3D model; determining a lowest level of detail as a first level of detail for the plurality of first stage partitions; determining the lowest level of detail as a second level of detail for the plurality of second tier partitions; providing one of the plurality of first tier partitions to a user device based on the first level of detail; providing one of the plurality of second tier partitions to the user device based on the second level of detail; Further comprising: The method for processing a 3D model according to claim 11.

17. the first original partition belongs to a first material and the second original partition belongs to a second material; The method for processing a 3D model according to claim 11.

18. the first original partition belongs to a first location and the second original partition belongs to a second location; The method for processing a 3D model according to claim 11.

19. extracting the 3D model into a texture map; baking the texture by converting a high resolution model of the texture of the 3D model into a low resolution model; applying the baked texture to a mesh of the 3D model; Further comprising: The method for processing a 3D model according to claim 11.

20. The processor: Obtain multiple nodes of a 3D model, dividing the 3D model into a first original partition including a first group of nodes and a second original partition including a second group of nodes based on position or material; generating a plurality of first stage partitions at different levels of detail for the first original partition based on the first group of nodes; generating a plurality of second stage partitions at different levels of detail for the second original partition based on the second group of nodes. storing a program code configured to Non-transient memory circuits.

Images