Mesh segmentation

Abstract

This specification describes a method for segmenting a mesh into submeshes based on the properties of triangles. First, triangles are classified according to certain properties using their respective areas. A filtering process can be used to modify the classification according to adjacent triangles. Next, connected components are generated, and adjacent connected components are merged according to specific criteria. In some embodiments, connected components that share the most edges are merged. Using this technique, submeshes can be automatically generated without prior knowledge of the mesh generation stage.

Description

Technical Field

[0004] ,

[0003] ,

[0001] [Cross - reference to Related Applications] This application claims priority under 35 U.S.C. § 119 to U.S. Provisional Patent Application No. 63 / 513,493, entitled "MESH SEGMENTATION", filed on July 13, 2023, which is hereby incorporated by reference in its entirety for all purposes.

[0002] The present invention relates to 3D graphics. More specifically, the present invention relates to the coding of 3D graphics.

Background Art

[0003] In recent years, new methods for compressing volumetric content such as point clouds based on projections from 3D to 2D are being standardized. This method, also known as V3C (Visual Volume Video - based Compression), maps 3D volume data into several 2D patches, further arranges the patches in an atlas image, and then encodes them with a video encoder. The atlas image corresponds to the geometry of the points, their respective textures, and an occupancy map indicating which positions should be considered for point cloud reconstruction.

[0004] In 2017, MPEG issued a Call for Proposals (CfP) for point cloud compression. After evaluating several proposals, MPEG is currently considering two different techniques for point cloud compression: 3D native coding techniques (based on octree and similar coding methods), or conventional video coding after 3D-to-2D projection. For dynamic 3D scenes, MPEG uses Test Model Software (TMC2) based on patch surface modeling, projection of patches from 3D to 2D images, and coding of the 2D images by a video encoder such as HEVC. This method has proven to be more efficient than native 3D coding and can achieve competitive bitrates with acceptable quality.

[0005] Since coding 3D point clouds using projection-based methods (also known as video-based methods or V-PCC) has been successful, future versions of this standard are expected to include further 3D data, such as 3D meshes. However, the current version of this standard is only suitable for transmitting sets of unconnected points and lacks a mechanism for transmitting point connectivity, as required for 3D mesh compression.

[0006] Methods have also been proposed to extend the functionality of V-PCC to meshes. One possible method is to encode vertices using V-PCC and then encode connectivity using a mesh compression method such as TFAN or Edgebreaker. A limitation of this method is that the original mesh must be dense so that the point cloud generated from the vertices is not sparse and can be efficiently encoded after projection. Furthermore, since the order of vertices affects the coding of connectivity, different methods have been proposed for reorganizing the connectivity of the mesh. An alternative method for encoding sparse meshes is to encode the positions of 3D vertices using RAW patch data. Since RAW patches directly encode (x,y,z), in this method all vertices are encoded as RAW data, while connectivity is encoded by a similar mesh compression method as described above. Note that in RAW patches, vertices can be sent in any preferred order, so the order generated from connectivity coding can be used. While this method can encode sparse point clouds, RAW patches are not efficient for encoding 3D data, and further data such as triangular face attributes may be missing from this method. [Overview of the project] [Problems that the invention aims to solve]

[0007] This specification describes a method for segmenting a mesh into submeshes based on the properties of triangles. First, triangles are classified according to certain properties using their respective areas. A filtering process can be used to modify the classification according to adjacent triangles. Next, connected components are generated, and adjacent connected components are merged according to specific criteria. In some embodiments, connected components that share the most edges are merged. Using this technique, submeshes can be automatically generated without prior knowledge of the mesh generation stage. [Means for solving the problem]

[0008] In one embodiment, a method programmed in the device's non-temporary memory includes the steps of: classifying triangles in a mesh into one of several classifications based on the characteristics of the triangles; changing the classification of a triangle when it is determined that the triangle is an isolated triangle; generating connected components of the triangles that are connected and belong to the same classification; and merging adjacent connected components to generate a submesh. The method further includes the step of extending each submesh by adding one or more triangles to the boundary of the submesh so that the submeshes overlap. The characteristic of the triangle is the area of ​​the triangle. The classification among the several classifications includes a first classification when the area of ​​the triangle is equal to or greater than a threshold, and the classification among the several classifications includes a second classification when the area of ​​the triangle is less than the threshold. The threshold is generated by the user. The threshold is generated using artificial intelligence and machine learning. The threshold is adjusted per frame or per sequence. The determination that a triangle is an isolated triangle is based on comparing the classification of the triangle with the classifications of adjacent triangles, and if the classification of the triangle is different from the classifications of all adjacent triangles, then the triangle is an isolated triangle. The method further includes the step of tracking the mesh and using the mesh segmentation from the previous frame for the current frame.

[0009] In another embodiment, the device includes non-temporary memory for storing an application, the application for classifying triangles in a mesh into one of several classifications based on the characteristics of the triangles, changing the classification of the triangle when it is determined that the triangle is an isolated triangle, generating connected components of the triangles that are connected and belong to the same classification, merging adjacent connected components to generate a submesh, and a processor coupled to the memory and configured to process the application. The application is further configured to extend each submesh by adding one or more triangles to the boundary of the submesh so that the submeshes overlap. The characteristic of the triangle is the area of ​​the triangle. The classification among the several classifications includes a first classification when the area of ​​the triangle is equal to or greater than a threshold, and the classification among the several classifications includes a second classification when the area of ​​the triangle is less than the threshold. The threshold is generated by the user. The threshold is generated using artificial intelligence and machine learning. The threshold is adjusted per frame or per sequence. The determination that a triangle is an isolated triangle is based on comparing the classification of the triangle with the classifications of adjacent triangles, and if the classification of the triangle is different from the classifications of all adjacent triangles, then the triangle is an isolated triangle. The application is further configured to track the mesh and use mesh segmentation from the previous frame for the current frame.

[0010] In another embodiment, the system includes an encoder configured to encode content, and a decoder configured to classify triangles in a mesh into one of several classifications based on the characteristics of the triangles, to change the classification of the triangle when it is determined that the triangle is an isolated triangle, to generate connected components of the triangle that are connected and belong to the same classification, and to merge adjacent connected components to generate a submesh. The decoder is further configured to extend each submesh by adding one or more triangles to the boundary of the submesh so that the submeshes overlap. The characteristic of the triangle is the area of ​​the triangle. The classification among the several classifications includes a first classification when the area of ​​the triangle is equal to or greater than a threshold, and the classification among the several classifications includes a second classification when the area of ​​the triangle is less than the threshold. The threshold is generated by the user. The threshold is generated using artificial intelligence and machine learning. The threshold is adjusted per frame or per sequence. The determination that the triangle is an isolated triangle is based on comparing the classification of the triangle with the classifications of adjacent triangles, and if the classification of the triangle is different from the classifications of all adjacent triangles, then the triangle is an isolated triangle. The decoder is further configured to track the mesh and use the mesh segmentation from the previous frame for the current frame. [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows flowcharts of mesh segmentation methods according to several embodiments. [Figure 2] This figure shows exemplary coupling components according to several embodiments. [Figure 3] This figure shows exemplary coupling components according to several embodiments. [Figure 4] This figure shows exemplary coupling components according to several embodiments. [Figure 5] This figure shows exemplary coupling components according to several embodiments. [Figure 6] This figure shows illustrative images of submesh overlaps according to several embodiments. [Figure 7] This is a block diagram of an exemplary computer device configured to implement a mesh segmentation method according to several embodiments. [Modes for carrying out the invention]

[0012] A mesh typically consists of a set of polygons that describe the surface of a volume. Recently, the V3C standard, an international standard for encoding volume objects using video compression, was published. The new V-DMC standard extends the V3C standard for dynamic meshes. Furthermore, a mesh can be divided into submeshes and encoded individually. Typically, when generating a dynamic mesh, areas containing more detail will have a higher triangle density, improving the surface description, and these particular areas may have their own mesh segments to improve encoding. However, this segmentation information is not always available. Therefore, this specification describes a method for identifying areas with high triangle density and segmenting the mesh in a meaningful way, without prior knowledge of the mesh generation stage.

[0013] This specification describes a method for segmenting a mesh into submeshes based on the properties of triangles. First, triangles are classified according to certain properties using their respective areas. A filtering process can be used to modify the classification according to adjacent triangles. Next, connected components are generated, and adjacent connected components are merged according to specific criteria. In some embodiments, connected components that share the most edges are merged. Using this technique, submeshes can be automatically generated without prior knowledge of the mesh generation stage.

[0014] Mesh coding using submeshes involves mesh segmentation. Mesh generation typically uses smaller triangles to describe high-resolution features such as a human face or hand. However, simply segmenting the mesh according to the dimensions of the triangles is not always sufficient, as smaller triangles may exist in meaningless areas used only to generate the surface of the mesh. This specification describes a method for dividing the mesh according to the properties of the triangles, while simultaneously ensuring some regularity in the surface of the submesh.

[0015] Figure 1 shows flowcharts of mesh segmentation methods according to several embodiments. In step 100, triangles are classified by area. For example, triangles with an area equal to or greater than a threshold are classified into a first classification / category (e.g., large triangle classification), and triangles with an area less than the threshold are classified into a second classification / category (e.g., small triangle classification). Taking this example further, if the area of ​​triangle 1 is 110, the area of ​​triangle 2 is 44, and the area of ​​triangle 3 is 132, and the threshold is 100, then triangles 1 and 3 are classified into the first classification (Class A / large triangle), and triangle 2 is classified into the second classification (Class B / small triangle). As shown in Figure 150, triangle 152 of Class A is a large triangle, and triangle 154 of Class B is a small triangle. Triangle 152 of Class A is mainly found on the hair, neck, and body, while triangle 154 of Class B is mainly found on the face. As shown in the diagram, some Class A triangles 152 are present on the face and surrounded by Class B triangles 154. Additionally, some Class B triangles 154 are present on the hair and body and surrounded by Class A triangles 152.

[0016] The threshold that determines which category each triangle belongs to can be determined, adjusted, or optimized per frame or per sequence. For example, the threshold might be 105 for frame 0, and 99 for frame 1, and so on. In some embodiments, the threshold is determined manually (e.g., by user input) or automatically. In some embodiments, the threshold is adjusted using artificial intelligence (AI) and / or machine learning (ML). For example, using AI / ML, the device learns whether the threshold is producing a proper classification that yields the best results (e.g., fewer isolated triangles or more accurate results). In some embodiments, learning is based on the user reviewing the computer-generated threshold and providing feedback. In another example, the distribution of triangles can be analyzed, and the threshold can be determined based on the distribution (e.g., based on the mean or median of the distribution). In yet another example, a clustering algorithm or other algorithm (e.g., Otsu's method, expectation maximization) can be used to determine / optimize the threshold.

[0017] In step 102, the classification of isolated triangles is changed. For example, if a triangle of class B is surrounded by triangles of class A, the triangle of class B is changed to a triangle of class A. The determination that a triangle of one classification is surrounded by triangles of another classification can be done in any way, such as by comparing the classifications of each triangle to compare adjacent triangles (for example, determining which triangles share vertices and / or edges with each other). If a triangle's classification is different from all of its adjacent triangles, its classification is changed to match that of its adjacent triangles. As shown in Figure 160, the classification of isolated triangles has been changed. Figure 160 still has some triangles that appear to be misclassified (for example, some large triangles remaining in the face). This is because those triangles are all two or more triangles of class A 152, and they are not isolated triangles because they are surrounded by triangles of class B 154.

[0018] In step 104, according to connectivity and classification, connected components (e.g., sub-meshes) are generated. For example, if several adjacent triangles have the same classification, these triangles can be used to generate a connected component. Image 170 shows various connected components 172, 174, 176, and 178.

[0019] In step 106, adjacent connected components are merged / joined. For example, if adjacent or related connected components are determined to have the same classification, these connected components can be joined. For example, in image 170, since connected component 174 surrounds connected component 176, the vicinity of connected component 176 is compared with connected component 174. Connected component 174 is a triangle 152 of class A and surrounds connected component 176, so connected component 176 is merged with connected component 174. The merged connected component retains the classification of the larger connected component (e.g., connected component 174). Connected component 178 is similar to connected component 176 and is merged with connected component 174. For connected components on the boundary of two larger connected components (e.g., the part where the face touches the hair or the part where the face touches the neck), depending on the implementation, it can be determined which larger connected component to merge with. For example, depending on the implementation, it can be alternately switched which larger connected component the smaller connected component merges with.

[0020] In some embodiments, the connected components are joined until only two connected components (e.g., the face and the body) remain. Image 180 shows the two remaining connected components 172 and 174.

[0021] In some embodiments, fewer or additional steps are implemented. For example, in some embodiments, to allow overlap, the sub-meshes can be extended by adding triangles to the boundary so that the sub-meshes overlap each other. In some embodiments, the order of the steps is changed.

[0022] In some embodiments, instead of or in addition to using area for classification, one or more other triangle characteristics (e.g., the size of the edges of the triangle) are used. In some embodiments, an initial classification (e.g., using the size of the edges of the triangle) is performed, and then another classification (e.g., the area of the triangle) is performed.

[0023] In some embodiments, when a mesh is being tracked, the mesh segmentation from the previous frame is utilized for the current frame. In some embodiments, instead of using only two classifications, additional classifications (e.g., three classifications of face, hand, and body) are implemented by using multiple thresholds (e.g., body if above the first threshold, hand if between the first and second thresholds, and face if below the second threshold). In some embodiments, based on the area of the triangles, more mesh segmentations can be performed (e.g., generating sub-meshes of the hand). For example, instead of merging all the connected components of the body into one sub-mesh, the merging of the connected components stops at a specific size or other threshold such that the hand becomes a separate sub-mesh from the body sub-mesh. In some embodiments, instead of using the area of the triangle for the initial segmentation, the normal direction of each triangle is used. The mesh is segmented according to the phase of the normal, and all parts of the body facing the front camera are grouped together, and all parts of the body facing the rear camera are grouped together.

[0024] Figure 2 shows exemplary connected components according to some embodiments. Since this example is a video sequence, 32 frames are shown. Image 200 shows a connected component of the body that includes larger triangles (e.g., class A). Image 202 shows a connected component of the face that includes smaller triangles (e.g., class B). Each slice is one frame.

[0025] Figure 3 shows exemplary connected components according to some embodiments. Image 300 shows a connected component of the body that includes larger triangles (e.g., class A). Image 302 shows a connected component of the face that includes smaller triangles (e.g., class B).

[0026] Figure 4 shows exemplary connecting components according to several embodiments. Image 400 shows a body connecting component including larger triangles (e.g., Class A). Image 402 shows a face connecting component including smaller triangles (e.g., Class B).

[0027] Figure 5 shows exemplary connecting components according to several embodiments. Image 500 shows a body connecting component including larger triangles (e.g., Class A). Image 502 shows a face connecting component including smaller triangles (e.g., Class B).

[0028] Figure 6 shows illustrative images of submesh overlaps according to several embodiments. The first submesh 600 adds triangle 602 from the second submesh 604. For example, a triangle from the second submesh 604 is duplicated and added to the first submesh 600. Similarly, a triangle 602' from the first submesh 600 is duplicated and added to the second submesh 604. By duplicating triangles and adding them to each submesh, an overlap exists between the two submeshes.

[0029] Figure 7 shows a block diagram of an exemplary computer device configured to implement a mesh segmentation method according to several embodiments. The computer device 700 can be used to acquire, store, compute, process, communicate, and / or display information such as images and videos, including 3D content. The computer device 700 can implement either an encoding / decoding mode. Generally, a suitable hardware structure for implementing the computer device 700 includes a network interface 702, memory 704, a processor 706, (one or more) I / O devices 708, a bus 710, and a storage device 712. The choice of processor is not critical as long as a suitable processor with sufficient speed is selected. The memory 704 can be any conventional computer memory known in the art. The storage device 712 can include a hard drive, CD-ROM, CDRW, DVD, DVDRW, high-definition disk / drive, ultra-high-definition drive, flash memory card, or any other storage device. The computer device 700 can include one or more network interfaces 702. An example of a network interface includes a network card connected to Ethernet or another type of LAN. The (single or multiple) I / O device 708 may include one or more of the following devices: keyboard, mouse, monitor, screen, printer, modem, touchscreen, button interface, and other devices. The (single or multiple) mesh segmentation application 730 used to implement the mesh segmentation implementation is likely to be stored in the storage device 712 and memory 704 and processed as the application would normally be processed. The computer device 700 may include more or fewer components than those shown in Figure 7. In some embodiments, mesh segmentation hardware 720 is included. The computer device 700 in Figure 7 includes the application 730 and hardware 720 for the mesh segmentation implementation, but the mesh segmentation method may be implemented in the computer device as hardware, firmware, software, or any combination thereof.For example, in some embodiments, the mesh segmentation application 730 is programmed in memory and executed using a processor. In another example, in some embodiments, the mesh segmentation hardware 720 is programmed hardware logic including gates specifically designed to implement a mesh segmentation method.

[0030] In some embodiments, the (single or multiple) mesh segmentation application 730 includes several applications and / or modules. In some embodiments, a module also includes one or more submodules. In some embodiments, fewer or additional modules may be included.

[0031] Examples of suitable computer devices include personal computers, laptop computers, computer workstations, servers, mainframe computers, handheld computers, personal digital assistants, cellular phones / mobile phones, smart home appliances, game consoles, digital cameras, digital camcorders, camera phones, smartphones, portable music players, tablet computers, mobile devices, video players, video disc writers / players (e.g., DVD writers / players, high-definition disc writers / players, ultra-high-definition disc writers / players), televisions, home entertainment systems, augmented reality devices, virtual reality devices, smart jewelry (e.g., smartwatches), vehicles (e.g., autonomous vehicles), or any other suitable computer devices.

[0032] To utilize the mesh segmentation method, the device acquires or receives 3D content (e.g., point cloud content). The mesh segmentation method can be implemented with or without user assistance.

[0033] During operation, the mesh segmentation method results in fewer artifacts and more detailed faces.

[0034] Several embodiments of mesh segmentation 1. A method programmed into the non-temporary memory of a device, The steps include classifying triangles within a mesh into one of several classifications based on the characteristics of the triangles, When it is determined that a triangle is an isolated triangle, the step of changing the classification of the triangle, The steps of generating connected components of the triangles that are connected and belong to the same classification, The steps include merging adjacent connected components to generate a submesh, A method that includes this.

[0035] 2. The method of the first paragraph, further comprising the step of extending each submesh by adding one or more triangles to the boundary of the submesh so that the submeshes overlap.

[0036] 3. The characteristic of the triangle is the area of ​​the triangle, as described in paragraph 1.

[0037] 4. The method according to paragraph 3, wherein the classification among the plurality of classifications includes a first classification when the area of ​​the triangle is equal to or greater than a threshold, and the classification among the plurality of classifications includes a second classification when the area of ​​the triangle is less than the threshold.

[0038] 5. The threshold is generated by the user as described in Section 4.

[0039] 6. The threshold is generated using artificial intelligence and machine learning, as described in paragraph 4.

[0040] 7. The method according to paragraph 4, wherein the threshold is adjusted for each frame or sequence.

[0041] 8. The method according to paragraph 1, wherein the determination that the triangle is an isolated triangle is based on comparing the classification of the triangle with the classification of adjacent triangles, and if the classification of the triangle is different from the classification of all adjacent triangles, the triangle is an isolated triangle.

[0042] 9. The method according to paragraph 1, further comprising the step of tracking the mesh and using the mesh segmentation from the previous frame for the current frame.

[0043] 10. Apparatus, Non-temporary memory for storing applications, wherein the applications are The triangles within the mesh are classified into one of several categories based on the characteristics of the triangles. When a triangle is determined to be an isolated triangle, the classification of the triangle is changed. The connected components of the triangles are generated, and the triangles are connected and belong to the same classification. Merge adjacent connected components to generate a submesh. Non-temporary memory is intended for this purpose, A processor coupled to the memory and configured to process the application, A device that includes this.

[0044] 11. The apparatus according to paragraph 10, wherein the application is further configured to extend each submesh by adding one or more triangles to the boundary of the submesh so that the submeshes overlap.

[0045] 12. The apparatus described in paragraph 10, wherein the characteristic of the triangle is the area of ​​the triangle.

[0046] 13. The apparatus according to paragraph 12, wherein the classification among the plurality of classifications includes a first classification when the area of ​​the triangle is equal to or greater than a threshold, and the classification among the plurality of classifications includes a second classification when the area of ​​the triangle is less than the threshold.

[0047] 14. The threshold is generated by the user, as described in paragraph 13.

[0048] 15. The apparatus described in paragraph 13, wherein the threshold is generated using artificial intelligence and machine learning.

[0049] 16. The apparatus described in paragraph 13, wherein the threshold is adjusted for each frame or sequence.

[0050] 17. The apparatus according to paragraph 10, wherein the determination that the triangle is an isolated triangle is based on comparing the classification of the triangle with the classification of adjacent triangles, and if the classification of the triangle is different from the classification of all adjacent triangles, then the triangle is an isolated triangle.

[0051] 18. The apparatus according to paragraph 10, wherein the application is further configured to track the mesh and use the mesh segmentation from the previous frame for the current frame.

[0052] 19. A system, An encoder configured to encode content, The triangles within the mesh are classified into one of several categories based on the characteristics of the triangles. When a triangle is determined to be an isolated triangle, the classification of the triangle is changed. The connected components of the triangles are generated, and the triangles are connected and belong to the same classification. Merge adjacent connected components to generate a submesh. A decoder configured as follows, A system that includes this.

[0053] 20. The system according to paragraph 19, wherein the decoder is further configured to extend each submesh by adding one or more triangles to the boundary of the submesh so that the submeshes overlap.

[0054] 21. The system described in paragraph 19, wherein the characteristic of the triangle is the area of ​​the triangle.

[0055] 22. The system according to paragraph 21, wherein the classification among the plurality of classifications includes a first classification when the area of ​​the triangle is equal to or greater than a threshold, and the classification among the plurality of classifications includes a second classification when the area of ​​the triangle is less than the threshold.

[0056] 23. The threshold is generated by the user, as described in paragraph 22.

[0057] 24. The threshold is generated using artificial intelligence and machine learning, as described in paragraph 22.

[0058] 25. The system described in paragraph 22, wherein the threshold is adjusted frame by frame or sequence by sequence.

[0059] 26. The determination that the triangle is an isolated triangle is based on comparing the classification of the triangle with the classifications of adjacent triangles, and if the classification of the triangle is different from the classifications of all adjacent triangles, then the triangle is an isolated triangle, as described in paragraph 19.

[0060] 27. The system according to paragraph 19, wherein the decoder is further configured to track the mesh and use the mesh segmentation from the previous frame for the current frame.

[0061] To facilitate understanding of the structure and operating principles of the present invention, the invention has been described in relation to specific embodiments incorporating detailed features. Such references to specific embodiments and their details in this specification are not intended to limit the claims appended herein. It will be readily apparent to those skilled in the art that various other modifications can be made to the embodiments selected for illustrative purposes without departing from the spirit and scope of the invention as defined by the claims. [Explanation of Symbols]

[0062] Classify 100 triangles 102 Change the category of isolated triangles Create 104 CC Merging 106 CCs 150 images 152 Class A triangle 154 Class B triangle 160 images 170 images 172,174,176,178 connected components 180 images 200 images 202 images 300 images 302 images 400 images 402 Images 500 images 502 images 600 First submesh 602 triangle 602' triangle 604 Second submesh 700 Computer equipment 702 Network Interface 704 memory 706 Processor 708 I / O devices 710 Bus 712 Storage device 720 Mesh Segmentation Hardware 730 Mesh Segmentation Applications

Claims

1. A method programmed into the device's non-temporary memory, The steps include classifying triangles within a mesh into one of several classifications based on the characteristics of the triangles, When it is determined that a triangle is an isolated triangle, the step of changing the classification of the triangle, The steps of generating connected components of the triangles that are connected and belong to the same classification, The steps include merging adjacent connected components to generate a submesh, A method characterized by including the following.

2. The method according to claim 1, further comprising the step of expanding each submesh by adding one or more triangles to the boundary of the submesh so that the submeshes overlap.

3. The method according to claim 1, characterized in that the characteristic of the triangle is the area of ​​the triangle.

4. The method according to claim 3, characterized in that the classification among the plurality of classifications includes a first classification when the area of ​​the triangle is equal to or greater than a threshold, and the classification among the plurality of classifications includes a second classification when the area of ​​the triangle is less than the threshold.

5. The method according to claim 4, characterized in that the threshold is generated by the user.

6. The method according to claim 4, characterized in that the threshold is generated using artificial intelligence and machine learning.

7. The method according to claim 4, characterized in that the threshold is adjusted for each frame or for each sequence.

8. The method according to claim 1, wherein the determination that the triangle is an isolated triangle is based on comparing the classification of the triangle with the classification of adjacent triangles, and if the classification of the triangle is different from the classification of all adjacent triangles, the triangle is an isolated triangle.

9. The method according to claim 1, further comprising the step of tracking the mesh and using the mesh segmentation from the previous frame for the current frame.

10. It is a device, Non-temporary memory for storing applications, wherein the applications are The triangles within the mesh are classified into one of several categories based on the characteristics of the triangles. When a triangle is determined to be an isolated triangle, the classification of the triangle is changed. The connected components of the triangles are generated, and the triangles are connected and belong to the same classification. Merge adjacent connected components to generate a submesh. Non-temporary memory is intended for this purpose, A processor coupled to the memory and configured to process the application, An apparatus characterized by including

11. The apparatus according to claim 10, wherein the application is further configured to expand each submesh by adding one or more triangles to the boundary of the submesh so that the submeshes overlap.

12. The apparatus according to claim 10, characterized in that the characteristic of the triangle is the area of ​​the triangle.

13. The apparatus according to claim 12, characterized in that the classification among the plurality of classifications includes a first classification when the area of ​​the triangle is equal to or greater than a threshold, and the classification among the plurality of classifications includes a second classification when the area of ​​the triangle is less than the threshold.

14. The apparatus according to claim 13, characterized in that the threshold is generated by the user.

15. The apparatus according to claim 13, characterized in that the threshold is generated using artificial intelligence and machine learning.

16. The apparatus according to claim 13, characterized in that the threshold is adjusted for each frame or for each sequence.

17. The apparatus according to claim 10, wherein the determination that the triangle is an isolated triangle is based on comparing the classification of the triangle with the classification of adjacent triangles, and if the classification of the triangle is different from the classification of all adjacent triangles, the triangle is an isolated triangle.

18. The apparatus according to claim 10, wherein the application is further configured to track the mesh and use mesh segmentation from the previous frame for the current frame.

19. It is a system, An encoder configured to encode content, The triangles within the mesh are classified into one of several categories based on the characteristics of the triangles. When a triangle is determined to be an isolated triangle, the classification of the triangle is changed. The connected components of the triangles are generated, and the triangles are connected and belong to the same classification. Merge adjacent connected components to generate a submesh. A decoder configured as follows, A system characterized by including

20. The system according to claim 19, wherein the decoder is further configured to expand each submesh by adding one or more triangles to the boundary of the submesh so that the submeshes overlap.

21. The system according to claim 19, characterized in that the characteristic of the triangle is the area of ​​the triangle.

22. The system according to claim 21, characterized in that the classification among the plurality of classifications includes a first classification when the area of ​​the triangle is equal to or greater than a threshold, and the classification among the plurality of classifications includes a second classification when the area of ​​the triangle is less than the threshold.

23. The system according to claim 22, characterized in that the threshold is generated by the user.

24. The system according to claim 22, characterized in that the threshold is generated using artificial intelligence and machine learning.

25. The system according to claim 22, characterized in that the threshold is adjusted for each frame or sequence.

26. The system according to claim 19, characterized in that the determination that the triangle is an isolated triangle is based on comparing the classification of the triangle with the classification of adjacent triangles, and if the classification of the triangle is different from the classification of all adjacent triangles, then the triangle is an isolated triangle.

27. The system according to claim 19, further characterized in that the decoder is configured to track the mesh and use the mesh segmentation from the previous frame for the current frame.

Images