Three-dimensional mesh coding method and apparatus, three-dimensional mesh decoding method and apparatus, and device

Abstract

Disclosed in the present application are a three-dimensional mesh coding method and apparatus, a three-dimensional mesh decoding method and apparatus, and a device, belonging to the technical field of computers. The three-dimensional mesh information coding method in the embodiments of the present application comprises: acquiring first texture map information of a crack region, the crack region being a crack region between sub-meshes of three-dimensional meshes; and coding the first texture map information to obtain a first code stream.

Description

3D mesh encoding methods, decoding methods, devices and equipment

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202410508141.8, filed in China on April 25, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application belongs to the field of computer technology, specifically relating to a three-dimensional mesh encoding method, decoding method, apparatus, and related equipment. Background Technology

[0004] In the encoding and decoding of 3D meshes, some related technologies often encode only the sub-mesh of the 3D mesh, specifically encoding the geometric, connectivity, and attribute information of the sub-mesh. However, this approach, which only encodes the sub-mesh, may result in poor encoding performance for the 3D mesh itself. Summary of the Invention

[0005] This application provides a three-dimensional mesh encoding method, decoding method, apparatus, and device that can solve the problem of poor encoding performance of three-dimensional meshes.

[0006] Firstly, a three-dimensional mesh encoding method is provided, including:

[0007] Obtain the first texture map information of the crack region, wherein the crack region is the crack region between sub-mesh of the three-dimensional mesh;

[0008] The first texture map information is encoded to obtain the first bitstream.

[0009] Secondly, a three-dimensional mesh decoding method is provided, including:

[0010] Obtain the first bitstream;

[0011] The first bitstream is decoded to obtain the first texture map information of the crack region, wherein the crack region is the crack region between sub-mesh of the three-dimensional mesh.

[0012] Thirdly, a three-dimensional mesh encoding device is provided, comprising:

[0013] The acquisition module is used to acquire the first texture map information of the crack region, wherein the crack region is the crack region between sub-mesh of the three-dimensional mesh;

[0014] The first encoding module is used to encode the first texture map information to obtain the first bitstream.

[0015] Fourthly, a three-dimensional mesh decoding device is provided, comprising:

[0016] The first acquisition module is used to acquire the first bitstream;

[0017] The first decoding module is used to decode the first bitstream to obtain the first texture map information of the crack region, wherein the crack region is the crack region between sub-mesh of the three-dimensional mesh.

[0018] Fifthly, a three-dimensional mesh encoding apparatus is provided, the apparatus being configured to perform the steps of the method described in the first aspect.

[0019] In a sixth aspect, a three-dimensional mesh decoding apparatus is provided, the apparatus being configured to perform the steps of the method described in the second aspect.

[0020] In a seventh aspect, an electronic device is provided, the terminal including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method as described in the first aspect, or implementing the steps of the method as described in the second aspect.

[0021] Eighthly, an electronic device is provided, including a processor and a communication interface, wherein the processor is configured to acquire first texture map information of a crack region, the crack region being a crack region between sub-mesh areas of a three-dimensional mesh; and to encode the first texture map information to obtain a first bitstream. Alternatively, the communication interface is configured to acquire the first bitstream, and the processor is configured to decode the first bitstream to obtain first texture map information of the crack region, the crack region being a crack region between sub-mesh areas of a three-dimensional mesh.

[0022] A ninth aspect provides an electronic device comprising: a memory configured to store video data, and processing circuitry configured to implement the steps of the method described in the first aspect, or the steps of the method described in the second aspect.

[0023] In a tenth aspect, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect, or implement the steps of the method described in the second aspect.

[0024] Eleventhly, an encoding / decoding system is provided, comprising: an encoding end device and a decoding end device, wherein the encoding end device can be used to perform the steps of the method described in the first aspect, and the decoding end device can be used to perform the steps of the method described in the second aspect.

[0025] In a twelfth aspect, a chip is provided, the chip including a processor and a communication interface coupled to the processor, the processor being configured to run a program or instructions to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the second aspect.

[0026] In a thirteenth aspect, a computer program / program product is provided, which is stored in a storage medium and is executed by at least one processor to implement the steps of the method as described in the first aspect, or to implement the steps of the method as described in the second aspect.

[0027] In this embodiment, first texture map information of the crack region is obtained, wherein the crack region is a crack region between sub-mesh of a 3D mesh; the first texture map information is encoded to obtain a first bitstream. Thus, by encoding the texture map information of the crack region, the encoding performance of the 3D mesh can be improved. Attached Figure Description

[0028] Figure 1 is a schematic diagram of a system provided in an embodiment of this application;

[0029] Figure 2 is a flowchart of a three-dimensional mesh encoding method provided in an embodiment of this application;

[0030] Figures 3a to 3c are schematic diagrams of sub-grids provided in embodiments of this application;

[0031] Figure 3d is a schematic diagram of a geometric mapping texture provided in an embodiment of this application;

[0032] Figure 4 is a schematic diagram of a texture overlap detection and adjustment provided in an embodiment of this application;

[0033] Figure 5 is a flowchart of a three-dimensional mesh decoding method provided in an embodiment of this application;

[0034] Figure 6 is a schematic diagram of a three-dimensional mesh encoding provided in an embodiment of this application;

[0035] Figure 7 is a schematic diagram of a three-dimensional mesh decoding provided in an embodiment of this application;

[0036] Figure 8 is a schematic diagram of another three-dimensional mesh encoding provided in an embodiment of this application;

[0037] Figure 9 is a schematic diagram of another three-dimensional mesh decoding provided in an embodiment of this application;

[0038] Figure 10 is a structural diagram of a three-dimensional mesh encoding device provided in an embodiment of this application;

[0039] Figure 11 is a structural diagram of a three-dimensional mesh decoding device provided in an embodiment of this application;

[0040] Figure 12 is a structural diagram of an electronic device provided in an embodiment of this application;

[0041] Figure 13 is a structural diagram of a terminal provided in an embodiment of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0043] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, "or" in this application indicates at least one of the connected objects. For example, the scope of protection for "A or B" covers at least three scenarios: Scenario 1: including A but not B; Scenario 2: including B but not A; Scenario 3: including both A and B. In addition, the terms "A and / or B," "at least one of A and B," and "at least one of A or B" also cover at least the above three scenarios. The character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0044] The term "instruction" in this application can be either a direct instruction (or explicit instruction) or an indirect instruction (or implicit instruction). A direct instruction can be understood as one in which the sender explicitly informs the receiver of specific information, the operation to be performed, or the requested result, etc., in the instruction sent. An indirect instruction can be understood as one in which the receiver determines the corresponding information based on the instruction sent by the sender, or makes a judgment and determines the operation to be performed or the requested result, etc., based on the judgment result.

[0045] Figure 1 shows a block diagram of a system applicable to an embodiment of this application. The system includes an encoding end device 11 and a decoding end device 12. The encoding end device 11 can be any one or more of the following: server, mobile phone, desktop computer, laptop computer, tablet computer, set-top box, mobile phone, wearable device, television, camera, display device, in-vehicle device, virtual reality (VR) device, augmented reality (AR) device, mixed reality (MR) device, digital media player, video game console, video conferencing equipment, video streaming equipment, broadcast receiver device, broadcast transmitter device, spacecraft, aircraft, robot, in-vehicle device, flight vehicle, satellite, shipborne equipment, pedestrian user equipment (PUE), smart home (home devices with wireless communication capabilities, such as refrigerators, televisions, washing machines, or furniture), game console, personal computer (PC), ATM, or self-service machine, etc.

[0046] The decoding device 12 can be any one or more of the following: server, mobile phone, desktop computer, laptop computer, tablet computer, set-top box, mobile phone, wearable device, television, camera, display device, in-vehicle device, virtual reality (VR) device, augmented reality (AR) device, mixed reality (MR) device, digital media player, video game console, video conferencing equipment, video streaming device, broadcast receiver device, broadcast transmitter device, spacecraft, aircraft, robot, in-vehicle device, flight vehicle, satellite, shipborne device, pedestrian user equipment (PUE), smart home (home devices with wireless communication capabilities, such as refrigerators, televisions, washing machines, or furniture), game console, personal computer (PC), ATM, or self-service machine.

[0047] The wearable devices mentioned above include: smartwatches, smart bracelets, smart earphones, smart glasses, smart jewelry (smart bracelets, smart chains, smart rings, smart necklaces, smart anklets, smart ankle chains, etc.), smart wristbands, smart clothing, etc.

[0048] The aforementioned vehicle-mounted equipment can also be referred to as a vehicle-mounted terminal, vehicle-mounted controller, vehicle-mounted module, vehicle-mounted component, vehicle-mounted chip, or vehicle-mounted unit, etc. It should be noted that the specific types of the encoding end device 11 and the decoding end device 12 are not limited in the embodiments of this application.

[0049] In recent years, with the rapid development of multimedia technology, related research results have been rapidly industrialized and have become an indispensable part of people's lives. Three-dimensional models have emerged as a new generation of digital media following audio, images, and video. Three-dimensional meshes and point clouds are two commonly used methods for representing three-dimensional models. Compared with traditional multimedia such as images and videos, three-dimensional mesh models have stronger interactivity and realism, leading to their increasingly widespread application in various fields such as commerce, manufacturing, construction, education, medicine, entertainment, art, and the military.

[0050] As people's demands for visual effects in 3D mesh models increase, and with the emergence of many more mature 3D scanning technologies and 3D modeling software, the data scale and complexity of 3D mesh models obtained through 3D scanning equipment or 3D modeling software are also growing rapidly.

[0051] A 3D mesh typically contains three main types of information: topological information, geometric information, and attribute information. Topological information, also known as connectivity information, describes the connections between elements such as vertices and faces in the mesh. Geometric information consists of the 3D coordinates of all vertices in the mesh. Attribute information records other information attached to the mesh, such as normal vectors, texture coordinates, and color. Compression of 3D mesh data usually involves compressing these three types of information according to their respective data characteristics. Additionally, for 3D meshes with texture maps, the texture maps can also be compressed.

[0052] The following description, in conjunction with the accompanying drawings, details a three-dimensional mesh encoding method, decoding method, apparatus, and device provided in this application through some embodiments and application scenarios.

[0053] Please refer to Figure 2, which is a flowchart of a three-dimensional mesh encoding method provided in an embodiment of this application. As shown in Figure 2, it includes the following steps:

[0054] Step 201: Obtain the first texture map information of the crack region, wherein the crack region is the crack region between sub-mesh of the three-dimensional mesh.

[0055] The aforementioned three-dimensional mesh can be divided into multiple sub-mesh, such as 2, 3, 4 or more sub-mesh.

[0056] The aforementioned crack region can be all or part of the crack region between sub-mesh of a three-dimensional mesh.

[0057] In some implementations, the aforementioned crack region may represent one or more crack regions. When there are multiple crack regions, step 201 obtains the first texture map information for each crack region.

[0058] The cracked areas mentioned above can be understood as gaps between sub-grids, i.e., boundary cracks of sub-grids.

[0059] In some embodiments, the aforementioned crack region may be a crack region generated during sub-mesh merging, or a crack region generated due to the loss of sub-region boundary points, or a crack region generated due to boundary point displacement caused by quantization, etc. The causes of crack region generation are not limited in the embodiments of this application.

[0060] For example, as shown in Figure 3a, after subdividing the original 3D mesh into submesh, we can obtain submesh 1 and submesh 2. The dashed lines in the submesh mark points or edges on the boundary. The numbers in the original 3D mesh represent the global index of the boundary point, i.e., the index of the boundary point in the 3D mesh. The numbers in the submesh represent the local index of the boundary point, i.e., the index of the boundary point in the submesh.

[0061] Sub-mesh 1 and sub-mesh 2 are decoded independently (e.g., lossy encoding and decoding). Since decoding requires simplification and quantization of each sub-mesh independently, the boundary points with index 0 of sub-mesh 1 are lost during the simplification process, as shown in Figure 3b. This causes the boundaries of the two sub-meshes to be misaligned, resulting in cracks, as shown in Figure 3c. The triangles with indices 1, 2, and 4 are the crack regions caused by decoding.

[0062] The first texture map information of the aforementioned crack region can be understood as texture map information used to fill the crack region. The aforementioned first texture map information can represent the texture of the aforementioned crack region, or it can be understood as the first texture map information used to represent the texture map of the aforementioned crack region.

[0063] In some implementations, the first texture map information may also be referred to as first texture information, or the first texture map information may also be referred to as supplementary information, that is, the first texture map information may serve as a supplement to the three-dimensional mesh encoding.

[0064] Step 202: Encode the first texture map information to obtain the first bitstream.

[0065] The encoding of the first texture map information described above can be performed using video encoding, or by quantizing data, predictive compression (parallelogram prediction), or entropy encoding, etc., without limitation.

[0066] In this embodiment of the application, the above-mentioned three-dimensional mesh encoding method is executed by the encoding end device, that is, the encoding end device executes the above-mentioned steps 201 and 202.

[0067] In some embodiments, the encoding device can also send the first bitstream to the decoding device, which decodes the first bitstream to obtain the first texture map information, thereby filling the texture of the cracked area and obtaining the reconstructed mesh of the three-dimensional mesh.

[0068] In this embodiment, the above steps enable the encoding of texture map information in the crack region, thereby improving the encoding performance of the 3D mesh. Furthermore, because the texture map information in the crack region is encoded, the decoding device can perform texture filling on the crack region based on the first texture map information, thus improving decoding performance.

[0069] As an optional implementation, the method further includes:

[0070] Determine the geometric patch to fill the crack area;

[0071] Determine the texture information of the geometric patch, wherein the texture information includes at least one of texture coordinates and texture triangles;

[0072] The first texture map information is the texture map information corresponding to the texture information.

[0073] The aforementioned geometric patches can be geometric patches used in the three-dimensional mesh compression standard V-DMC defined by the Moving Pictures Experts Group (MPEG), an international standards organization for the video image field, such as geometric triangular patches.

[0074] The geometric patch that fills the crack region can be understood as the boundary of the stitched sub-mesh.

[0075] The geometric patch used to fill the crack area can be determined by filling.

[0076] The aforementioned determination of the texture information of the geometric facet can be achieved by determining the texture coordinates or texture triangles corresponding to the geometric facet in the texture map or texture domain.

[0077] The texture information of the aforementioned geometric facets can also be referred to as the texture information that distinguishes the cracks.

[0078] The aforementioned first texture map information is the texture map information corresponding to the texture information, which can be understood as the first texture map information being the texture map information of the region represented by the texture information.

[0079] In this embodiment, since the geometric patch is used to fill the crack area, the first texture map information of the crack area is determined by the texture information of the geometric patch. This allows for accurate acquisition of the first texture map information of the crack area, thereby improving coding performance.

[0080] It should be noted that the embodiments of this application are not limited to determining the first texture map information through the above-mentioned geometric patches and texture information. In some implementations, the first texture map information can also be determined directly based on the geometric position of the crack region, such as a pre-established mapping relationship between geometric position and texture map, and the first texture map information can be determined based on the mapping relationship.

[0081] In some implementations, the texture map information corresponding to the texture coordinates or texture triangles can be extracted or cropped from the texture map of the three-dimensional network as the first texture map information.

[0082] Optionally, determining the geometric patch for filling the crack region includes:

[0083] Obtain the reconstructed sub-mesh of the sub-mesh of the 3D mesh;

[0084] The crack region of the reconstructed submesh of the submesh of the three-dimensional mesh is filled with boundary to obtain a geometric patch for filling the crack region.

[0085] The reconstructed submesh of the aforementioned three-dimensional mesh can be obtained by reconstructing all or part of the submesh of the three-dimensional mesh.

[0086] The boundary filling methods described above can include, but are not limited to, boundary growth or boundary connection. For example, taking boundary filling of a reconstructed submesh as an example, it can be done by statistically analyzing the triangles shared by the mesh edges. If an edge is shared by two triangles, then the edge is an internal edge; if the edge is shared by one triangle, then the edge is a boundary edge. Boundary growth is performed along the boundary edges of the crack region, that is, geometric patches are added outward along the direction of the boundary edges. When the patches grown in the crack region intersect, the boundary growth stops, completing the geometric filling of the crack region, thereby determining the geometric patches used to fill the crack region.

[0087] In this embodiment, the geometric patches used to fill the crack area can be determined by boundary filling, which can improve the accuracy of the geometric patches and thus improve the accuracy of the texture map information.

[0088] It should be noted that the embodiments of this application do not limit the encoding end device to determine the above-mentioned geometric surface by boundary filling. For example, in some embodiments, other devices may configure the above-mentioned geometric surface and send it to the above-mentioned encoding end device.

[0089] Optionally, determining the texture information of the geometric patch includes:

[0090] Based on the characteristic relationship between geometric patches and texture information, the texture information of the geometric patches is predicted.

[0091] The aforementioned characteristic relationships can be pre-configured, such as the similarity in shape between geometric triangles and textured triangles. For example, as shown in Figure 3d, triangle ABC is the geometric triangle filling the crack region, and the geometric coordinates of its three vertices are (x, y, y) and (x, y, y). a ,y a ,z a ), (x b ,y b ,z b ) and (x c ,y c ,z c Then, the texture coordinates C' of point C can be mapped from the geometric coordinates of points A, B, and C to the texture coordinates A' and B' of A and B. This method calculates the ratio factor between the length h_geo of the perpendicular line from C to AB in the geometric triangle and the length of AB. Based on this ratio factor and the length of A'B' in the texture, the height h_uv of C' to A'B' in the texture triangle can be calculated, thus mapping the texture coordinates C' of C.

[0092] In this embodiment, the accuracy of texture information is improved based on the characteristic relationship between geometric patches and texture information.

[0093] It should be noted that the embodiments of this application are not limited to predicting the texture information of geometric patches through the above-mentioned characteristic relationships. For example, the texture information of geometric patches can be predicted by methods such as difference prediction coding or parallelogram prediction coding.

[0094] As an optional implementation, the texture triangle corresponding to the first texture map information does not overlap with the original texture triangle of the sub-mesh of the three-dimensional mesh.

[0095] The texture triangle corresponding to the first texture map information can be the texture triangle that the first texture map information needs to fill, such as the texture triangle of the geometric patch in the above embodiment.

[0096] In this context, the original texture triangle of the sub-mesh of the aforementioned 3D mesh can refer to the texture triangle corresponding to the original texture map of the sub-mesh.

[0097] In this embodiment, since the texture triangle corresponding to the first texture map information does not overlap with the original texture triangle of the sub-mesh of the three-dimensional mesh, the texture map of the sub-mesh will not be interfered with when the crack area is textured, thereby improving the texture filling effect.

[0098] Optionally, the texture triangle corresponding to the first texture map information is determined in the following way:

[0099] Determine the texture triangle of the crack area;

[0100] When the texture triangle in the crack region overlaps with the original texture triangle of the sub-mesh of the 3D mesh, the texture triangle in the crack region is shrunk to obtain a texture triangle corresponding to the first texture map information that does not overlap with the original texture triangle.

[0101] The texture triangle used to determine the crack region can be found in the method for determining the texture information of the geometric facets described in the above embodiments, and will not be repeated here.

[0102] The shrinking of texture triangles in the crack region described above can be achieved by iteratively shrinking the texture triangles based on a preset scaling factor until they no longer overlap with the original texture triangles or the distance is greater than a preset distance threshold. For example, as shown in Figure 4 (top), triangle A'B'C' is a newly added texture triangle in the texture domain corresponding to the geometric triangle facet ABC, which is the texture triangle in the crack region. However, the added texture triangle overlaps with the texture triangle of the original mesh, which will cause the texture information to be disordered during the final texture mapping, thus affecting the quality of the reconstructed mesh. Therefore, the scaling factor, i.e., the ratio of the height h_geo from C to AB in the geometric triangle to the length of AB, is scaled to a certain extent by multiplying by a scaling factor such as 2 / 3 or 3 / 4, and iteratively scaling with the same scaling factor until the newly added texture triangle no longer overlaps with the texture triangle of the original mesh, as shown in Figure 4 (bottom). At this time, the added texture triangle A'B'C' is the texture triangle corresponding to the geometric triangle ABC used to fill the crack region, realizing the addition of texture information.

[0103] It should be noted that the embodiment of this application does not limit the method of shrinking the texture triangle in the crack area. For example, it can be shrunk according to the preset scaling factor, or it can be shrunk according to the crack size, etc.

[0104] In the above embodiments, by shrinking the texture triangles in the crack area, texture triangles that do not overlap with the original texture triangles can be effectively obtained, thereby improving the texture filling effect.

[0105] It should be noted that in some implementations, in order to improve encoding and decoding efficiency, the overlap detection of texture triangles in the crack area can be omitted, and the information of the first texture map can be obtained directly, which can also improve encoding performance.

[0106] As an optional implementation, encoding the first texture map information to obtain the first bitstream includes:

[0107] The first texture map information and the second texture map information are encoded to obtain a first bitstream, wherein the second texture map information is the texture map information of the sub-mesh of the three-dimensional mesh.

[0108] The aforementioned second texture map information may include the texture map information of all or part of the sub-mesh of the 3D mesh. In this embodiment, the method of obtaining the second texture map information is not limited; for example, the texture map information of each sub-mesh may be determined directly based on the texture map of the 3D mesh.

[0109] In this embodiment, since the first texture map information and the second texture map information are encoded to obtain the first bitstream, the decoding device can more reliably and effectively fill the sub-mesh and crack areas with textures to obtain the reconstructed mesh of the three-dimensional mesh.

[0110] It should be noted that, in some implementations, the first texture map information and the second texture map information mentioned above can be encoded separately, or they can be transmitted to the decoding device through different bitstreams.

[0111] As an optional implementation, the method further includes:

[0112] The instruction information is encoded to obtain a second bitstream;

[0113] or,

[0114] The first bitstream includes: the encoding result of encoding the indication information;

[0115] The indication information is used to indicate whether the three-dimensional mesh contains the first texture map information.

[0116] The aforementioned indication information can be determined based on the geometric patch used to fill the crack area.

[0117] Whether the aforementioned three-dimensional mesh contains the first texture map information can be understood as whether the texture map information of the three-dimensional network contains the aforementioned first texture map information, or it can be understood as whether the bitstream corresponding to the three-dimensional mesh contains the encoding result of the first texture map information.

[0118] The aforementioned indication information is used to indicate whether the three-dimensional mesh contains the first texture map information, which can be understood as whether the first texture map information needs to be transmitted or added. After obtaining the indication information, the decoding device performs texture filling on the crack area based on the first texture map information.

[0119] The first bitstream mentioned above includes the encoding result of encoding the indication information, which can be understood as the indication information and the first texture map information being encoded together.

[0120] In some implementations, when the first texture map information is encoded, the value of the indication information is an indication that the first texture map information is included, that is, the indication information is used to indicate that the three-dimensional mesh includes the first texture map information.

[0121] The above-mentioned instruction information enables the encoding device to fill the crack area with texture based on the first texture map information when the instruction contains the first texture map information, thereby improving the decoding performance.

[0122] It should be noted that in some implementations, the first texture map information may be included by default, thereby saving bitstream overhead.

[0123] In some embodiments, the first texture map information may include, in addition to texture map information, identification information of the boundary of the crack region (such as a gap) to be filled, or texture coordinates or texture triangle information. This allows the decoding device to more reliably perform texture filling on the corresponding region based on the first texture map information. Alternatively, in some embodiments, when multiple crack regions are included, the first texture map information does not include the identification information, texture coordinates, or texture triangle information of the boundary of the crack region to be filled. The first texture information of multiple crack regions is stored according to a specific rule, so that the decoding device can determine the first texture map information corresponding to each crack region based on the specific rule.

[0124] In this embodiment, first texture map information of the crack region is obtained, wherein the crack region is a crack region between sub-mesh of a 3D mesh; the first texture map information is encoded to obtain a first bitstream. Thus, by encoding the texture map information of the crack region, the encoding performance of the 3D mesh can be improved.

[0125] Please refer to Figure 5, which is a flowchart of a three-dimensional mesh decoding method provided in an embodiment of this application. As shown in Figure 5, it includes the following steps:

[0126] Step 501: Obtain the first bitstream;

[0127] Step 502: Decode the first bitstream to obtain the first texture map information of the crack region, wherein the crack region is the crack region between sub-mesh of the three-dimensional mesh.

[0128] In this embodiment of the application, the above-mentioned three-dimensional mesh information encoding method is executed by the decoding end device, that is, the decoding end device executes the above-mentioned steps 501 and 502.

[0129] In this embodiment of the application, the above steps can be used to decode the first texture map information of the crack region, thereby improving the decoding performance of the three-dimensional mesh.

[0130] Optionally, the above method further includes:

[0131] The cracked area is textured based on the first texture map information.

[0132] A more accurate reconstructed mesh is obtained by texturing the cracked areas.

[0133] Optionally, the method further includes:

[0134] Determine the geometric patch to fill the crack area;

[0135] Determine the texture information of the geometric patch, wherein the texture information includes at least one of texture coordinates and texture triangles;

[0136] The crack region is texture-filled based on the texture information and the first texture map information.

[0137] The above-mentioned texture filling of the crack region based on the texture information and the first texture map information can be achieved by filling the crack region with the first texture map information based on the texture information, such as filling the texture domain of the crack region with the first texture map information, or filling the crack region with the first texture map information in the texture map obtained by the decoding device.

[0138] Optionally, determining the geometric patch for filling the crack region includes:

[0139] Obtain the reconstructed sub-mesh of the sub-mesh of the 3D mesh;

[0140] The crack region of the reconstructed submesh of the submesh of the three-dimensional mesh is filled with boundary to obtain a geometric patch for filling the crack region.

[0141] Optionally, determining the texture information of the geometric patch includes:

[0142] Based on the characteristic relationship between geometric patches and texture information, the texture information of the geometric patches is predicted.

[0143] Optionally, the texture triangle corresponding to the first texture map information does not overlap with the original texture triangle of the sub-mesh of the three-dimensional mesh.

[0144] Optionally, the texture triangle corresponding to the first texture map information is determined in the following way:

[0145] Determine the texture triangle of the crack area;

[0146] When the texture triangle in the crack region overlaps with the original texture triangle of the sub-mesh of the 3D mesh, the texture triangle in the crack region is shrunk to obtain a texture triangle corresponding to the first texture map information that does not overlap with the original texture triangle.

[0147] Optionally, decoding the first bitstream to obtain the first texture map information of the crack region includes:

[0148] The first bitstream is decoded to obtain the first texture map information and the second texture map information of the crack region, wherein the second texture map information is the texture map information of the sub-mesh of the three-dimensional mesh.

[0149] The various implementation methods described above can be found in the descriptions of the embodiments shown in Figure 2, which will not be repeated here.

[0150] In some implementations, the texture information and geometric patches mentioned above can also be received and sent by the encoding device. For example, the encoding device can encode the texture information and geometric patches and send them to the decoding device, thereby saving the overhead of the decoding device.

[0151] Optionally, the method further includes:

[0152] Acquire the second bitstream and decode it to obtain the indication information;

[0153] Alternatively, the information obtained by decoding the first bitstream may also include indication information;

[0154] The indication information is used to indicate whether the three-dimensional mesh contains the first texture map information.

[0155] It should be noted that this embodiment is an implementation of the decoding end corresponding to the embodiment shown in Figure 2. For the specific implementation, please refer to the relevant description of the embodiment shown in Figure 2. In order to avoid repeated description, this embodiment will not be repeated.

[0156] The method provided in the embodiments of this application is illustrated below using the framework shown in Figures 6 and 7:

[0157] Encoding end:

[0158] As shown in Figure 6, the process includes submesh generation, submesh compression, crack region geometry filling, crack region texture filling, and texture map conversion. These processes are described in detail below:

[0159] 1. Sub-mesh generation:

[0160] Input: Original mesh (i.e., 3D network)

[0161] Output: Submesh after partitioning

[0162] In this embodiment, the sub-mesh partitioning method is not limited; for example, it can be based on bounding box partitioning or other methods. The specific partitioning method is not emphasized here. The following description uses bounding box partitioning as an example.

[0163] First, based on the geometric coordinates of the input original mesh, its bounding box in 3D space can be obtained. Then, the mesh is divided equally based on the x-axis, y-axis, or z-axis coordinate range. The number of divisions can be preset or set according to the required number of sub-mesh elements. After division, the bounding box range of each sub-mesh is obtained. For each bounding box range, all faces in the original mesh are traversed, and vertices in the input mesh that satisfy the bounding box range are divided into the current sub-mesh on a face-by-face basis. After traversing all the divided bounding boxes, the final divided sub-mesh is obtained.

[0164] II. Subgrid Coding

[0165] Input: Submesh after partitioning

[0166] Output: Subgrid bitstream

[0167] In this process, each sub-mesh is independently encoded. The encoding of each sub-mesh mainly consists of connectivity encoding, geometric information encoding, and attribute information encoding. An edgebreaker-based method can be used to encode each sub-mesh; the specific encoding method is not emphasized here. The following description uses the edgebreaker-based encoding method as an example.

[0168] The Edgebreaker-based 3D mesh encoding method is shown in Figure 8. First, the non-manifold structures in the input sub-mesh are decomposed to obtain a manifold mesh. Then, the connectivity, geometric information, and attribute information of the manifold mesh are encoded. For connectivity encoding, the Edgebreaker method can be used to encode the connectivity relationships of the 3D mesh. A CornerTable is established to represent the mesh connectivity, and all triangles in the mesh are traversed using the CornerTable to generate the Edgebreaker's CLERS pattern string. Then, entropy encoding can be used to encode the CLERS pattern string to obtain the connectivity information bitstream. Geometric information can be encoded according to the encoding order of connectivity relationships using various methods such as difference prediction encoding and parallelogram prediction encoding to obtain the geometric information bitstream. Specific encoding methods are not emphasized here. 3D mesh attribute information generally includes texture coordinates and normal vectors; taking texture coordinates as an example, texture coordinates can be encoded according to the encoding order of connectivity relationships using methods such as difference prediction encoding, parallelogram prediction encoding, and similar triangle prediction encoding to obtain the attribute information bitstream. Specific encoding methods are not limited here.

[0169] III. Geometric Filling of Crack Areas

[0170] Input: Reconstruct submesh

[0171] Output: Geometry-free reconstructed mesh

[0172] There are various ways to geometrically fill crack regions, such as boundary growth or boundary connection. This application does not limit the filling method. The following is an example of one filling method:

[0173] The reconstructed sub-mesh is then filled with boundaries, which involves statistically analyzing the triangles shared by mesh edges: if an edge is shared by two triangles, it is considered an internal edge; if it is shared by one triangle, it is considered a boundary edge. Boundary growth is then performed along the boundary edges of the crack region, adding geometric triangular facets outwards along the direction of the boundary edges. Boundary growth stops when the grown triangular facets in the crack region intersect, completing the geometric filling of the crack region.

[0174] IV. Texture Filling in Crack Areas

[0175] Input: Geometrically crack-free reconstructed mesh

[0176] Output: Supplementary information identifier bitstream (such as the second bitstream in the above embodiment), reconstructed mesh.

[0177] This module adds corresponding texture information to the newly added geometric patches in the crack region geometry filling module. Based on the geometric patches, it determines whether texture information needs to be added, thereby obtaining a supplementary information identifier (i.e., the indication information in the above embodiment). This identifier indicates whether supplementary texture information (i.e., the first texture map information in the above embodiment) needs to be transmitted to the decoding end. This identifier information is then encoded using a specific encoding method to obtain the supplementary information identifier bitstream.

[0178] If corresponding texture information needs to be added, the texture information corresponding to the geometric facet needs to be obtained (i.e., texture coordinates or texture triangles in the above embodiment). There are multiple ways to obtain texture information, and this embodiment does not limit the specific method of acquisition. The following is an example of prediction based on similar triangles:

[0179] The texture triangle corresponding to the filled geometric triangle is predicted based on the similarity in shape between geometric triangles and texture triangles. For example, as shown in Figure 3d, triangle ABC is a filled geometric patch (i.e., a geometric triangle), and the geometric coordinates of its three vertices are (x, y, x, y) and (x, y, y) respectively. a ,y a ,z a ), (x b ,y t ,z b ) and (x c ,y c ,z c Then, the texture coordinates C' of point C can be mapped from the geometric coordinates of points A, B, and C to the texture coordinates A' and B' of A and B. This method calculates the scaling factor between the length h_geo of the perpendicular line from C to AB in the geometric triangle and the length of AB. Based on this scaling factor and the length of A'B' in the texture, the height h_uv of C' to A'B' in the texture triangle is calculated, and the texture coordinates C' of C are mapped out.

[0180] During texture triangle prediction, the predicted texture triangles must not overlap with the original sub-mesh texture triangles. Overlap detection is required between newly added texture triangles and the original texture triangles. If the new texture triangle overlaps with the original texture triangle, the new texture triangle is iteratively scaled down according to a preset scaling factor until it no longer overlaps with the original texture triangle or the distance is greater than a preset distance threshold. This embodiment does not limit the detection and adjustment methods; one method is given below as an example:

[0181] As shown in Figure 4 (top), triangle A'B'C' is the newly added texture triangle in the texture domain corresponding to the grown face ABC. However, the added texture triangle overlaps with the texture triangle of the original mesh, which will cause the texture information to be disordered during the final texture mapping, thus affecting the quality of the reconstructed mesh. Therefore, the scaling factor, i.e., the ratio of the height h_geo from C to AB in the geometric triangle to the length of AB, is scaled to a certain extent by multiplying by a scaling factor such as 2 / 3 or 3 / 4. The same scaling factor is used iteratively until the newly added texture triangle no longer overlaps with the texture triangle of the original mesh, as shown in Figure 4 (bottom). At this time, the added texture triangle A'B'C' is the texture triangle corresponding to the newly grown geometric triangle ABC, realizing the addition of texture information.

[0182] V. Texture Map Conversion and Encoding

[0183] Input: Reconstructed mesh, texture map

[0184] Output: Texture map bitstream

[0185] This module primarily regenerates texture map information for the original mesh (i.e., the second texture map information in the above embodiment), and adds supplementary information (i.e., the first texture map information in the above embodiment) to newly added geometric faces, then packages and encodes the original texture map information and supplementary information together. The texture filling algorithm at the decoding end can only obtain texture coordinates and texture triangles, lacking the corresponding texture map information (i.e., the first texture map information in the above embodiment). Therefore, this module regenerates the texture map for the original mesh and adds texture map information to the newly added geometric faces as supplementary information needed for texture filling at the decoding end. This embodiment does not limit the method of obtaining the supplementary information; specifically, it can combine the original texture map with texture map conversion for the reconstructed mesh used to fill crack areas. Encoding the supplementary information and the original mesh texture information together can be done using methods such as video encoding.

[0186] Decoding end:

[0187] As shown in Figure 7, the decoding mainly includes: sub-mesh decoding, supplementary information decoding, crack region geometry filling, and crack region texture filling, which will be described in detail below:

[0188] I. Subgrid Decoding

[0189] Input: Subgrid bitstream

[0190] Output: Reconstructed submesh

[0191] In this process, each sub-mesh is decoded independently. The decoding of each sub-mesh mainly consists of connection relationship decoding, geometric information decoding, and attribute information decoding. A decoding method corresponding to the encoding method used at the encoding end is employed to decode each piece of information. The following description uses the Edgebreaker-based decoding method shown in Figure 9 as an example.

[0192] First, the manifold mesh is decoded for connectivity, geometry, and properties. For connectivity decoding, pattern strings are first obtained. These strings are traversed in a certain order (forward or reverse), and connectivity is reconstructed based on the corresponding patterns. The traversal order of the vertices is then output to the geometry and property decoding modules. Geometry information is decoded using the same decoding method as the encoder, following the connectivity decoding order. Similarly, 3D mesh properties are decoded using the same decoding method as the encoder, following the connectivity decoding order. Based on the decoded connectivity, geometry, and property information, the manifold submesh can be directly reconstructed.

[0193] II. Texture Map Decoding

[0194] Input: Texture map bitstream (i.e., the first bitstream in the above embodiment)

[0195] Output: Supplementary texture map information (i.e., the first texture map information and the second texture map information in the above embodiments).

[0196] This module can decode the texture map bitstream using video decoding to obtain supplementary texture map information and the original mesh texture map (i.e., the first texture map information and the second texture map information in the above embodiment).

[0197] III. Geometric Filling of Crack Areas

[0198] Input: Reconstruct submesh

[0199] Output: Geometry-free reconstructed mesh

[0200] This module uses the same geometric filling algorithm as the encoding end to geometrically fill the reconstructed sub-mesh, resulting in a geometrically crack-free reconstructed mesh.

[0201] IV. Texture Filling in Crack Areas

[0202] Input: Geometrically crack-free reconstructed mesh, supplementary information identifier (i.e., the indication information in the above embodiments), supplementary information (i.e., the first texture map information in the above embodiments).

[0203] Output: Reconstructed Mesh

[0204] This module employs the same texture triangle prediction method as the encoding end to fill the texture of the geometrically crack-free reconstructed mesh. Simultaneously, it combines supplementary information identifiers to obtain supplementary information, and then applies the obtained texture map information, i.e., the supplementary information, to the cracked areas of the reconstructed mesh to achieve texture filling of the cracked areas, thus obtaining the reconstructed mesh.

[0205] This application proposes a method for filling geometric and texture information of cracks in the boundary of a 3D mesh containing multiple sub-mesh. In this embodiment, at the encoding end, geometric triangles are used to fill the crack regions between decoded sub-mesh, and corresponding texture triangles are generated based on these geometric triangles. The corresponding texture is then obtained, such as filling the texture map with the corresponding texture. Finally, the decoding device decodes the bitstream to guide it in filling the crack regions between decoded sub-mesh. The decoded information includes an identifier indicating whether the bitstream contains sub-mesh boundary stitching information and first texture map information. If the bitstream contains the first texture map information, the decoding device performs geometric triangle filling on the gaps between sub-mesh in the same manner as at the encoding end and generates corresponding texture triangles. Filling the triangle textures with the first texture map information completes the filling of the gaps between sub-mesh.

[0206] The three-dimensional mesh encoding method provided in this application can be executed by a three-dimensional mesh encoding device. As an example, the three-dimensional mesh encoding device can be an electronic device or a component within an electronic device, such as a chip or circuit. This application uses the execution of the three-dimensional mesh encoding method by a three-dimensional mesh encoding device as an example to illustrate the three-dimensional mesh encoding device provided in this application.

[0207] The three-dimensional mesh decoding method provided in this application can be executed by a three-dimensional mesh decoding device. As an example, the three-dimensional mesh decoding device can be an electronic device or a component within an electronic device, such as a chip or circuit. This application uses the execution of the three-dimensional mesh decoding method by a three-dimensional mesh decoding device as an example to illustrate the three-dimensional mesh decoding device provided in this application.

[0208] Please refer to Figure 10, which is a structural diagram of a three-dimensional mesh encoding device provided in an embodiment of this application. As shown in Figure 10, the three-dimensional mesh encoding device 1000 includes:

[0209] The acquisition module 1001 is used to acquire the first texture map information of the crack region, wherein the crack region is the crack region between sub-mesh of the three-dimensional mesh;

[0210] The first encoding module 1002 is used to encode the first texture map information to obtain the first bitstream.

[0211] Optionally, the device further includes:

[0212] The first determining module is used to determine the geometric patch to fill the crack region;

[0213] The second determining module is used to determine the texture information of the geometric patch, wherein the texture information includes at least one of texture coordinates and texture triangles;

[0214] The first texture map information is the texture map information corresponding to the texture information.

[0215] Optionally, determining the geometric patch for filling the crack region includes:

[0216] Obtain the reconstructed sub-mesh of the sub-mesh of the 3D mesh;

[0217] The crack region of the reconstructed submesh of the submesh of the three-dimensional mesh is filled with boundary to obtain a geometric patch for filling the crack region.

[0218] Optionally, determining the texture information of the geometric patch includes:

[0219] Based on the characteristic relationship between geometric patches and texture information, the texture information of the geometric patches is predicted.

[0220] Optionally, the texture triangle corresponding to the first texture map information does not overlap with the original texture triangle of the sub-mesh of the three-dimensional mesh.

[0221] Optionally, the texture triangle corresponding to the first texture map information is determined in the following way:

[0222] Determine the texture triangle of the crack area;

[0223] When the texture triangle in the crack region overlaps with the original texture triangle of the sub-mesh of the 3D mesh, the texture triangle in the crack region is shrunk to obtain a texture triangle corresponding to the first texture map information that does not overlap with the original texture triangle.

[0224] Optionally, encoding the first texture map information to obtain the first bitstream includes:

[0225] The first texture map information and the second texture map information are encoded to obtain a first bitstream, wherein the second texture map information is the texture map information of the sub-mesh of the three-dimensional mesh.

[0226] Optionally, the device further includes:

[0227] The second encoding module is used to encode the indication information to obtain the second bitstream;

[0228] or,

[0229] The first bitstream includes: the encoding result of encoding the indication information;

[0230] The indication information is used to indicate whether the three-dimensional mesh contains the first texture map information.

[0231] The aforementioned three-dimensional mesh encoding device can improve the encoding performance of three-dimensional meshes.

[0232] The three-dimensional mesh encoding device provided in this application embodiment can implement all the processes implemented in the method embodiment of FIG2 and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0233] Please refer to Figure 11, which is a structural diagram of a three-dimensional mesh decoding device provided in an embodiment of this application. As shown in Figure 11, the three-dimensional mesh decoding device 1100 includes:

[0234] The first acquisition module 1101 is used to acquire the first bitstream;

[0235] The first decoding module 1102 is used to decode the first bitstream to obtain the first texture map information of the crack region, wherein the crack region is the crack region between sub-mesh of the three-dimensional mesh.

[0236] Optionally, the device further includes:

[0237] The first determining module is used to determine the geometric patch to fill the crack region;

[0238] The second determining module is used to determine the texture information of the geometric patch, wherein the texture information includes at least one of texture coordinates and texture triangles;

[0239] A filling module is used to fill the crack region with texture based on the texture information and the first texture map information.

[0240] Optionally, determining the geometric patch for filling the crack region includes:

[0241] Obtain the reconstructed sub-mesh of the sub-mesh of the 3D mesh;

[0242] The crack region of the reconstructed submesh of the submesh of the three-dimensional mesh is filled with boundary to obtain a geometric patch for filling the crack region.

[0243] Optionally, determining the texture information of the geometric patch includes:

[0244] Based on the characteristic relationship between geometric patches and texture information, the texture information of the geometric patches is predicted.

[0245] Optionally, the texture triangle corresponding to the first texture map information does not overlap with the original texture triangle of the sub-mesh of the three-dimensional mesh.

[0246] Optionally, the texture triangle corresponding to the first texture map information is determined in the following way:

[0247] Determine the texture triangle of the crack area;

[0248] When the texture triangle in the crack region overlaps with the original texture triangle of the sub-mesh of the 3D mesh, the texture triangle in the crack region is shrunk to obtain a texture triangle corresponding to the first texture map information that does not overlap with the original texture triangle.

[0249] Optionally, decoding the first bitstream to obtain the first texture map information of the crack region includes:

[0250] The first bitstream is decoded to obtain the first texture map information and the second texture map information of the crack region, wherein the second texture map information is the texture map information of the sub-mesh of the three-dimensional mesh.

[0251] Optionally, the device further includes:

[0252] The second decoding module is used to acquire the second bitstream and decode the second bitstream to obtain indication information;

[0253] or,

[0254] The information obtained by decoding the first bitstream also includes indication information;

[0255] The indication information is used to indicate whether the three-dimensional mesh contains the first texture map information.

[0256] The aforementioned 3D mesh decoding device can improve the encoding performance of 3D meshes.

[0257] The three-dimensional mesh decoding device provided in this application embodiment can implement the various processes implemented in the method embodiment of FIG5 and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0258] As shown in Figure 12, this application embodiment also provides an electronic device 1200, including a processor 1201 and a memory 1202. The memory 1202 stores a program or instructions that can run on the processor 1201. For example, when the electronic device 1200 is an encoding device, the program or instructions executed by the processor 1201 implement the various steps of the above-described three-dimensional mesh encoding method or three-dimensional mesh decoding method embodiments, and can achieve the same technical effect. When the electronic device 1200 is a decoding device, the program or instructions executed by the processor 1201 implement the various steps of the above-described three-dimensional mesh encoding method or three-dimensional mesh decoding method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0259] This application also provides an electronic device, including: a memory configured to store video data; and a processing circuit configured to implement the various steps of the above-described three-dimensional mesh encoding method or three-dimensional mesh decoding method embodiments.

[0260] This application also provides an electronic device, including a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps in the three-dimensional mesh encoding method shown in FIG2 or the three-dimensional mesh decoding method shown in FIG5. This device embodiment corresponds to the above method embodiment, and all implementation processes and methods of the above method embodiments can be applied to this terminal embodiment and can achieve the same technical effect.

[0261] The processor or processing circuit in this application embodiment may include general-purpose processors, special-purpose processors, etc., such as central processing units (CPUs), microprocessors, digital signal processors (DSPs), artificial intelligence (AI) processors, graphics processing units (GPUs), application-specific integrated circuits (ASICs), network processors (NPs), field-programmable gate arrays (FPGAs), or other programmable logic devices, gate circuits, transistors, discrete hardware components, etc. The communication interface in this application embodiment may include transceivers, pins, circuits, buses, etc.

[0262] The aforementioned electronic devices can be terminals or other devices besides terminals, such as servers, network attached storage (NAS), etc.

[0263] The terminal can be a mobile phone, tablet computer, laptop computer, notebook computer, personal digital assistant (PDA), handheld computer, netbook, ultra-mobile personal computer (UMPC), mobile internet device (MID), augmented reality (AR), virtual reality (VR) device, mixed reality (MR) device, robot, wearable device, flight vehicle, vehicle user equipment (VUE), shipboard equipment, pedestrian user equipment (PUE), smart home (home devices with wireless communication capabilities, such as refrigerators, televisions, washing machines, or furniture), game console, personal computer (PC), ATM or self-service machine, etc. Wearable devices include: smartwatches, smart bracelets, smart earphones, smart glasses, smart jewelry (smart bracelets, smart chains, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc. Among these, in-vehicle devices can also be referred to as in-vehicle terminals, in-vehicle controllers, in-vehicle modules, in-vehicle components, in-vehicle chips, or in-vehicle units, etc. It should be noted that the embodiments in this application do not limit the specific type of terminal.

[0264] A server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server. A cloud server can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), or cloud computing services based on big data and artificial intelligence platforms.

[0265] Taking an electronic device as an example, Figure 13 is a schematic diagram of the hardware structure of a terminal implementing an embodiment of this application.

[0266] The terminal 1300 includes, but is not limited to, at least some of the following components: radio frequency unit 1301, network module 1302, audio output unit 1303, input unit 1304, sensor 1305, display unit 1306, user input unit 1307, interface unit 1308, memory 1309, and processor 1310.

[0267] Those skilled in the art will understand that terminal 1300 may also include a power supply (such as a battery) for powering various components. The power supply can be logically connected to processor 1310 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The terminal structure shown in Figure 13 does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0268] It should be understood that, in this embodiment, the input unit 1304 may include a graphics processor 13041 and a microphone 13042. The graphics processor 13041 processes image data of still images or videos obtained by an image acquisition device (such as a camera) in video acquisition mode or image acquisition mode, or it may process the obtained point cloud data. The display unit 1306 may include a display panel 13061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 1307 includes at least one of a touch panel 13071 and other input devices 13072. The touch panel 13071 is also called a touch screen. The touch panel 13071 may include a touch detection device and a touch controller. Other input devices 13072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0269] In this embodiment, after receiving downlink data from the network-side device, the radio frequency unit 1301 can transmit it to the processor 1310 for processing; in addition, the radio frequency unit 1301 can send uplink data to the network-side device. Typically, the radio frequency unit 1301 includes, but is not limited to, antennas, amplifiers, transceivers, couplers, low-noise amplifiers, duplexers, etc.

[0270] The memory 1309 can be used to store software programs or instructions, as well as various data. The memory 1309 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1309 may include volatile memory or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1309 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0271] Processor 1310 may include one or more processing units; optionally, processor 1310 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1310.

[0272] In one embodiment:

[0273] The processor 1310 is configured to acquire first texture map information of a crack region, wherein the crack region is a crack region between sub-mesh of a three-dimensional mesh; and to encode the first texture map information to obtain a first bitstream.

[0274] Optionally, the processor 1310 is also used for:

[0275] Determine the geometric patch to fill the crack area;

[0276] Determine the texture information of the geometric patch, wherein the texture information includes at least one of texture coordinates and texture triangles;

[0277] The first texture map information is the texture map information corresponding to the texture information.

[0278] Optionally, determining the geometric patch for filling the crack region includes:

[0279] Obtain the reconstructed sub-mesh of the sub-mesh of the 3D mesh;

[0280] The crack region of the reconstructed submesh of the submesh of the three-dimensional mesh is filled with boundary to obtain a geometric patch for filling the crack region.

[0281] Optionally, determining the texture information of the geometric patch includes:

[0282] Based on the characteristic relationship between geometric patches and texture information, the texture information of the geometric patches is predicted.

[0283] Optionally, the texture triangle corresponding to the first texture map information does not overlap with the original texture triangle of the sub-mesh of the three-dimensional mesh.

[0284] Optionally, the texture triangle corresponding to the first texture map information is determined in the following way:

[0285] Determine the texture triangle of the crack area;

[0286] When the texture triangle in the crack region overlaps with the original texture triangle of the sub-mesh of the 3D mesh, the texture triangle in the crack region is shrunk to obtain a texture triangle corresponding to the first texture map information that does not overlap with the original texture triangle.

[0287] Optionally, encoding the first texture map information to obtain the first bitstream includes:

[0288] The first texture map information and the second texture map information are encoded to obtain a first bitstream, wherein the second texture map information is the texture map information of the sub-mesh of the three-dimensional mesh.

[0289] Optionally, the processor 1310 is also used for:

[0290] The instruction information is encoded to obtain a second bitstream;

[0291] or,

[0292] The first bitstream includes: the encoding result of encoding the indication information;

[0293] The indication information is used to indicate whether the three-dimensional mesh contains the first texture map information.

[0294] In one embodiment:

[0295] Radio frequency unit 1301 is used to acquire the first code stream;

[0296] The processor 1310 is used to decode the first bitstream to obtain first texture map information of the crack region, wherein the crack region is a crack region between sub-mesh of a three-dimensional mesh.

[0297] Optionally, the processor 1310 is also used for:

[0298] Determine the geometric patch to fill the crack area;

[0299] Determine the texture information of the geometric patch, wherein the texture information includes at least one of texture coordinates and texture triangles;

[0300] The crack region is texture-filled based on the texture information and the first texture map information.

[0301] Optionally, determining the geometric patch for filling the crack region includes:

[0302] Obtain the reconstructed sub-mesh of the sub-mesh of the 3D mesh;

[0303] The crack region of the reconstructed submesh of the submesh of the three-dimensional mesh is filled with boundary to obtain a geometric patch for filling the crack region.

[0304] Optionally, determining the texture information of the geometric patch includes:

[0305] Based on the characteristic relationship between geometric patches and texture information, the texture information of the geometric patches is predicted.

[0306] Optionally, the texture triangle corresponding to the first texture map information does not overlap with the original texture triangle of the sub-mesh of the three-dimensional mesh.

[0307] Optionally, the texture triangle corresponding to the first texture map information is determined in the following way:

[0308] Determine the texture triangle of the crack area;

[0309] When the texture triangle in the crack region overlaps with the original texture triangle of the sub-mesh of the 3D mesh, the texture triangle in the crack region is shrunk to obtain a texture triangle corresponding to the first texture map information that does not overlap with the original texture triangle.

[0310] Optionally, decoding the first bitstream to obtain the first texture map information of the crack region includes:

[0311] The first bitstream is decoded to obtain the first texture map information and the second texture map information of the crack region, wherein the second texture map information is the texture map information of the sub-mesh of the three-dimensional mesh.

[0312] Optionally, the processor 1310 is also used for:

[0313] Acquire the second bitstream and decode it to obtain the indication information;

[0314] or,

[0315] The information obtained by decoding the first bitstream also includes indication information;

[0316] The indication information is used to indicate whether the three-dimensional mesh contains the first texture map information.

[0317] The aforementioned equipment can improve the encoding performance of 3D meshes.

[0318] It is understood that the implementation process of each implementation method mentioned in this embodiment can refer to the relevant description of the three-dimensional mesh encoding method or the three-dimensional mesh decoding method embodiment, and achieve the same or corresponding technical effects. To avoid repetition, it will not be described again here.

[0319] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described three-dimensional mesh encoding method or three-dimensional mesh decoding method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0320] The processor mentioned above is the processor in the terminal described in the above embodiments. The readable storage medium includes computer-readable storage media, such as ROM, RAM, magnetic disk, or optical disk. In some examples, the readable storage medium may be a non-transient readable storage medium.

[0321] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described three-dimensional mesh encoding method or three-dimensional mesh decoding method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0322] It should be understood that the chips mentioned in the embodiments of this application may include system-on-a-chip (also known as system chip, chip system, or system-on-a-chip) or discrete display chips, etc.

[0323] This application also provides a computer program / program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described three-dimensional mesh encoding method or three-dimensional mesh decoding method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0324] This application also provides an encoding / decoding system, including: an encoding end device and a decoding end device, wherein the encoding end device can be used to perform steps such as a three-dimensional mesh encoding method, and the decoding end device can be used to perform steps such as a three-dimensional mesh decoding method.

[0325] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0326] From the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of computer software products plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes several instructions to cause the terminal or network-side device to execute the methods described in the various embodiments of this application.

[0327] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other implementations under the guidance of this application without departing from the spirit and scope of the claims. All of these implementations are within the protection scope of this application.

Claims

1. A three-dimensional mesh encoding method, comprising: Obtain the first texture map information of the crack region, wherein the crack region is the crack region between sub-mesh of the three-dimensional mesh; The first texture map information is encoded to obtain the first bitstream.

2. The method as described in claim 1, wherein, The method further includes: Determine the geometric patch to fill the crack area; Determine the texture information of the geometric patch, wherein the texture information includes at least one of texture coordinates and texture triangles; The first texture map information is the texture map information corresponding to the texture information.

3. The method as described in claim 2, wherein, The determination of the geometric patch for filling the crack region includes: Obtain the reconstructed sub-mesh of the sub-mesh of the 3D mesh; The crack region of the reconstructed submesh of the submesh of the three-dimensional mesh is filled with boundary to obtain a geometric patch for filling the crack region.

4. The method as described in claim 2 or 3, wherein, The determination of the texture information of the geometric patch includes: Based on the characteristic relationship between geometric patches and texture information, the texture information of the geometric patches is predicted.

5. The method according to any one of claims 1 to 4, wherein, The texture triangle corresponding to the first texture map information does not overlap with the original texture triangle of the sub-mesh of the three-dimensional mesh.

6. The method of claim 5, wherein, The texture triangle corresponding to the first texture map information is determined in the following way: Determine the texture triangle of the crack area; When the texture triangle in the crack region overlaps with the original texture triangle of the sub-mesh of the 3D mesh, the texture triangle in the crack region is shrunk to obtain a texture triangle corresponding to the first texture map information that does not overlap with the original texture triangle.

7. The method according to any one of claims 1 to 6, wherein, The encoding of the first texture map information to obtain the first bitstream includes: The first texture map information and the second texture map information are encoded to obtain a first bitstream, wherein the second texture map information is the texture map information of the sub-mesh of the three-dimensional mesh.

8. The method according to any one of claims 1 to 7, wherein, The method further includes: The instruction information is encoded to obtain a second bitstream; or, The first bitstream includes: the encoding result of encoding the indication information; The indication information is used to indicate whether the three-dimensional mesh contains the first texture map information.

9. A three-dimensional mesh decoding method, comprising: Obtain the first bitstream; The first bitstream is decoded to obtain the first texture map information of the crack region, wherein the crack region is the crack region between sub-mesh of the three-dimensional mesh.

10. The method of claim 9, wherein, The method further includes: Determine the geometric patch to fill the crack area; Determine the texture information of the geometric patch, wherein the texture information includes at least one of texture coordinates and texture triangles; The crack region is texture-filled based on the texture information and the first texture map information.

11. The method of claim 10, wherein, The determination of the geometric patch for filling the crack region includes: Obtain the reconstructed sub-mesh of the sub-mesh of the 3D mesh; The crack region of the reconstructed submesh of the submesh of the three-dimensional mesh is filled with boundary to obtain a geometric patch for filling the crack region.

12. The method of claim 10 or 11, wherein, The determination of the texture information of the geometric patch includes: Based on the characteristic relationship between geometric patches and texture information, the texture information of the geometric patches is predicted.

13. The method according to any one of claims 9 to 12, wherein, The texture triangle corresponding to the first texture map information does not overlap with the original texture triangle of the sub-mesh of the three-dimensional mesh.

14. The method of claim 13, wherein, The texture triangle corresponding to the first texture map information is determined in the following way: Determine the texture triangle of the crack area; When the texture triangle in the crack region overlaps with the original texture triangle of the sub-mesh of the 3D mesh, the texture triangle in the crack region is shrunk to obtain a texture triangle corresponding to the first texture map information that does not overlap with the original texture triangle.

15. The method according to any one of claims 9 to 14, wherein, Decoding the first bitstream to obtain the first texture map information of the crack region includes: The first bitstream is decoded to obtain the first texture map information and the second texture map information of the crack region, wherein the second texture map information is the texture map information of the sub-mesh of the three-dimensional mesh.

16. The method according to any one of claims 9 to 15, wherein, The method further includes: Acquire the second bitstream and decode it to obtain the indication information; or, The information obtained by decoding the first bitstream also includes indication information; The indication information is used to indicate whether the three-dimensional mesh contains the first texture map information.

17. A three-dimensional mesh encoding device, comprising: The acquisition module is used to acquire the first texture map information of the crack region, wherein the crack region is the crack region between sub-mesh of the three-dimensional mesh; The first encoding module is used to encode the first texture map information to obtain the first bitstream.

18. The apparatus of claim 17, wherein, The device further includes: The first determining module is used to determine the geometric patch to fill the crack region; The second determining module is used to determine the texture information of the geometric patch, wherein the texture information includes at least one of texture coordinates and texture triangles; The first texture map information is the texture map information corresponding to the texture information.

19. A three-dimensional mesh decoding device, comprising: The first acquisition module is used to acquire the first bitstream; The first decoding module is used to decode the first bitstream to obtain the first texture map information of the crack region, wherein the crack region is the crack region between sub-mesh of the three-dimensional mesh.

20. The apparatus of claim 19, wherein, The device further includes: The first determining module is used to determine the geometric patch to fill the crack region; The second determining module is used to determine the texture information of the geometric patch, wherein the texture information includes at least one of texture coordinates and texture triangles; A filling module is used to fill the crack region with texture based on the texture information and the first texture map information.

21. An electronic device comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the three-dimensional mesh encoding method as claimed in any one of claims 1 to 8, or implementing the steps of the three-dimensional mesh decoding method as claimed in any one of claims 9 to 16.

22. A readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the three-dimensional mesh encoding method as claimed in any one of claims 1 to 8, or the steps of the three-dimensional mesh decoding method as claimed in any one of claims 9 to 16.

23. A chip comprising a processor and a communication interface coupled to the processor, the processor being configured to run a program or instructions to implement the steps of the three-dimensional mesh encoding method as claimed in any one of claims 1 to 8, or to implement the steps of the three-dimensional mesh decoding method as claimed in any one of claims 9 to 16.

24. A computer program product stored in a storage medium, the computer program product being executed by at least one processor to implement the steps of the three-dimensional mesh encoding method as claimed in any one of claims 1 to 8, or to implement the steps of the three-dimensional mesh decoding method as claimed in any one of claims 9 to 16.

Images