Mesh decoding method and apparatus, mesh encoding method and apparatus, and device

Abstract

The present application relates to the technical field of encoding and decoding, and discloses a mesh decoding method and apparatus, a mesh encoding method and apparatus, and a device. The mesh decoding method in the embodiments of the present application comprises: a decoding end acquires a first reference point of a first reconstructed sub-mesh; and the decoding end determines first crack information of the first reconstructed sub-mesh on the basis of the first reference point, wherein the first crack information comprises a boundary edge or a boundary point of the first reconstructed sub-mesh, and the first crack information is used for determining a first crack region between the first reconstructed sub-mesh and a second reconstructed sub-mesh.

Description

Grid decoding method, grid encoding method, device and equipment

[0001] Cross-reference to related applications

[0002] This application claims priority from the Chinese patent application No. 202410894938.6 filed on July 4, 2024 and entitled "Grid decoding method, grid encoding method, device and equipment", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present application belongs to the field of coding and decoding technology, and particularly relates to a grid decoding method, a grid encoding method, a device and equipment. BACKGROUND

[0004] In the field of three-dimensional grid coding and decoding, a three-dimensional grid is divided into multiple sub-grids, each of which is independently encoded and decoded. After decoding, the multiple reconstructed sub-grids are merged to obtain a merged grid, and the merged grid is processed to obtain a reconstructed grid.

[0005] However, in the lossy mode, the encoding end may simplify and quantize the sub-grids when independently encoding the sub-grids. The simplification of the sub-grids may cause the loss of boundary points of the sub-grids, and the quantization may cause the displacement of the boundary points. The loss or displacement of the boundary points of the sub-grids may cause the occurrence of crack regions in the merged grid, thereby failing to recover a decoded grid with high quality.

[0006] In related technologies, for each boundary point of each sub-grid, the encoding end can transmit a corresponding distance threshold to the decoding end, so that the decoding end can identify the matching boundary point of each boundary point. Then, the decoding end can identify the crack region based on the matching boundary point of each boundary point, and improve the quality of the reconstructed grid by post-processing the crack region. Specifically, the decoding end traverses the boundary points of the current sub-grid. If the distance between the current boundary point and a boundary point of another sub-grid is less than the distance threshold, it is considered that the two boundary points belong to the boundary points used to form the crack region.

[0007] However, transmitting a distance threshold for each boundary point of each sub-grid may result in a large amount of transmitted data, thereby reducing the grid decoding efficiency and the real-time performance of grid decoding. SUMMARY

[0008] The embodiments of the present application provide a grid decoding method, a grid encoding method, a device and equipment, which can improve the grid decoding efficiency and the real-time performance of grid decoding.

[0009] In a first aspect, a mesh decoding method is provided, which is performed by a decoding end and includes:

[0010] The decoding end acquires a first reference point of a first reconstructed sub-mesh;

[0011] The decoding end determines first crack information of the first reconstructed sub-mesh based on the first reference point;

[0012] The first crack information includes a boundary edge or a boundary point of the first reconstructed sub-mesh, and the first crack information is used to determine a first crack region between the first reconstructed sub-mesh and a second reconstructed sub-mesh.

[0013] In a second aspect, a mesh encoding method is provided, which is performed by an encoding end and includes:

[0014] The encoding end acquires at least one boundary point of a first reconstructed sub-mesh, which is generated based on sub-mesh division;

[0015] The encoding end determines a first reference point of the first reconstructed sub-mesh based on the at least one boundary point;

[0016] The first reference point is used to determine first crack information of the first reconstructed sub-mesh, the first crack information includes a boundary edge or a boundary point of the first reconstructed sub-mesh, and the first crack information is used to determine a first crack region between the first reconstructed sub-mesh and a second reconstructed sub-mesh.

[0017] In a third aspect, a mesh decoding device is provided, which includes:

[0018] An acquisition module is configured to acquire a first reference point of a first reconstructed sub-mesh;

[0019] A processing module is configured to determine first crack information of the first reconstructed sub-mesh based on the first reference point;

[0020] The first crack information includes a boundary edge or a boundary point of the first reconstructed sub-mesh, and the first crack information is used to determine a first crack region between the first reconstructed sub-mesh and a second reconstructed sub-mesh.

[0021] In a fourth aspect, a mesh encoding device is provided, which includes:

[0022] An acquisition module is configured to acquire at least one boundary point of a first reconstructed sub-mesh, which is generated based on sub-mesh division;

[0023] A processing module is configured to determine a first reference point of the first reconstructed sub-mesh based on the at least one boundary point;

[0024] The first reference point is used to determine first crack information of the first reconstructed sub-grid, the first crack information includes a boundary edge or a boundary point of the first reconstructed sub-grid, and the first crack information is used to determine a first crack region between the first reconstructed sub-grid and a second reconstructed sub-grid.

[0025] In a fifth aspect, a coding device is provided, which is configured to perform the steps of the method according to the first aspect, or implement the steps of the method according to the second aspect.

[0026] In a sixth aspect, an electronic device is provided, which includes a processor and a memory, the memory stores a program or instructions executable on the processor, and the program or instructions, when executed by the processor, implement the steps of the method according to the first aspect, or implement the steps of the method according to the second aspect.

[0027] In a seventh aspect, an electronic device is provided, which includes a processor and a communication interface, wherein the communication interface is configured to obtain a first reference point of a first reconstructed sub-grid, and the processor is configured to determine first crack information of the first reconstructed sub-grid based on the first reference point.

[0028] The first crack information includes a boundary edge or a boundary point of the first reconstructed sub-grid, and the first crack information is used to determine a first crack region between the first reconstructed sub-grid and a second reconstructed sub-grid.

[0029] In an eighth aspect, an electronic device is provided, which includes a processor and a communication interface, wherein the communication interface is configured to obtain at least one boundary point of a first reconstructed sub-grid generated based on sub-grid division, and the processor is configured to determine a first reference point of the first reconstructed sub-grid based on the at least one boundary point.

[0030] The first reference point is used to determine first crack information of the first reconstructed sub-grid, the first crack information includes a boundary edge or a boundary point of the first reconstructed sub-grid, and the first crack information is used to determine a first crack region between the first reconstructed sub-grid and a second reconstructed sub-grid.

[0031] In a ninth aspect, an electronic device is provided, which includes a memory configured to store video data, and a processing circuit configured to implement the steps of the method according to the first aspect, or implement the steps of the method according to the second aspect.

[0032] In a tenth aspect, a readable storage medium is provided, which stores a program or instructions, and the program or instructions, when executed by a processor, implement the steps of the method according to the first aspect, or implement the steps of the method according to the second aspect.

[0033] In an eleventh aspect, a codec system is provided, comprising: an encoding device and a decoding device, the encoding device being configured to perform the steps of the method of the first aspect, and the decoding device being configured to perform the steps of the method of the second aspect.

[0034] In a twelfth aspect, a chip is provided, comprising a processor and a communication interface, the communication interface being coupled to the processor, and the processor being configured to run a program or an instruction to implement the steps of the method of the first aspect or the steps of the method of the second aspect.

[0035] In a thirteenth aspect, a computer program / program product is provided, stored in a storage medium, and executed by at least one processor to implement the steps of the method of the first aspect or the steps of the method of the second aspect.

[0036] In the embodiments of the present application, the decoding device obtains a first reference point of a first reconstructed sub-mesh; and determines, based on the first reference point, first crack information of the first reconstructed sub-mesh, wherein the first crack information comprises a boundary edge or a boundary point of the first reconstructed sub-mesh, and the first crack information is used to determine a first crack region between the first reconstructed sub-mesh and a second reconstructed sub-mesh. That is, the first reference point can assist the decoding device to determine the boundary edge or the boundary point of the first reconstructed sub-mesh for forming the first crack region, thereby avoiding the transmission of information for determining the first crack region for each boundary point of the first reconstructed sub-mesh, so as to reduce the amount of data decoded by the decoding device, and further improve the mesh decoding efficiency and the real-time performance of mesh decoding. BRIEF DESCRIPTION OF DRAWINGS

[0037] FIG. 1 is a schematic diagram of a codec system provided by the embodiments of the present application.

[0038] FIG. 2a is an encoding flowchart of V3C provided by the embodiments of the present application.

[0039] FIG. 2b is a decoding flowchart of V3C provided by the embodiments of the present application.

[0040] FIG. 3 is an encoding flowchart of VDMC provided by the embodiments of the present application.

[0041] FIG. 4 is an example of the principle of mesh simplification provided by the embodiments of the present application.

[0042] FIG. 5 is an example of the basic idea of the subdivision and deformation module provided by the embodiments of the present application.

[0043] FIG. 6 is an example of the subdivision principle provided by the embodiments of the present application.

[0044] FIG. 7 is a schematic flowchart of a compression process of a base mesh according to an embodiment of the present application.

[0045] FIG. 8 is an example of a pattern of a triangular face according to an embodiment of the present application.

[0046] FIG. 9 is a schematic flowchart of a texture map conversion according to an embodiment of the present application.

[0047] FIG. 10 is a decoding flowchart of a VDMC according to an embodiment of the present application.

[0048] FIG. 11 is a schematic flowchart of a mesh decoding method according to an embodiment of the present application.

[0049] FIG. 12 is an example of a sub-mesh partition according to an embodiment of the present application.

[0050] FIG. 13 is an example of a sub-mesh independent encoding according to an embodiment of the present application.

[0051] FIG. 14 is an example of a merged mesh according to an embodiment of the present application.

[0052] FIG. 15 is an example of a process of obtaining crack indication information at a decoding end according to an embodiment of the present application.

[0053] FIG. 16 is an example of a decoding framework of a reconstructed sub-mesh according to an embodiment of the present application.

[0054] FIG. 17 is a schematic flowchart of a mesh encoding method according to an embodiment of the present application.

[0055] FIG. 18 is an example of a process of encoding crack indication information at an encoding end according to an embodiment of the present application.

[0056] FIG. 19 is an example of an encoding framework of a reconstructed sub-mesh according to an embodiment of the present application.

[0057] FIG. 20 is a schematic block diagram of a mesh decoding apparatus according to an embodiment of the present application.

[0058] FIG. 21 is a schematic block diagram of a mesh encoding apparatus according to an embodiment of the present application.

[0059] FIG. 22 is a schematic block diagram of an electronic device according to an embodiment of the present application.

[0060] FIG. 23 is a schematic diagram of a hardware structure of a terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0061] With reference to the drawings and the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly described. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art are within the scope of protection of the present application.

[0062] The terms "first", "second", and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second" are generally a class, not limited to the number of objects, for example, the first object can be one or more. In addition, "or" in the present application means at least one of the connected objects. For example, the protection scope of "A or B" at least covers three schemes, namely, scheme one: including A and not including B; scheme two: including B and not including A; scheme three: including A and B. In addition, the terms "A and / or B", "at least one of A and B", "at least one of A or B" also at least cover the above three schemes, respectively. The character " / " generally represents that the objects before and after are in a "or" relationship.

[0063] Before introducing the technical solutions provided by the embodiments of the present application, the meanings of some terms therein will be introduced first.

[0064] Three-dimensional mesh (3D Mesh): A three-dimensional mesh is a three-dimensional object surface composed of countless polygons in space, and a polygon is composed of vertices and edges.

[0065] Dynamic mesh: refers to a mesh whose shape can change over time, and such changes can be caused by animation, simulation or other dynamic processes.

[0066] In recent years, with the rapid development of multimedia technology, related research results have been rapidly industrialized and have become an important part of people's lives. Three-dimensional models have become a new generation of digital media after audio, images, and videos. Three-dimensional mesh is a common three-dimensional model representation method. Compared with traditional images, videos and other multimedia, three-dimensional mesh models have stronger interactivity and realism, making them more and more widely used in various fields such as business, manufacturing, construction, education, medicine, entertainment, art, and military. With the progress of three-dimensional mesh modeling and scanning technology, people have higher requirements for the precision and details of three-dimensional graphics, which has led to a doubling of the amount of original three-dimensional mesh data, which has posed a major challenge to the transmission, storage and computer processing of three-dimensional mesh. Therefore, it is particularly important to achieve efficient compression of three-dimensional mesh while ensuring quality.

[0067] Although there are many kinds of representation methods for three-dimensional meshes, triangle meshes are still the most common representation method. A three-dimensional mesh can be regarded as consisting of three basic elements: vertices, edges, and faces. Vertices are the most basic elements in a mesh, which define a position in a three-dimensional space. Edges are line segments connecting two vertices in a mesh. Faces can be regarded as polygons formed by a closed path of edges. For a triangle mesh, each face is a triangle.

[0068] A three-dimensional mesh is composed of multiple elements at different levels, including vertices, edges, and faces. Vertices are the basic elements that make up a three-dimensional mesh, described by coordinates in a three-dimensional space. Edges are parts that connect two vertices in a three-dimensional mesh. Faces are polygons connected by closed edges. Most faces in current three-dimensional meshes are triangle faces. A three-dimensional mesh mainly contains the following three kinds of information:

[0069] 1. Geometric information.

[0070] Geometric information mainly includes the position coordinates of all vertices in a three-dimensional mesh in a three-dimensional space.

[0071] 2. Connection relationship.

[0072] Connection relationship, also known as topological information, is used to describe the association between elements in a three-dimensional mesh, such as the connection relationship between vertices and the correspondence between vertices and faces.

[0073] 3. Other optional attribute information.

[0074] Attribute information includes other information attached to a three-dimensional mesh, including color information, normal vectors, texture coordinates, etc. Attribute information is optional and can be associated with corresponding mesh elements (such as vertex color, normal vector, etc. which can be associated with mesh vertices). Mesh parameterization can also be used to map a mesh from a three-dimensional space to a two-dimensional plane area. This mapping relationship is usually described by a set of parameter coordinates, called UV coordinates or texture coordinates, which are associated with the geometry vertices of the mesh. This two-dimensional mapping can be used to represent high-resolution attribute information, such as texture, normal vector, etc.

[0075] When a three-dimensional mesh has texture coordinates, a texture map needs to be rendered to the surface of the three-dimensional mesh using texture connection relationship. When the texture connection relationship of a three-dimensional mesh is consistent with the geometric connection relationship, the texture connection relationship does not need to be transmitted separately; when the texture connection relationship of a three-dimensional mesh is inconsistent with the geometric connection relationship, the texture connection relationship needs to be encoded separately to ensure that the decoding end can correctly reconstruct the texture connection relationship, so that the texture map can be correctly rendered to the surface of the three-dimensional mesh.

[0076] FIG. 1 is a schematic diagram of a coding system 10 according to an embodiment of the present application. The technical solutions of the embodiments of the present application relate to coding (CODEC) of grid data, including encoding or decoding.

[0077] As shown in FIG. 1, the coding system 10 includes a source device 100 that provides encoded grid data to be decoded and displayed by a destination device 110. Specifically, the source device 100 provides the grid data to the destination device 110 via a communication medium 120. The source device 100 and the destination device 110 can include any one or more of a desktop computer, a notebook (i.e., laptop) computer, a tablet computer, a set-top box, a mobile phone, a wearable device (e.g., a smart watch or a wearable camera), a television, a camera, a display device, a vehicle-mounted device, a virtual reality (VR) device, an augmented reality (AR) device, a mixed reality (MR) device, a digital media player, a video gaming console, a video conferencing device, a video streaming device, a broadcast receiver device, a broadcast transmitter device, a spacecraft, an airplane, a robot, a satellite, and the like.

[0078] In the example of FIG. 1, the source device 100 includes a data source 101, a memory 102, an encoder 200, and an output interface 104. The destination device 110 includes an input interface 111, a decoder 300, a memory 113, and a display device 114. The source device 100 represents an example of an encoding device, while the destination device 110 represents an example of a decoding device. In other examples, the source device 100 and the destination device 110 can not include some of the components in FIG. 1, or can include other components not shown in FIG. 1. For example, the source device 100 can acquire the grid data through an external capturing device. Also, the destination device 110 can interface with an external display device, rather than include an integrated display device. For another example, the memory 102, the memory 113 can be external memories.

[0079] Although FIG. 1 depicts the source device 100 and the destination device 110 as separate devices, in some examples, the source device 100 and the destination device 110 can be integrated in one device. In such embodiments, the corresponding functions of the source device 100 and the destination device 110 can be implemented using the same hardware or software, or using separate hardware or software, or any combination thereof.

[0080] In some examples, the source device 100 and the destination device 110 can perform unidirectional or bidirectional transmission of data. If bidirectional transmission of data is performed, the source device 100 and the destination device 110 can operate in a substantially symmetrical manner. That is, each of the source device 100 and the destination device 110 can include an encoder and a decoder.

[0081] Data source 101 represents a source of mesh data (i.e., raw, uncoded mesh data) and provides the encoder 200 with mesh data containing, which the encoder 103 encodes. Source device 100 can include a capture device (e.g., a camera device, a sensor device, or a scanning device), an archive including previously captured mesh data, or a feed interface for receiving mesh data from a data content provider. Among others, the camera device can include a normal camera, a stereo camera, and a light field camera, etc., the sensor device can include a laser device, a radar device, etc., and the scanning device can include a three-dimensional laser scanning device, etc. The mesh data can be obtained by capturing a visual scene of a real world through the capture device. Alternatively, the data source 101 can generate computer graphics based data as source data, or combine real-time data, archived data, and computer generated data. For example, the data source generates mesh data based on a virtual object (e.g., a virtual three-dimensional object and a virtual three-dimensional scene obtained by three-dimensional modeling).

[0082] Memory 102 of source device 100 and memory 113 of destination device 110 represent general purpose memories. In some examples, memory 102 can store raw data from data source 101, and memory 113 can store decoded mesh data from decoder 300. Additionally or alternatively, memory 102, 113 can store software instructions executable by, for example, encoder 200 and decoder 300, respectively. Although memory 102 and memory 113 are shown separately from encoder 200 and decoder 300 in this example, it should be understood that encoder 200 and decoder 300 can also include internal memories for functionally similar or equivalent purposes. If encoder 200 and decoder 300 are deployed on the same hardware device, memory 102 and memory 113 can be the same memory. Furthermore, memory 102, 113 can store, for example, encoded mesh data output from encoder 200 and input to decoder 300. In some examples, portions of memory 102, 113 can be allocated as one or more mesh buffers, e.g., for storing raw, decoded, or encoded mesh data.

[0083] In some examples, source device 100 can output encoded data from output interface 104 to storage 113. Similarly, destination device 110 can access encoded data from storage 113 via input interface 111. Storage 113 or storage 102 can include any of a variety of distributed or locally accessed data storage media such as a hard drive, Blu-ray discs, Digital Versatile Discs (DVDs), Compact Disc Read-Only Memory (CD-ROMs), flash drives, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded grid data.

[0084] Output interface 104 can include any type of medium or device capable of sending encoded grid data from source device 100 to destination device 110. For example, output interface 104 can include a transmitter or a transceiver, such as an antenna, configured to transmit encoded grid data from source device 100 directly to destination device 110 in real-time. The encoded grid data can be modulated according to a communication standard of a wireless communication protocol and transmitted to destination device 110.

[0085] Communication medium 120 can include transient media, such as wireless broadcasts or wired networks transmitted over physical transmission lines. For example, communication medium 120 can include radio frequency (RF) spectrum or one or more physical transmission lines, such as coaxial cable, copper wires, or fiber optics. Communication medium 120 can form part of a packet-based network, such as a local area network, a wide-area network, or a global network such as the Internet. Communication medium 120 can also be in a form of a storage medium, such as a non-transitory storage medium, such as a hard disk, flash drive, compact disc, digital grid disc, Blu-ray disc, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded grid data.

[0086] In some embodiments, the communication medium 120 can include a router, switch, base station, or any other device by which communication from the source device 100 to the destination device 110 can be facilitated. For example, a server (not shown) can receive the encoded mesh data from the source device 100 and provide to the destination device 110, e.g., via a network transmission to the destination device 110. The server can include a web server (e.g., for a website), a server configured to provide a file transfer protocol service such as a File Transfer Protocol (FTP) or File Delivery Over Unidirectional Transport (FLUTE) protocol, a content delivery network (CDN) device, a Hypertext Transfer Protocol (HTTP) server, a Multimedia Broadcast Multicast Services (MBMS) or evolved Multimedia Broadcast Multicast Service (eMBMS) server, or a Network-attached storage (NAS) device, etc. The server can implement one or more HTTP streaming protocols such as an MPEG Media Transport (MMT) protocol, a Dynamic Adaptive Streaming over HTTP (DASH) protocol, an HTTP Live Streaming (HLS) protocol, or a Real Time Streaming Protocol (RTSP), etc.

[0087] The destination device 110 can access the encoded mesh data from the server, e.g., through a wireless channel (e.g., a Wi-Fi connection) or a wired connection (e.g., a Digital subscriber line (DSL), a cable modem, etc.) for accessing the encoded mesh data stored on the server.

[0088] The output interface 104 and the input interface 111 can represent wireless transmitters / receivers, modems, wired networking components (e.g., Ethernet cards), wireless communication components that operate according to the IEEE 802.11 standards or the IEEE 802.15 standards (e.g., ZigBee™), the Bluetooth standard, etc., or other physical components. In examples where the output interface 104 and the input interface 111 comprise wireless components, the output interface 104 and the input interface 111 can be configured to transfer data, such as encoded mesh data, according to WIFI, Ethernet, cellular networks (such as 4G, LTE (Long-Term Evolution), LTE-Advanced, 5G, 6G, etc.), or the like.

[0089] The input interface 111 of the destination device 110 receives the encoded bitstream from the communication medium 120. The encoded bitstream can include high-level syntax elements and encoded data units (e.g., sequences, groups of pictures, pictures, slices, blocks, etc.), where the high-level syntax elements are used to decode the encoded data units to yield decoded mesh data. The display device 114 displays the decoded mesh data to a user. The display device 114 can include a cathode ray tube (CRT), a liquid-crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other type of display device. In some examples, the destination device 110 can not have the display device 114, e.g., if the decoded mesh data is used to determine a position of a physical object, the display device 114 can be replaced with a processor.

[0090] The encoder 200 and the decoder 300 can be implemented as one or more of various processing circuitry, which can include one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic circuitry, hardware, or any combinations thereof. When the techniques are implemented partially in software, a device can store instructions for the software in a suitable, non- transitory computer-readable storage medium and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure.

[0091] In almost all application fields using three-dimensional mesh, such as computational simulation, entertainment, medical imaging, digital cultural relics, computer design, e-commerce, etc., with the increasing demand for three-dimensional mesh model in visual effect, the model is becoming more and more complex, and the precision of the model is also higher and higher, so the amount of data required to represent the three-dimensional mesh is also increasing. The above problems lead to the increasing complexity of processing, visualization, transmission and storage of three-dimensional mesh. Three-dimensional mesh compression can be regarded as a way to solve the above problems, which reduces the size of the model data, and is beneficial to the processing, storage and transmission of three-dimensional mesh, so it is necessary to propose an efficient and general three-dimensional mesh compression algorithm.

[0092] Recently, the international standardization organization MPEG in the field of audio and video coding compression began to formulate a compression standard for three-dimensional mesh, namely Video-based dynamic mesh coding (VDMC), which is based on the existing Visual Volumetric Video-based Coding (V3C) standard. The V3C standard provides a general method for compressing three-dimensional models, which can be presented in the form of point cloud, mesh or panoramic video. The compression method of three-dimensional mesh model compatible with the standard helps the promotion and applicability of the method. Therefore, it is of great significance to optimize the three-dimensional mesh coding method in VDMC and combine the optimization method with the V3C standard. A possible optimization method is to optimize the basic mesh coding, which divides a basic mesh or a sub-mesh of a basic mesh into two kinds of slices (slices) for intra-frame and inter-frame coding.

[0093] The V3C standard provides a method for encoding and decoding various three-dimensional media through video or image coding technology. Specifically, it converts three-dimensional media content from three-dimensional representation to multiple two-dimensional representations (called V3C components) through projection and other methods before encoding, and then encodes the two-dimensional representations using existing video or image coding technology.

[0094] The example diagram of the V3C standard is shown in Figures 2a and 2b, and the V3C components mainly include occupancy components, geometry components and attribute components. The occupancy component can represent which areas in the two-dimensional representation are associated with the data of the three-dimensional representation; the geometry component represents information related to the position of the three-dimensional data in space, and the attribute component can provide attribute information corresponding to the vertex, such as material, texture, etc. In addition, the components also contain information on how to reconstruct the three-dimensional model through these components, which is called atlas information.

[0095] The atlas information is used to associate all components, and additional information for reconstructing from two dimensions to three dimensions is also contained in the atlas component. The atlas is composed of a plurality of basic units, which are referred to as patches. For each patch, it represents a block area in the available two-dimensional component, and contains information required for projecting the area back to the three-dimensional space.

[0096] VDMC is a standard for compressing three-dimensional meshes formulated by MPEG, and the main idea is to compress the three-dimensional meshes by using the existing V3C standard. Since the three-dimensional meshes have connection information to be encoded, the specific encoding process is slightly different from V3C, and the syntax semantics and decoding operation of the decoding end of the V3C standard need to be extended to support the decoding and reconstruction of the three-dimensional meshes.

[0097] The overall framework of the encoder and decoder provided by the embodiments of the present application is described below taking VDMC as an example.

[0098] The overall framework of the encoding end is shown in FIG. 3. For the input mesh, first, simplification is performed by a simplification module, then new texture coordinates are generated by mesh parameterization, then the parameterized mesh is subdivided and deformed, that is, new vertices are inserted on the mesh according to a specific subdivision method, and the distance from the subdivided mesh vertices to the nearest neighbor points of the input mesh is calculated, which is referred to as displacement information. Then, the vertex positions of the mesh before subdivision and deformation are adjusted according to the displacement information, and the adjusted mesh is referred to as the base mesh, which is sent to the base mesh encoding module for compression.

[0099] It should be noted that when the input mesh is encoded, the input mesh can be divided into a plurality of sub-meshes, and the encoding end can independently encode each sub-mesh, wherein each sub-mesh can have a base mesh, displacement information, or a texture map.

[0100] The base mesh is reconstructed after encoding, and then the order of the displacement is adjusted according to the vertex order of the reconstructed base mesh. Then, the vertex displacement information after adjusting the order is first wavelet transformed, the transformed coefficients are quantized, then the quantized coefficients are arranged into a two-dimensional image according to a specific scanning order, and the two-dimensional image is encoded by using a video encoder. Then, the reconstructed displacement information is applied to the subdivided base mesh to obtain the reconstructed subdivided and deformed mesh, and the mesh, the original input mesh, and the corresponding texture map are input into the corresponding texture map conversion module to obtain the texture map corresponding to the reconstructed mesh, and the texture map is also encoded by using the video encoder. The parameters used in the encoding process, such as the type of the video encoder, the type of the mesh encoder, the transformation parameters, the quantization parameters, etc. are transmitted to the decoding end through the auxiliary information.

[0101] The specific process of encoding is described below:

[0102] (1) Mesh simplification.

[0103] Mesh simplification is to simplify the current input mesh to a base mesh with relatively less number of faces and vertices, and to preserve the shape of the original mesh as much as possible. The focus of mesh simplification is the operation of simplification and the corresponding error metric. A feasible mesh simplification operation is shown in Fig. 4, which merges the two vertices at the ends of an edge into one vertex and deletes the connection between the two vertices. Repeating this process in the whole mesh according to certain rules will reduce the number of faces and vertices to the target value.

[0104] In the process of simplification, an error metric can be selected to optimize the result of simplification. For example, the sum of the equation coefficients of all adjacent faces of a vertex can be selected as the error metric of the vertex, and the error metric of the corresponding edge is the sum of the error metrics of the two vertices of the edge. In other words, the error generated by merging an edge is the sum of the distances from the merged vertex to all adjacent planes of the original two vertices of the edge.

[0105] After determining the simplification operation and the corresponding error metric, mesh simplification is iteratively performed. First, the vertex error of the initial mesh is calculated, thereby obtaining the error of each edge. Then, each edge is arranged in ascending order of error, and the edge with the smallest error is selected for merging each time. At the same time, the position of the merged vertex is calculated, and the error of all edges related to the merged vertex is updated. That is, the order of edge arrangement is updated to ensure that each iteration is based on the global error metric. Through iteration, the number of faces of the mesh is simplified to meet the requirements of lossy coding.

[0106] (2) Mesh parameterization.

[0107] The texture coordinates can be regenerated according to the reconstructed base mesh. This step needs to generate texture coordinates for each attribute map of the input mesh, and if multiple attribute maps have similar characteristics, the same texture coordinates can be shared. The way to generate texture coordinates includes mesh parameterization, and the algorithms used for mesh parameterization include but are not limited to Isochart algorithm, orthogonal projection algorithm, etc.

[0108] Two algorithms will be briefly introduced below.

[0109] Isochart algorithm uses spectral analysis to achieve stretch-driven three-dimensional mesh parameterization, which unfolds, slices and packs the three-dimensional mesh to a two-dimensional texture domain. Specifically, a stretch threshold can be set, and the algorithm is outlined as follows:

[0110] 1. Calculate the surface spectral analysis to provide an initial parameterization;

[0111] 2. Perform iterations of stretch optimization;

[0112] 3. If the stretch of this derived parameterization is less than a threshold, stop;

[0113] 4. Perform surface spectral clustering to divide the surface into charts;

[0114] 5. Optimize the chart boundaries using a graph cut algorithm;

[0115] 6. Iteratively split the charts until the stretch criterion is satisfied.

[0116] The orthogonal projection algorithm is a projection-based mesh parameterization method, which generates texture coordinates for a mesh by orthogonal projection. The main process includes:

[0117] 1. Compute mesh properties, including the adjacent faces of each face and the area and normal vector of each face;

[0118] 2. Determine the projection plane of each face according to the normal vector;

[0119] 3. Start clustering all faces into a connected region according to the projection plane. First, select the starting face of clustering;

[0120] 4. From the starting face, iterate to determine whether the adjacent face of the face joining the connected region can join the connected region;

[0121] 5. After each connected region iteration is completed, multiple connected regions are obtained;

[0122] 6. Determine whether to merge adjacent connected regions according to the error metric;

[0123] 7. Detect whether there is an overlapping region during projection. Remove the overlapping face and regenerate the connected region;

[0124] 8. Arrange all the projected regions into a two-dimensional image.

[0125] (3) Subdivision deformation.

[0126] The subdivision and deformation module is an optional module that can improve the quality of the reconstructed mesh at the decoding end. When the quality of the basic mesh can meet the application requirements, this module can not be used.

[0127] The basic idea of the subdivision and morphing module is shown in Fig. 5. The same concept is applied to the input 3D mesh to generate displacement vector information. In Fig. 5, the input 2D curve (represented by a 2D polyline), referred to as the "original" curve, is first down-sampled to generate a base curve / polyline, referred to as the "simplified" curve. Then a subdivision scheme is applied to the simplified polyline to generate a "subdivided" curve. Subsequently, the subdivided polyline is morphed to obtain a better approximation of the original curve. That is, a geometric displacement vector (shown by the arrow in Fig. 5) is calculated for each vertex of the subdivided mesh, such that the shape of the subdivided curve is as close as possible to the shape of the original curve. Adding the calculated geometric displacement vector to the coordinates of each vertex of the subdivided mesh is the morphed mesh containing displacement information output by the module. The same morphing process is also applied to the attribute information corresponding to the vertices, thereby obtaining the corresponding attribute displacement vectors.

[0128] The subdivision and morphing module subdivides the input mesh. The subdivision scheme can be arbitrarily selected. One possible scheme is the midpoint subdivision scheme, which subdivides each triangle into four sub-triangles in each iteration of subdivision, as shown in Fig. 6, S 0 is the mesh before subdivision, 1 is the mesh after one iteration of subdivision, and 2 is the mesh after two iterations of subdivision. A new vertex is introduced in the middle of each edge. The subdivision of the geometric information and the attribute information is performed independently, because the connection relationship between the geometric information and the attribute information is usually different.

[0129] The position Pos(v 12 ) of the midpoint v 12 of the newly introduced edge (v1, v2) is calculated as shown in equation (1):

[0130] where Pos(v1) and Pos(v2) are the geometric coordinates of vertices v1 and v2, respectively.

[0131] For the subdivided mesh, the nearest neighbor of each point on the original input mesh (including the points on the faces of the original mesh) is found. The search can be accelerated by using a data structure such as a k-dimensional tree (KD-tree). The displacement vector of the geometric coordinates of each vertex of the subdivided mesh is obtained by calculating the distance between each vertex of the subdivided mesh and its nearest neighbor on the original input mesh.

[0132] The subdivision and morphing module can also convert the coordinate system of the calculated vertex displacement (e.g., Cartesian coordinate system) into a local coordinate system. For example, the coordinates of each vertex displacement are converted into the coordinates in a coordinate system composed of the normal vector of the corresponding vertex and the two vectors tangent to the normal vector. The conversion process is shown in equation (2):

[0133] where, represents the vertex displacement before coordinate system conversion, represents the vertex displacement after coordinate system conversion, are unit vectors mutually orthogonal to each other, and is a vertex normal vector, are two vectors tangent to . For the calculation of the vertex normal vector, one feasible method is: equal to the area-weighted sum of the vertex adjacent face normal vectors.

[0134] (4) Compression of the base mesh.

[0135] The base mesh compression module compresses the base mesh information output by the subdivision deformation module. The base mesh compression mainly has two different modes, which are respectively: intra-frame mode and inter-frame mode. As shown in FIG. 7, in the intra-frame mode, the input of the base mesh compression module is a three-dimensional mesh, which includes geometric coordinates, connection relationship and attribute information associated with the vertex. In the inter-frame mode, the input of the base mesh compression module is a motion vector and its identifier and possibly existing intra-frame sub-mesh. After encoding, the encoded mesh needs to be reconstructed to provide for the subsequent modules for processing.

[0136] The encoding of the two modes is briefly introduced as follows:

[0137] Intra-frame mode: The mesh encoding in the intra-frame mode can use the existing mesh encoder to encode the input base mesh, such as Draco, etc. The type of mesh encoder is encoded and transmitted to the decoding end through auxiliary information. A usable Draco is introduced below.

[0138] The main process of Draco encoding mesh includes: first, for the input mesh, the connection relationship of the vertex in the three-dimensional space is generated according to the geometric information, that is, the connection relationship of the vertex in the three-dimensional space. After the connection relationship of the face is constructed, the initial face is selected to start traversing all the faces of the current mesh, that is, the traversal generation symbol is generated. Here, the traversal generation symbol uses the Edgebreaker algorithm.

[0139] The EdgeBreaker algorithm is one of the best performing algorithms for connection encoding. The EdgeBreaker algorithm has the advantage of effectively utilizing the topology of the mesh to reduce the amount of data required for connection encoding. By dividing the mesh into traversed and untraversed parts, and using the CLERS pattern to guide the traversal process, the EdgeBreaker can generate a compact encoding representation, which is very useful for the storage and transmission of three-dimensional meshes. The EdgeBreaker algorithm divides the three-dimensional mesh into traversed and untraversed parts, starting from an initial triangle, and constantly inserts the traversed triangle into the traversed part, and traverses other triangles according to the relative orientation of the newly added triangle. For example, according to the relative orientation of the vertex opposite the incident edge (e.g., the common edge with the previous triangle) of the current triangle and the left and right triangles (e.g., the triangles in contact with the other two edges), the CLERS pattern of the current triangle is determined, and the traversal direction of the next triangle is determined.

[0140] The CLERS pattern can include L mode, C mode, R mode, S mode, and E mode as shown in FIG. 8. V is the vertex opposite the incident edge (e.g., the common edge with the previous triangle) of the current triangle. The encoding end can assign the CLERS pattern of the current triangle according to the following five different cases:

[0141] The C mode indicates that v is untraversed and traverses to the right of the current triangle.

[0142] The L mode indicates that v is traversed and traverses to the right of the current triangle.

[0143] The R mode indicates that v is traversed and traverses to the left of the current triangle.

[0144] The S mode indicates that v is traversed and first traverses to the right branch and then traverses to the left branch after traversing to the bottom.

[0145] The E mode indicates that v is traversed and ends the traversal.

[0146] After the encoding end determines the CLERS pattern of the current triangle, the traversal direction of the next triangle is determined, and the next triangle is taken as the current triangle to determine its CLERS pattern. In this way, all triangles of the mesh are traversed, and the connection relationship of the three-dimensional mesh is represented as a string composed of CLERS, and the encoding ends.

[0147] It should be noted that the five modes also define the direction of traversing the next patch after the current patch (i.e. the arrow shown in the figure). According to the above traversal method, a corresponding symbol is generated for each patch defined by the current mesh geometry information, and then the symbols are entropy encoded to obtain the code stream of the connection relationship defined by the current mesh geometry information. At the same time, the traversal of each patch also obtains the order of traversing the corresponding vertices, which is passed to the geometry information encoder, which is rearranged in the order of traversal and quantized according to the predetermined quantization parameter, and then predicted. The prediction can use the parallelogram prediction method.

[0148] Inter-frame mode: In the inter-frame mode, the motion vector and the identifier need to be encoded. The motion vector identifier is an array composed of numbers greater than or equal to 0, and the array size is the same as the number of vertices of the base mesh of the reference frame corresponding to the time domain motion vector identifier. The array can be directly encoded using existing entropy encoding algorithms, such as CABAC.

[0149] (5) Reconstruct the base mesh.

[0150] After the base mesh is encoded, the base mesh needs to be reconstructed.

[0151] For the mesh encoded in the intra-frame mode, the reconstructed base mesh can be obtained by decoding the encoded mesh.

[0152] For the base mesh information encoded in the inter-frame mode, the inter-frame sub-mesh needs to be reconstructed according to the motion vector and the identifier. If there is an intra-frame sub-mesh in the inter-frame mode, the reconstructed inter-frame sub-mesh needs to be combined with the intra-frame sub-mesh to obtain the reconstructed base mesh.

[0153] (6) Subdivision.

[0154] After obtaining the reconstructed base mesh, it needs to be subdivided for subsequent vertex displacement encoding and reconstruction of the deformed mesh.

[0155] (7) Encoding of vertex displacement.

[0156] To encode the vertex displacement, first, the order of the optimized deformed mesh vertices needs to be adjusted according to the subdivided reconstructed base mesh. Then, the coordinates of the optimized deformed mesh vertices are subtracted from the coordinates of the subdivided reconstructed base mesh vertices to obtain displacement vectors. Next, wavelet transform and quantization are performed on the displacement to obtain quantized wavelet transform coefficients. Finally, the wavelet transform coefficients of the current frame are arranged into a two-dimensional image to form a YUV video, and the displacement code stream is obtained by encoding the video using a video encoder.

[0157] Next, each module will be introduced in detail:

[0158] 1. Adjust the order of the deformed mesh vertices and calculate the displacement.

[0159] The main function of this module is to adjust the order of the deformed mesh vertices after optimization according to the order of the subdivided base mesh vertices, so that the order of the deformed mesh vertices after optimization is the same as the order of the subdivided base mesh vertices, and then subtract the corresponding vertex coordinates of the two to calculate a displacement vector for each vertex. There is an optional step in this process, which is to convert the coordinate system of the vertex displacement (such as the Cartesian coordinate system) to the local coordinate system.

[0160] 2. Wavelet transform.

[0161] A transform can be applied to the displacement vector to reduce the correlation between its data. An optional transform is the linear wavelet transform, whose prediction process is defined as shown in equation (3):

[0162] where v is the midpoint newly inserted on the edge (v1, v2), Signal(v), Signal(v1) and Signal(v2) are the displacement vectors corresponding to vertices v, v1 and v2 respectively. The displacement vector of vertex v is predicted and then updated, and the update process is defined as shown in equation (4):

[0163] where v * is the set of all adjacent vertices of vertex v. The transformed displacement vector is called the wavelet coefficient.

[0164] 3. Quantization of coefficients.

[0165] The transformed displacement vector, i.e. the wavelet coefficient, can be quantized. There are many ways to quantize, one method is shown in equations (5) and (6):

[0166] where disp[v] represents the transformed value of the displacement vector of the vth vertex, d[k] represents the kth value of the displacement vector, floor represents rounding down. bitDepthPosition represents the bit depth of the current mesh vertex geometry position, and qp[k] represents the quantization parameter of the kth coefficient. As mentioned earlier, after converting the coordinate system of the displacement vector, the normal component has a more significant impact on quality than the tangential component, so a larger quantization parameter can be used for the tangential component.

[0167] Meanwhile, according to the characteristics of wavelet transform, different quantization parameters can be used for the newly generated vertices and the original vertices. That is, for the vertices after subdivision, the quantization parameter update is shown in equation (7): scale[k] = scale[k] * lodScale[k] (7)

[0168] wherein, lodScale[k] represents the coefficient of the quantization parameter of the current subdivision level.

[0169] 4, two-dimensional arrangement.

[0170] After quantization, the quantized wavelet transform coefficients can be arranged on a two-dimensional image in the following manner:

[0171] a. Traverse the wavelet coefficients in the order from low frequency to high frequency.

[0172] b. For each coefficient, determine the index of the NxM pixel block in which it is stored (for example, N = M = 16) according to the raster scan order of the block.

[0173] c. Calculate the position of the corresponding NxM pixel block on the image according to the Morton order.

[0174] Here, the arrangement is not limited to this manner, and other arrangement schemes such as zigzag order, raster order, etc. can also be used. The encoder can explicitly specify the corresponding arrangement scheme in the bitstream.

[0175] 5, video compression.

[0176] After arranging the wavelet coefficients on a two-dimensional image, a video encoder can be directly used for encoding, obtaining the bitstream of the displacement.

[0177] (8) Reconstruction of the morphed mesh

[0178] The displacement vector encoding module needs to obtain the reconstructed displacement vector value during encoding, that is, to obtain the displacement vector by inverse quantization and inverse transform (consistent with the decoding end). After obtaining the reconstructed geometric displacement vector, the subdivision mesh is obtained according to the corresponding geometric displacement vector to obtain the reconstructed subdivision morphed mesh, which is transmitted to the attribute map (for example, texture map) conversion module.

[0179] (9) Attribute map conversion.

[0180] The attribute map conversion module performs attribute map conversion according to the input mesh, the input attribute map, and the reconstructed morphed mesh.

[0181] As shown in FIG. 9, the steps of attribute map conversion are as follows:

[0182] 1. Calculate the texture coordinate of each pixel in the attribute map to be generated, such as the texture coordinate P(u, v) corresponding to pixel A(i, j).

[0183] 2. Determine whether the texture coordinate is in a certain triangular face after the subdivision deformation mesh parameterization.

[0184] 3. If the texture coordinate does not belong to any triangular face, mark the pixel as empty, and then fill it with a filling algorithm.

[0185] 4. If the texture coordinate belongs to a triangular face, then:

[0186] a. Mark the pixel as filled.

[0187] b. Calculate the barycentric coordinate of the pixel in the current triangular face according to the texture coordinate.

[0188] c. Map the two-dimensional texture coordinate to the three-dimensional geometric coordinate according to the barycentric coordinate and the corresponding triangular face, i.e. map it to the point on the subdivision deformation mesh corresponding to the texture coordinate, as shown by M(x, y, z) in the figure.

[0189] d. Find the nearest point to the three-dimensional coordinate on the input original mesh, as shown by M'(x, y, z) in the figure.

[0190] e. Calculate the barycentric coordinate of the three-dimensional coordinate according to the triangular face it belongs to, and map it to two dimensions to calculate its texture coordinate, i.e. P'(u', v').

[0191] f. Sample the input attribute map through the texture coordinate to obtain the value A'(i', j') of the corresponding pixel position.

[0192] Assign the value to the corresponding pixel A(i, j) in the attribute map to be generated.

[0193] (10) Attribute map compression.

[0194] After obtaining the converted attribute map, the existing filling algorithm (such as Push-Pull algorithm) can be used to fill the empty pixels in the attribute map. Then the existing video encoder, such as H.264 / AVC, H.265 / HEVC, H.266 / VVC, etc. can be used to encode it to obtain the code stream of the output attribute map, such as the texture map code stream. In addition, color space conversion and chroma subsampling operations can be selectively applied to make the video encoding obtain better rate-distortion performance, such as color space conversion from RGB 444 to YUV420. For the case where there are multiple texture maps, the corresponding texture coordinates of each texture map need to be identified through auxiliary information during encoding.

[0195] (11) auxiliary information encoding.

[0196] In the encoding process, there are some alternative schemes in each module, such as the type of mesh encoder, the type of video encoder, the mesh subdivision scheme, the spatial displacement vector transform scheme and the transform parameters, etc. The proposed framework allows the use of different schemes. Therefore, the selected schemes need to be passed to the decoding end to guide the correct decoding.

[0197] The auxiliary information includes the type of static mesh encoder, the type of video encoder, the mesh subdivision scheme, the number of mesh iterations, the geometric displacement vector transform scheme and the transform parameters, the coefficient arrangement scheme, and the color conversion scheme, etc. For the mesh part encoded using the inter-frame coding mode, the auxiliary information also includes the corresponding reference frame list, etc., which identifies the index list of the used reference frames. The auxiliary information can also include the subdivision identifier, which indicates whether the subdivision deformation operation is needed, i.e., whether it contains displacement information.

[0198] After the encoding of all modules is completed, the base mesh part code stream, the texture coordinate part code stream, the displacement vector video code stream, the texture map video code stream, and the auxiliary information code stream are mixed to obtain the encoding end, and the final output of the encoding code stream.

[0199] The overall framework of the decoding end is shown in FIG. 10. For the received code stream, the decoding end first demultiplexes each part of the code stream to obtain the base mesh code stream, the displacement video code stream, the texture map video code stream, and the auxiliary information code stream. For the base mesh code stream, the mesh decoder indicated by the auxiliary information is used to decode the base mesh. The displacement video code stream and the texture map video code stream can be decoded by the video decoder. For the displacement part, the displacement needs to be taken out from the image after video decoding through the displacement decoding module, and steps such as inverse quantization and inverse transform are performed, and then it is applied to the subdivided base mesh to obtain the reconstructed deformed mesh at the decoding end. The decoded texture map is the texture map corresponding to the reconstructed deformed mesh. The subsequent application or rendering module processes the reconstructed deformed mesh and the decoded texture map as input.

[0200] The specific process of decoding is described below:

[0201] (1) auxiliary information decoding.

[0202] The auxiliary information includes the type of static mesh encoder, the type of video encoder, the mesh subdivision scheme, the number of mesh iterations, the geometry displacement vector transform scheme, the coefficient arrangement scheme, the displacement truncation flag, and the color conversion scheme, etc. For the mesh part encoded using the inter-frame coding mode, the auxiliary information also includes the corresponding reference frame list, etc. The reference frame list identifies the index list of the used reference frames. The auxiliary information can also include the subdivision flag, which indicates whether the subdivision deformation operation is needed, i.e., whether the displacement information is contained. The auxiliary information can guide the correct decoding at the decoding end, and the auxiliary information is represented after being decoded through the corresponding syntax parameters.

[0203] (2) Decoding of the base mesh.

[0204] The decoding of the base mesh can be divided into the intra-frame mode and the inter-frame mode, etc. This module is only responsible for decoding the information representing the base mesh. That is, for the intra-frame mode, the output of the decoding of this module is the three-dimensional mesh, which contains the geometry information and the connection relationship, and possibly contains the attribute information and the texture coordinate information, etc. For the inter-frame mode, this module is responsible for decoding the motion vector and the identifier, and possibly the intra-frame sub-mesh, i.e., the output information is the motion vector, the identifier, and the intra-frame sub-mesh.

[0205] Intra-frame mode: the decoding of the base mesh information under the intra-frame mode is also the decoding of the base mesh. This module uses the corresponding mesh decoder according to the type of the static mesh decoder indicated by the auxiliary information to decode the base mesh.

[0206] Under the inter-frame mode, the motion vector, the identifier, and possibly the intra-frame sub-mesh need to be decoded. For the intra-frame sub-mesh, the mesh decoder indicated by the auxiliary information is used for decoding. The motion vector identifier is obtained through entropy decoding. After the entropy decoding of the motion vector, the base mesh is obtained through the inverse prediction and the inverse quantization.

[0207] (3) Subdivision.

[0208] The operation of this subdivision module is the same as the subdivision operation at the encoding end. The auxiliary information indicates the number of subdivision iterations of different regions of the current base mesh, etc.

[0209] (4) Displacement video decoding.

[0210] The displacement video code stream is decoded according to the type of the video decoder indicated in the auxiliary information to obtain the displacement information.

[0211] It is worth noting that the displacement video code stream can include the geometry displacement vector video code stream and possibly the displacement vector video code stream of different attributes. For example, the displacement vector video code stream of attributes can be the normal vector displacement vector video code stream or the texture coordinate displacement vector video code stream.

[0212] (5) Displacement vector decoding.

[0213] This module is responsible for restoring the decoded displacement vector image to displacement vector.

[0214] (6) Reconstruct the deformed mesh.

[0215] After the base mesh and displacement vector decoding and reconstruction are completed, the deformed mesh is reconstructed according to the two parts. The corresponding displacement vector is added to each vertex of the subdivision mesh, that is, formula (8) is added: deformedmesh[i].v[k] = subdivmesh[i].v[k] + displacement[k] (8)

[0216] Where subdivmesh[i].v[k] is the geometric coordinates of the kth vertex of the current base mesh (index i) after subdivision, displacement[k] is the spatial displacement vector corresponding to the kth vertex, and deformedmesh[i].v[k] is the geometric coordinates of the kth vertex of the current base mesh after subdivision and deformation.

[0217] The attribute displacement vector can be applied to the corresponding attribute value in the same way. Taking the texture coordinate as an example, formula (9) is added: deformedmesh[i].vt[k] = subdivmesh[i].vt[k] + att displacement[k] (9)

[0218] Where subdivmesh[i].vt[k] represents the kth texture coordinate after subdivision of the current base mesh (index i), att displacement[k] represents the kth value of the attribute displacement vector, and deformedmesh[i].vt[k] represents the value of the kth texture coordinate after subdivision and deformation of the current base mesh.

[0219] (7) Attribute map decoding

[0220] The attribute map decoder is responsible for decoding the attribute map code stream. The attribute map is decoded using the video decoder indicated in the auxiliary information. An optional color space conversion is performed to obtain an image format consistent with the input attribute map at the encoding end, and the final decoded output attribute map is obtained. For multiple attribute map code streams, after decoding, each attribute map is matched with the corresponding texture coordinate according to the identifier.

[0221] After the processing of each module is completed, the deformed mesh reconstructed at the decoding end and the corresponding attribute map are finally obtained, and the reconstructed deformed mesh and attribute map are used as input for subsequent application processing.

[0222] It should be noted that the VDMC also adopts the concept of submesh, allowing the input three-dimensional mesh to be divided into multiple submeshes and each submesh to be independently encoded and decoded.

[0223] Specifically, the input mesh is divided into divided submeshes using a bounding box-based division scheme before encoding, and then the connection information, geometry information and UV coordinates of each submesh are independently encoded based on the Edgebreaker algorithm and stored. The core module, i.e., the module for encoding connection information, uses the Edgebreaker algorithm. The encoding of geometry information and attribute information such as texture coordinates can use conventional compression methods, i.e., quantization, prediction compression (parallelogram prediction) and entropy encoding. Since the connection relationship driven encoding method is used, the encoding of geometry information and attribute information will follow the encoding order of the connection information. At the decoding end, after decoding each submesh, the information of each submesh is directly added to a new mesh to obtain the final decoded mesh.

[0224] However, for the implementation scheme of V-DMC, in the lossy mode, when the encoding end independently encodes the submesh, it is possible to simplify and quantize the submesh. The simplification of the submesh will cause the loss of the boundary points of the submesh, and the quantization will cause the displacement of the boundary points, and the loss or displacement of the boundary points of the submesh will cause the merged mesh to have a crack area, so that a decoded mesh with high quality cannot be recovered. In a possible implementation, for each boundary point of each submesh, the encoding end can transmit its corresponding distance threshold to the decoding end, so that the decoding end can identify the matching boundary point of each boundary point, and then the decoding end can identify the crack area based on the matching boundary point of each boundary point, and improve the quality of the reconstructed mesh by post-processing the crack area. Specifically, the decoding end traverses the boundary points of the current submesh, and if the distance between the current boundary point and a boundary point of another submesh is less than the distance threshold, it is considered that the two boundary points belong to the boundary points used to form the crack area.

[0225] However, transmitting a distance threshold for each boundary point of each submesh will result in a large amount of data transmission, thereby reducing the mesh decoding efficiency and the real-time performance of mesh decoding.

[0226] Therefore, the embodiments of the present application provide a mesh decoding method, which can improve the mesh decoding efficiency and the real-time performance of mesh decoding.

[0227] The grid encoding method provided in the embodiments of the present application can be executed by an encoding end, for example, the encoder 200 shown in FIG. 1 or FIG. 2. The grid decoding method provided in the embodiments of the present application can be executed by a decoding end, for example, the decoder 300 shown in FIG. 1 or FIG. 3. The encoding end and the decoding end can be implemented by software, hardware or a combination thereof. When implemented by hardware, the encoding end can be referred to as an encoding end device or an encoding device, and the decoding end can be referred to as a decoding end device or a decoding device.

[0228] FIG. 11 is a schematic flowchart of a grid decoding method 400 according to an embodiment of the present application.

[0229] As shown in FIG. 11, the grid decoding method 400 can include at least part of the following contents:

[0230] S401, the decoding end acquires a first reference point of a first reconstructed sub-grid.

[0231] Exemplarily, the first reference point is a geometric vertex of the first reconstructed sub-grid.

[0232] S401, the decoding end determines first crack information of the first reconstructed sub-grid based on the first reference point; wherein the first crack information includes a boundary edge or a boundary point of the first reconstructed sub-grid, and the first crack information is used to determine a first crack region between the first reconstructed sub-grid and a second reconstructed sub-grid.

[0233] Exemplarily, the first crack information includes all boundary edges or boundary points in the first reconstructed sub-grid used to determine the first crack region. Alternatively, the decoding end determines all boundary edges or boundary points in the first reconstructed sub-grid used to determine the first crack region based on the first reference point.

[0234] Exemplarily, after determining the first crack information, the decoding end determines the first crack region based on the first crack information. For example, after determining the first crack information and second crack information of the second reconstructed sub-grid, the decoding end determines the first crack region based on the first crack information and the second crack information. For example, the decoding end determines a region between a boundary edge (or a boundary point) in the first crack information and a boundary edge (or a boundary point) in the second crack information as the first crack region.

[0235] In this embodiment, the first reference point can assist the decoding end to determine the boundary edge or the boundary point of the first reconstructed sub-grid used to form the first crack region, which can reduce the amount of data decoded by the decoding end, and further can improve the grid decoding efficiency and the real-time performance of the grid decoding.

[0236] It should be understood that the crack region referred to in the present application can also be replaced with a term having similar meaning such as crack, and the crack information referred to in the present application can also be replaced with a term having similar meaning such as crack boundary or crack region boundary, and the present application does not make specific limitation thereto.

[0237] The scheme of the present application will be described below in combination with the accompanying drawings.

[0238] In the encoding process, as shown in FIG. 12, the encoding end can obtain sub-grid 1 and sub-grid 2 after performing sub-grid division on the original grid. The dotted line in the sub-grid marks the point on the boundary (i.e. boundary point) or the edge (i.e. boundary edge). After performing independent lossy coding on sub-grid 1 and sub-grid 2 respectively, due to the need of simplification and quantization of each sub-grid in the lossy coding, the boundary point with index 0 of sub-grid 1 is lost in the simplification process, as shown in FIG. 13, which results in misalignment of the boundary of sub-grid 1 and sub-grid 2 and generates a crack region, as shown in FIG. 14, i.e. the region formed by vertex 1 and vertex 3 of sub-grid 1 and vertex 1, vertex 2 and vertex 4 of sub-grid 2 is the crack region generated due to the lossy coding. The vertex 1 of sub-grid 1 and the vertex 1 of sub-grid 2 can be overlapped or not overlapped, and similarly, the vertex 3 of sub-grid 1 and the vertex 4 of sub-grid 2 can be overlapped or not overlapped. In this case, the encoding end needs to transmit the relevant information for determining the crack region to the decoding end for post-processing through a corresponding transmission method. For the convenience of description, the relevant information for determining the crack region transmitted by the encoding end is referred to as crack indication information.

[0239] Generally, the crack indication information transmitted by the encoding end includes the distance threshold of each boundary point of each sub-grid. For example, taking FIG. 14 as an example, the crack indication information includes the distance threshold of the vertex 1 of the sub-grid 1 and the distance threshold of the vertex 2 of the sub-grid 1. For example, the encoding end can traverse the vertex 1, the vertex 2 and the vertex 4 of the sub-grid 2, and since the distance between the vertex 1 of the sub-grid 1 and the vertex 1 of the sub-grid 1 is the smallest, the distance (denoted as distance 1) between the vertex 1 of the sub-grid 2 and the vertex 1 of the sub-grid 1 can be taken as the distance threshold of the vertex 1 of the sub-grid 1; similarly, the distance (denoted as distance 2) between the vertex 4 of the sub-grid 2 and the vertex 3 of the sub-grid 1 can be taken as the distance threshold of the vertex 3 of the sub-grid 1. In an improved scheme, the maximum distance between the distance 1 and the distance 2 can be directly taken as the distance threshold of the sub-grid 1, that is, all the boundary points (that is, the vertex 1 and the vertex 3) in the sub-grid 1 share one distance threshold, so as to reduce the number of distance thresholds transmitted by the encoding end. In another improved scheme, the maximum distance among the distance thresholds of a plurality of sub-grids can be taken as the distance threshold of the plurality of sub-grids, that is, all the boundary points in the plurality of sub-grids share one distance threshold, so as to further reduce the number of distance thresholds transmitted by the encoding end.

[0240] Since the traversal method for determining the distance threshold will cause excessive calculation of the encoding end, in an improved scheme, the encoding end can search a plurality of nearest neighbor points of the current boundary point in space, and then determine the minimum distance from the plurality of nearest neighbor points as the distance threshold of the current boundary point. Taking FIG. 13 as an example, the encoding end can search 6 nearest neighbor points of the vertex 1 of the sub-grid 1 in space, and then determine the minimum distance from the 6 nearest neighbor points as the distance threshold of the vertex 1 of the sub-grid 1. However, when the crack region is large, the 6 nearest neighbor points searched by the encoding end for the current boundary point may all be internal points of the sub-grid 1, that is, the actual distance between the current boundary point and the boundary points of other sub-grids may exceed the distance threshold, so that the decoding end may not be able to identify the boundary points belonging to the crack region and matching the current boundary point, that is, the information for determining the crack region is partially missing, thereby reducing the grid decoding performance.

[0241] In the embodiment, the boundary edge or boundary point for forming the first crack region for determining the first reconstructed sub-grid is directly determined based on the first reference point, so that the information for determining the first crack region is avoided to be transmitted for each boundary point of the first reconstructed sub-grid, thereby not only being capable of reducing the data amount decoded by the decoding end, and further being capable of improving the grid decoding efficiency and the real-time performance of the grid decoding, but also being capable of breaking through the limitation that the information for determining the crack region is partially missing due to the distance threshold when the crack region is large, and further improving the grid decoding performance.

[0242] In some embodiments, the S401 comprises:

[0243] The decoding end decodes first indication information; wherein the first indication information is used to indicate the first reference point.

[0244] Exemplarily, the first indication information can be an index of the first reference point, or the first indication information can be used to indicate an index of the first reference point.

[0245] The index of the first reference point can be a unique identifier of the first reference point in the first reconstructed sub-mesh. Alternatively, the index of the first reference point can be a unique identifier of the first reference point among all boundary points of the first reconstructed sub-mesh.

[0246] Exemplarily, the first indication information can be geometric coordinates of the first reference point, or the first indication information can be used to indicate geometric coordinates of the first reference point.

[0247] In this embodiment, the first indication information is used to indicate the first reference point, so that the decoding end can accurately obtain the first reference point, and then can directly determine the boundary edge or boundary point used to form the first crack region for determining the first reconstructed sub-mesh based on the first reference point, avoiding the transmission of information used to determine the first crack region for each boundary point of the first reconstructed sub-mesh. Therefore, not only can the amount of data decoded by the decoding end be reduced, thereby improving the efficiency and real-time performance of mesh decoding, but also the limitation of partial missing information used to determine the crack region based on the distance threshold when the crack region is large can be broken through, thereby improving the performance of mesh decoding.

[0248] In addition, since the first reference point is indicated by the first indication information, the encoding end can flexibly indicate the first reference point through the first indication information, thereby improving the flexibility of the decoding end in obtaining the first reference point.

[0249] Of course, in other alternative embodiments, the rules for the decoding end to obtain the first reference point or the default position of the first reference point can also be agreed by the protocol. For example, the decoding end merges a plurality of reconstructed sub-meshes to obtain a merged mesh; the plurality of reconstructed sub-meshes include the first reconstructed sub-mesh and the second reconstructed sub-mesh; and the default position of the first reference point can be a boundary point in the merged mesh that satisfies a preset condition, i.e., the decoding end can not need to decode the first indication information, thereby improving the decoding efficiency.

[0250] In some embodiments, before the S402, the method 400 further comprises:

[0251] The decoding end merges a plurality of reconstructed sub-grids to obtain a merged grid; the plurality of reconstructed sub-grids include the first reconstructed sub-grid and the second reconstructed sub-grid.

[0252] The S402 includes:

[0253] The decoding end traverses a boundary edge or a boundary point of the merged grid based on the first reference point until a non-boundary edge or a non-boundary point of the merged grid to obtain the first crack information.

[0254] For example, the decoding end traverses a boundary edge of the merged grid based on the first reference point until a non-boundary edge of the merged grid to obtain the first crack information; the first crack information includes all the traversed boundary edges.

[0255] For example, the decoding end traverses a boundary point of the merged grid based on the first reference point until a non-boundary point of the merged grid to obtain the first crack information; the first crack information includes all the traversed boundary points.

[0256] For example, the boundary edge refers to an edge of the merged grid that shares only one patch. The boundary point refers to a point forming a boundary edge or a point of the merged grid that shares only one or two patches. The non-boundary edge refers to an edge of the merged grid that shares multiple patches. The non-boundary point refers to a point forming a non-boundary edge. The non-boundary edge can also be an internal edge, and the non-boundary point can also be referred to as an internal point.

[0257] In this embodiment, the decoding end traverses a boundary edge or a boundary point of the merged grid based on the first reference point until a non-boundary edge or a non-boundary point of the merged grid to obtain the first crack information, which can ensure the integrity of the first crack information.

[0258] In some embodiments, the first reference point is a starting point of traversing a boundary or a boundary point of the merged grid.

[0259] For example, the decoding end obtains a starting point and traverses a boundary edge or a boundary point of the merged grid based on the starting point until a non-boundary edge or a non-boundary point of the merged grid to obtain the first crack information.

[0260] In this embodiment, the first reference point is a starting point of traversing a boundary or a boundary point of the merged grid, which can make the understanding of the decoding end on the first crack region consistent with the understanding of the encoding end on the first crack region, thereby improving the accuracy of the decoding end in determining the first crack region.

[0261] Of course, in other alternative embodiments, the first reference point can also not be the starting point, for example, the first reference point can be any one boundary point in the first reconstructed sub-grid for forming the first crack region, which is not limited in the present application.

[0262] In some embodiments, the decoding end traverses the boundary edge or boundary point of the merged grid based on the first reference point until the non-boundary edge or non-boundary point of the merged grid to obtain the first crack information, including:

[0263] The decoding end decodes the second indication information; and the decoding end traverses the boundary edge or boundary point of the first reconstructed sub-grid in the merged grid based on the traversal order indicated by the second indication information until the non-boundary edge or non-boundary point of the merged grid to obtain the first crack information.

[0264] Illustratively, the boundary edge of the first reconstructed sub-grid in the merged grid refers to the common boundary edge of the first reconstructed sub-grid and the merged grid. The boundary point of the first reconstructed sub-grid in the merged grid refers to the common boundary point of the first reconstructed sub-grid and the merged grid.

[0265] Illustratively, the decoding end first decodes the first indication information and the second indication information mentioned above, and then takes the first reference point indicated by the first indication information as the starting point, traverses the boundary edge or boundary point of the first reconstructed sub-grid in the merged grid according to the traversal order indicated by the second indication information until the non-boundary edge or non-boundary point of the merged grid to obtain the first crack information.

[0266] In the present embodiment, by introducing the second indication information, the decoding end can keep consistent with the encoding end in understanding the first crack region, thereby improving the accuracy of the decoding end in determining the first crack region.

[0267] In some embodiments, the second indication information includes at least one of the following:

[0268] the index of the point traversed after the first reference point;

[0269] an identifier for indicating the traversal direction;

[0270] an identifier indicating the relationship between the point traversed after the first reference point and the adjacent point of the first reference point.

[0271] Exemplarily, the point traversed after the first reference point can be referred to as a sequential point, and the second indication information can include an index of the sequential point. Of course, in other alternative embodiments, the second indication information can also be used to indicate the index of the point traversed after the first reference point.

[0272] Exemplarily, when the value of the identifier is a first numerical value, the traversal direction is a direction pointing to a neighboring point of the first reference point with a largest index; and when the value of the identifier is a second numerical value, the traversal direction is a direction pointing to a neighboring point of the first reference point with a smallest index. Optionally, the first numerical value is 1 and the second numerical value is 0; or the first numerical value is 0 and the second numerical value is 1.

[0273] In this embodiment, the second indication information can be designed in multiple forms of information, which can improve the indication flexibility of the second indication information.

[0274] In some embodiments, before the S401, the method 400 further includes:

[0275] The decoding end decodes third indication information; wherein the third indication information is used to indicate the number of crack regions between the first reconstructed sub-grid and the second reconstructed sub-grid.

[0276] Exemplarily, the decoding end decodes the third indication information to obtain the number of crack regions between the first reconstructed sub-grid and the second reconstructed sub-grid, decodes crack indication information for determining each crack region according to the number, for example, including the first indication information and the second indication information mentioned above, and selects the first reference point indicated by the first indication information as a starting point to sequentially traverse along a boundary edge (an edge shared by only one triangle) or a boundary point (a point forming a boundary edge) of the merged grid (wherein the sequence is that the first point is the initial point and the second point is the next point in the direction indicated by the second indication information), and the traversed boundary edge (or boundary point) is the boundary edge (or boundary point) in the first crack information. When all the boundary edges (or boundary points) are traversed, it is considered that the complete boundary edge (or boundary point) has been identified, which is used as the first crack information to identify the first crack region.

[0277] In this embodiment, the third indication information is decoded by the decoding end, so that the decoding end can support obtaining multiple information for determining multiple crack regions, for example, so that the decoding end can support obtaining multiple reference points for determining multiple crack regions. In particular, when there are multiple crack regions between the first reconstructed sub-grid and the second reconstructed sub-grid, the third indication information can accurately obtain information for determining each crack region, so that the decoding end can accurately locate each crack region, thereby providing better auxiliary information for the corresponding post-processing crack filling algorithm, and improving the processing effect of post-processing.

[0278] In some embodiments, the method 400 further includes:

[0279] The decoding end decodes fourth indication information; wherein the fourth indication information is used to indicate the second reconstructed sub-grid.

[0280] For example, the fourth indication information can be used to indicate the index of the second reconstructed sub-grid.

[0281] For example, in the case that the number of crack regions between the first reconstructed sub-grid and the second reconstructed sub-grid is greater than or equal to 1, the encoding end encodes the fourth indication information; correspondingly, the decoding end decodes the fourth indication information.

[0282] For example, after the decoding end decodes the first crack information, the decoding end obtains the second crack information of the second reconstructed grid based on the fourth indication information, and then determines the first crack region based on the first crack information and the second crack information.

[0283] In some embodiments, the S401 includes:

[0284] The decoding end decodes fifth indication information; in the case that the fifth indication information indicates that the decoding end determines the first crack information based on the first reference point, the decoding end obtains the first reference point.

[0285] For example, in the case that the fifth indication information indicates that the decoding end determines the first crack information based on the first reference point, the decoding end obtains the first reference point. In the case that the fifth indication information indicates that the decoding end determines the first crack region based on other information, the decoding end obtains the other information.

[0286] For example, the other information can be a distance threshold of the boundary point of the first reconstructed sub-grid. For example, in a case where the fifth indication information indicates that the decoding end determines the first crack region based on the distance threshold of the boundary point of the first reconstructed sub-grid, the decoding end acquires the distance threshold of the boundary point of the first reconstructed sub-grid.

[0287] For another example, the other information can be an index of all boundary points of the first reconstructed sub-grid. For example, in a case where the fifth indication information indicates that the decoding end determines the first crack region based on the index of all boundary points of the first reconstructed sub-grid, the decoding end acquires the index of all boundary points of the first reconstructed sub-grid.

[0288] The grid decoding method provided in the application will be described below in combination with specific embodiments.

[0289] Embodiment 1

[0290] As shown in FIG. 15, the decoding end mainly includes the following modules when reconstructing the grid: a sub-grid decoding module, a sub-grid merging module, a crack indication information decoding module, a crack region identification module, a merged grid post-processing module, and a texture image decoding module.

[0291] The sub-grid decoding module is configured to decode the sub-grid code stream to obtain a reconstructed sub-grid. The sub-grid merging module is configured to merge the reconstructed sub-grids to obtain a merged grid, which can also be referred to as a decoded grid with cracks. In addition, the crack indication information decoding module is configured to decode the crack indication information code stream to obtain crack indication information, which is used by the crack region identification module to identify a crack region. The merged grid post-processing module is configured to post-process the merged grid based on the identified crack region to obtain a reconstructed grid. The texture image decoding module is configured to decode the texture image code stream to obtain a texture image.

[0292] In this embodiment, the crack indication information can include at least one of the first indication information, the second indication information, the third indication information, the fourth indication information, and the fifth indication information.

[0293] The following will be described in detail respectively.

[0294] 1) Sub-grid decoding module

[0295] Input: sub-grid code stream

[0296] Output: reconstructed sub-grid

[0297] In the sub-grid decoding process, the decoding end independently decodes each sub-grid. The decoding of each sub-grid mainly includes connection relationship decoding, geometry information decoding, and attribute information decoding. Each information is decoded by using a decoding method corresponding to the encoding method used by the encoding end.

[0298] The decoding method based on Edgebreaker is taken as an example for detailed description.

[0299] As shown in FIG. 16, the decoding end first decodes the connection relationship, geometry information and attribute information of the manifold mesh. For connection relationship decoding, the mode string is first decoded, and then the mode string is traversed in a certain order (forward order or reverse order), and the connection relationship is reconstructed according to the corresponding mode in the string. In addition, the traversal order of the vertex is output to the geometry information and attribute information decoding module. When decoding the geometry information, the decoding method corresponding to the encoding end is used according to the decoding order of the connection relationship, and the geometry information is obtained. When decoding the three-dimensional mesh attribute, the decoding method corresponding to the encoding end is used according to the decoding order of the connection relationship, and the three-dimensional mesh attribute information is decoded. According to the decoded connection relationship, geometry information and attribute information, the manifold submesh can be directly reconstructed. Each edge in the manifold submesh intersects with exactly two other edges, and there is no intersecting edge or degenerate geometry feature, which guarantees the consistency and continuity of the mesh topology.

[0300] 2) Submesh merging module.

[0301] Input: Reconstructed submesh.

[0302] Output: Merged mesh.

[0303] The submesh merging module adds the information of each reconstructed submesh obtained by decoding to a new mesh to obtain a decoded mesh with cracks, i.e., a merged mesh.

[0304] 3) Crack region identification module.

[0305] Input: Crack indication information.

[0306] Output: Crack region.

[0307] The crack region identification module combines the crack indication information and uses a certain method to identify the complete crack region. The application does not limit the specific identification method. The representation method of the crack indication information can be various.

[0308] The following takes an identification method as an example:

[0309] When the representation method of Method=1 is selected, the decoding end determines the crack region based on the distance threshold of each sub-grid. For example, the decoding end obtains the distance threshold of the current boundary point in the current reconstructed sub-grid, and traverses the boundary points of other reconstructed sub-girds. If the distance between the current boundary point and other boundary points in other reconstructed sub-girds is less than the distance threshold, the two boundary points match each other, that is, they are used to form the same crack region. After the boundary points of other reconstructed sub-girds are traversed, the identification of the crack region between the current reconstructed sub-grid and other reconstructed sub-girds is completed.

[0310] When the representation method of Method=4 is selected, the decoding end obtains the index of the current boundary point in the current reconstructed sub-grid, and traverses the boundary points of other reconstructed sub-girds. If the index of the current boundary point is the same as the index of other boundary points in other reconstructed sub-girds, the two boundary points match each other, that is, they are used to form the same crack region. After the boundary points of other reconstructed sub-girds are traversed, the identification of the crack region between the current reconstructed sub-grid and other reconstructed sub-girds is completed.

[0311] When the representation method of Method=5 is selected, the decoding end decodes the number of crack regions between the first reconstructed sub-grid and the second reconstructed sub-grid, and decodes the indication information of the starting point (such as the first indication information mentioned above) and the indication information of the traversal order (such as the second indication information mentioned above) according to the number. Then, based on the starting point, the traversal order is followed along the boundary edge (an edge shared by only one triangle) or the boundary point (a point forming the boundary edge) of the merged grid. The traversed boundary edge (or boundary point) is the boundary edge (or boundary point) in the first crack information. When all the boundary edges (or boundary points) are traversed, it is considered that the complete boundary edge (or boundary point) has been identified, which is used as the first crack information to identify the first crack region. For example, after the decoding end decodes the first crack information, it can also obtain the second crack information of the second reconstructed grid by decoding. Then, based on the first crack information and the second crack information, the first crack region is determined.

[0312] 4), merged grid post-processing module.

[0313] Input: merged grid, crack region.

[0314] Output: reconstructed grid

[0315] The merged mesh post-processing module can be used to fill or connect the crack region to generate a crack-free reconstructed mesh. Filling refers to adding new faces or vertices in the crack region to eliminate the crack. This can involve inserting new vertices on the boundary of the crack region and then creating new faces to cover the crack. Connecting refers to connecting the two sides of the crack if the two sides of the crack region can be aligned by adjusting the position of the vertices or edges to eliminate the discontinuity.

[0316] FIG. 17 is a schematic flowchart of a mesh encoding method 500 according to an embodiment of the present application.

[0317] As shown in FIG. 17, the mesh encoding method 500 can include at least part of the following contents:

[0318] S501, an encoding end obtains at least one boundary point of a first reconstructed sub-mesh based on sub-mesh division;

[0319] S502, the encoding end determines a first reference point of the first reconstructed sub-mesh based on the at least one boundary point;

[0320] The first reference point is used to determine first crack information of the first reconstructed sub-mesh, the first crack information includes a boundary edge or a boundary point of the first reconstructed sub-mesh, and the first crack information is used to determine a first crack region between the first reconstructed sub-mesh and a second reconstructed sub-mesh.

[0321] In some embodiments, the method 500 further includes:

[0322] The encoding end encodes first indication information;

[0323] The first indication information is used to indicate the first reference point.

[0324] In some embodiments, the first reference point is a starting point of traversing a boundary or a boundary point of a merged mesh, the merged mesh is a mesh obtained by merging a plurality of reconstructed sub-meshes, and the plurality of reconstructed sub-meshes include the first reconstructed sub-mesh.

[0325] In some embodiments, the method 500 further includes:

[0326] The encoding end encodes second indication information;

[0327] The second indication information is used to indicate a traversal order.

[0328] In some embodiments, the second indication information includes at least one of the following:

[0329] an index of a point traversed after the first reference point;

[0330] an identifier for indicating a traversal direction;

[0331] an identifier indicating a relationship between a point traversed after the first reference point and a neighboring point of the first reference point.

[0332] In some embodiments, the method 500 further comprises:

[0333] encoding, by the encoding end, third indication information;

[0334] The third indication information is used to indicate a number of crack regions between the first reconstructed sub-mesh and the second reconstructed sub-mesh.

[0335] In some embodiments, the method 500 further comprises:

[0336] encoding, by the encoding end, fourth indication information;

[0337] The fourth indication information is used to indicate the second reconstructed sub-mesh.

[0338] In some embodiments, the method 500 further comprises:

[0339] encoding, by the encoding end, fifth indication information;

[0340] The fifth indication information is used to indicate that the decoding end determines the first crack information based on the first reference point.

[0341] It should be understood that the encoding method can be understood as the inverse process (or called reverse process) of the decoding method, therefore, the specific scheme of the mesh encoding method 500 can refer to the related content of the mesh decoding method 400, and for the convenience of description, the present application will not repeat it here.

[0342] The mesh encoding method provided by the present application will be described below in combination with specific embodiments.

[0343] Embodiment 2:

[0344] As shown in FIG. 18, the encoding framework is mainly divided into several parts: sub-mesh division, sub-mesh compression, crack region identification, crack indication information compression, texture map encoding and the like modules.

[0345] The sub-mesh division module is used to divide the original mesh into a plurality of sub-meshes, the sub-mesh compression module is used to encode the sub-meshes to obtain sub-mesh code streams; the crack region identification module is used to identify the crack regions between the sub-meshes; the crack indication information encoding module is used to encode the crack indication information to obtain crack indication information code streams. The texture map encoding module is used to encode the texture map code streams to obtain the texture map.

[0346] In this embodiment, the crack indication information can include at least one of the first indication information, the second indication information, the third indication information, the fourth indication information, and the fifth indication information.

[0347] The following are described respectively:

[0348] 1) Sub-grid division module.

[0349] Input: original grid.

[0350] Output: divided sub-grid.

[0351] The present application does not limit the division method of the sub-grid, for example, a variety of methods such as bounding box-based division can be used. The specific division method is not emphasized here. The following describes an example based on the bounding box-based division method.

[0352] First, according to the geometric coordinates of the input original grid, the range of the bounding box in the three-dimensional space where it is located can be obtained. Then, the coordinate range based on the x-axis or y-axis or z-axis is divided, and the number of divisions can be pre-set or set according to the number of sub-grids required by the input. After the division is completed, the range of the bounding box of each sub-grid is obtained. For the range of each bounding box, all the facets in the original grid are traversed, and for the vertices in the input grid that meet the range of each bounding box, the facets are divided into the current sub-grid. If the vertices of a facet are all located in the same bounding box, the facet is completely divided into the corresponding sub-grid. During the division process, some facets may have vertices located on the boundary of two or more bounding boxes. For these cases, a more specific division strategy can be used to determine how to allocate these facets. After traversing the divided bounding boxes, the final divided sub-grids are obtained.

[0353] 2) Sub-grid compression module.

[0354] Input: divided sub-grid.

[0355] Output: sub-grid code stream.

[0356] The sub-grid compression module independently encodes each divided sub-grid. The encoding of each sub-grid mainly includes connection relationship encoding, geometric information encoding, and attribute information encoding. The Edgebreaker-based method can be used to encode each sub-grid, and the specific encoding method is not emphasized here.

[0357] The following describes an example based on the Edgebreaker-based method.

[0358] As shown in FIG. 19, the encoding end splits the non-manifold structure in the input sub-grid to obtain a manifold grid. Then, the connection relationship, geometric information and attribute information of the manifold grid are encoded. For connection relationship encoding, the Edgebreaker method can be used to encode the connection relationship of the three-dimensional grid. The connection relationship of the grid is represented by establishing a CornerTable, and all triangles in the grid are traversed using the CornerTable to generate the CLERS mode string of Edgebreaker. Then, the CLERS mode string can be encoded in an entropy encoding manner to obtain the connection information code stream. The geometric information can be encoded in the encoding order of the connection relationship using various methods such as difference prediction encoding algorithm, parallelogram prediction encoding algorithm, etc., to obtain the geometric information code stream. The specific encoding method is not emphasized here. The attribute information of the three-dimensional grid generally includes texture coordinates, normal vectors, etc. Taking the texture coordinates as an example, the texture coordinates can be encoded in the encoding order of the connection relationship using various methods such as difference prediction encoding, parallelogram prediction encoding and similar triangle prediction encoding, etc., to obtain the attribute information code stream. The specific encoding method is not emphasized here.

[0359] 3) Crack region identification.

[0360] Input: reconstructed sub-grid.

[0361] Output: crack indication information.

[0362] There are various ways to identify the crack region, and the application does not limit the specific identification method.

[0363] The following takes one identification method as an example:

[0364] The boundary of the reconstructed sub-grid is identified, that is, the shared triangles of the grid edge are counted; if an edge is shared by 2 triangles, the edge is an internal edge; if the edge is shared by 1 triangle, the edge is a boundary edge. The encoding end first obtains the boundary edge of the sub-grid, then combines the sub-grid division axis space coordinates, removes the boundary edge of the original grid, and retains the boundary edge of the crack region generated by the division, to obtain the boundary edge as the crack information. The encoding end can determine the crack indication information based on the crack information. There are various methods to represent the crack indication information.

[0365] When the representation method of Method = 1 is enabled, the crack region is represented as the distance between the crack boundary points being less than or equal to the distance threshold. Specifically, when the distance between the points in different decoded sub-grids is less than the represented distance threshold, it is considered that the two points match in distance and are located in the same crack region, that is, the encoding end can encode this distance threshold information as crack indication information.

[0366] When the representation method of Method=4 is enabled, the crack region is represented as index matching between crack boundary points, and when the points between different decoding sub-grids have the same global index, it is considered that the two points index match and are in the same crack region, that is, the index of the boundary point can be encoded as crack indication information at the encoding end.

[0367] When the representation method of Method=5 is enabled, the crack region is represented as the starting point of the crack region and the traversal order. The encoding end traverses the entire crack region along the boundary edge (the boundary edge shared by only one triangle) to obtain the starting point and the traversal order (for example, the next traversal order point) of the crack region. Considering that there can be multiple crack regions between a sub-grid and other sub-grids, there are multiple starting points and traversal orders (for example, the next traversal order point) of the crack regions, therefore, the crack indication information includes the number of crack regions of the sub-grid, and each crack region corresponds to a starting point and a traversal order (for example, the next traversal order point). In order to facilitate the post-processing process of the decoding end, the encoding end can add an indication information for each crack region, indicating that the crack region is formed by the sub-grid and the sub-grid with index x.

[0368] 4), crack indication information compression module.

[0369] Input: crack indication information.

[0370] Output: crack indication information code stream.

[0371] The crack indication information compression module encodes the crack indication information by using a certain encoding method and writes it into the code stream. The present application does not limit the specific encoding method, for example, the indication information of the starting point can be encoded by using unsigned exponential Golomb code, and the indication information of the traversal order can be encoded by using the relative size relationship between the index of the next traversal order point and the index of the starting point.

[0372] The execution subject of the grid decoding method provided in the embodiments of the present application can be a grid decoding device. As an example, the device can be an electronic device, or a component in an electronic device, such as a chip, a circuit, etc. In the embodiments of the present application, the grid decoding device is taken as an example to execute the grid decoding method, and the grid decoding device provided in the embodiments of the present application is described.

[0373] FIG. 20 is a schematic block diagram of a grid decoding device 600 according to an embodiment of the present application.

[0374] As shown in FIG. 20, the grid decoding device 600 includes:

[0375] The acquisition module 601 is configured to acquire a first reference point of a first reconstructed sub-grid.

[0376] The processing module 602 is configured to determine first crack information of the first reconstructed sub-grid based on the first reference point.

[0377] The first crack information includes a boundary edge or a boundary point of the first reconstructed sub-grid, and the first crack information is used to determine a first crack region between the first reconstructed sub-grid and a second reconstructed sub-grid.

[0378] In some embodiments, the first reference point of the first reconstructed sub-grid is obtained by:

[0379] The first indication information is decoded.

[0380] The first indication information is used to indicate the first reference point.

[0381] In some embodiments, before the processing module 602 determines the first crack information of the first reconstructed sub-grid based on the first reference point, the processing module 602 is further configured to:

[0382] The plurality of reconstructed sub-grids includes the first reconstructed sub-grid and the second reconstructed sub-grid.

[0383] The first crack information is determined based on the first reference point, including:

[0384] The first crack information is obtained by traversing a boundary edge or a boundary point of the merged grid based on the first reference point until a non-boundary edge or a non-boundary point of the merged grid.

[0385] In some embodiments, the first reference point is a starting point of traversing the boundary or the boundary point of the merged grid.

[0386] In some embodiments, the first crack information is obtained by traversing a boundary edge or a boundary point of the merged grid based on the first reference point until a non-boundary edge or a non-boundary point of the merged grid, including:

[0387] The second indication information is decoded.

[0388] The first crack information is obtained by traversing a boundary edge or a boundary point of the first reconstructed sub-grid in the merged grid based on the traversal order indicated by the second indication information until a non-boundary edge or a non-boundary point of the merged grid.

[0389] In some embodiments, the second indication information includes at least one of:

[0390] An index of a point traversed after the first reference point.

[0391] an identifier indicating a traversal direction;

[0392] an identifier indicating a relationship between a point traversed after the first reference point and a neighboring point of the first reference point.

[0393] In some embodiments, before the obtaining module 601 obtains the first reference point of the first reconstructed sub-grid, the obtaining module 601 is further configured to:

[0394] decode third indication information;

[0395] The third indication information is used to indicate a number of crack regions between the first reconstructed sub-grid and the second reconstructed sub-grid.

[0396] In some embodiments, the obtaining module 601 is further configured to:

[0397] decode fourth indication information;

[0398] The fourth indication information is used to indicate the second reconstructed sub-grid.

[0399] In some embodiments, the obtaining the first reference point of the first reconstructed sub-grid comprises:

[0400] decode fifth indication information;

[0401] In a case where the fifth indication information indicates that the decoding end determines the first crack information based on the first reference point, the first reference point is obtained.

[0402] It should be understood that the grid decoding apparatus 600 provided by the embodiments of the present application can correspond to the execution subject in the method embodiments of the present application, and each unit in the grid decoding apparatus 600 is respectively for realizing the corresponding process of the grid decoding method 400 shown in FIG. 11, and for the sake of brevity, it will not be repeated here.

[0403] The grid decoding apparatus 600 provided by the embodiments of the present application can realize each process realized by the method embodiments of FIG. 11, and achieve the same technical effects, to avoid repetition, which will not be repeated here.

[0404] The grid encoding method provided by the embodiments of the present application, the execution subject can be a grid encoding apparatus. As an example, the apparatus can be an electronic device, or a component in an electronic device, such as a chip, a circuit, etc. In the embodiments of the present application, the grid encoding apparatus performs the grid encoding method as an example, and the grid encoding apparatus provided by the embodiments of the present application is described.

[0405] FIG. 21 is a schematic block diagram of a grid encoding apparatus 700 according to an embodiment of the present application.

[0406] As shown in FIG. 21, the mesh encoding apparatus 700 includes:

[0407] An acquisition module 701 is configured to acquire at least one boundary point of a first reconstructed sub-mesh generated based on sub-mesh division.

[0408] A processing module 702 is configured to determine a first reference point of the first reconstructed sub-mesh based on the at least one boundary point.

[0409] The first reference point is used to determine first crack information of the first reconstructed sub-mesh, the first crack information includes a boundary edge or a boundary point of the first reconstructed sub-mesh, and the first crack information is used to determine a first crack region between the first reconstructed sub-mesh and a second reconstructed sub-mesh.

[0410] In some embodiments, the processing module 702 is further configured to:

[0411] Encode first indication information.

[0412] The first indication information is used to indicate the first reference point.

[0413] In some embodiments, the first reference point is a starting point of a boundary or a boundary point of a merged mesh, and the merged mesh is a mesh obtained by merging a plurality of reconstructed sub-meshes, and the plurality of reconstructed sub-meshes include the first reconstructed sub-mesh.

[0414] In some embodiments, the processing module 702 is further configured to:

[0415] Encode second indication information.

[0416] The second indication information is used to indicate a traversal order.

[0417] In some embodiments, the second indication information includes at least one of:

[0418] An index of a point traversed after the first reference point;

[0419] An identifier used to indicate a traversal direction;

[0420] An identifier indicating a relationship between a point traversed after the first reference point and a neighboring point of the first reference point.

[0421] In some embodiments, the processing module 702 is further configured to:

[0422] Encode third indication information.

[0423] The third indication information is used to indicate a number of crack regions between the first reconstructed sub-mesh and the second reconstructed sub-mesh.

[0424] In some embodiments, the processing module 702 is further configured to:

[0425] encode fourth indication information;

[0426] The fourth indication information is used to indicate the second reconstructed sub-grid.

[0427] In some embodiments, the processing module 702 is further configured to:

[0428] encode fifth indication information;

[0429] The fifth indication information is used to indicate that the decoding end determines the first crack information based on the first reference point.

[0430] It should be understood that the grid encoding apparatus 700 provided by the embodiments of the present application can correspond to the execution subject in the method embodiments of the present application, and each unit in the grid encoding apparatus 700 is respectively used to implement the corresponding process of the grid encoding method 500 shown in FIG. 17. For the sake of brevity, no longer be repeated here.

[0431] The grid encoding apparatus provided by the embodiments of the present application can implement each process of the method embodiments shown in FIG. 17, and achieve the same technical effects. To avoid repetition, no longer be repeated here.

[0432] As shown in FIG. 22, the embodiments of the present application further provide an electronic device 800, which includes a processor 801 and a memory 802. The memory 802 has a program or instruction stored thereon, which can be run on the processor 801. For example, when the electronic device 800 is a decoding end device, the program or instruction is executed by the processor 801 to implement each step of the above-mentioned grid decoding method embodiments, and achieve the same technical effects. When the electronic device 800 is an encoding end device, the program or instruction is executed by the processor 801 to implement each step of the above-mentioned grid encoding method embodiments, and achieve the same technical effects. To avoid repetition, no longer be repeated here. Optionally, the memory 802 can be the memory 102 or the memory 113 in the embodiments shown in FIG. 1, and the processor 801 can realize the functions of the encoder 200 or the decoder 300 in the embodiments shown in FIGS. 1-3.

[0433] The embodiments of the present application further provide an electronic device, which includes a memory configured to store video data, and a processing circuitry configured to implement each step of the above-mentioned grid decoding method embodiments or the above-mentioned grid encoding method embodiments. Optionally, the memory can be the memory 102 or the memory 113 in the embodiments shown in FIG. 1, and the processing circuitry can realize the functions of the encoder 200 or the decoder 300 in the embodiments shown in FIGS. 1-3.

[0434] The embodiment of the present application further provides an electronic device, comprising a processor and a communication interface, the communication interface and the processor are coupled, the processor is used for running programs or instructions, and the steps in the method embodiment shown in FIG. 11 or FIG. 17 are realized. The device embodiment corresponds to the above-mentioned method embodiment, each implementation process and implementation manner of the above-mentioned method embodiment can be applied to the terminal embodiment, and the same technical effects can be achieved.

[0435] The processor or processing circuit of the embodiment of the present application can include a general-purpose processor, a special-purpose processor, etc., for example, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), an artificial intelligent (AI) processor, a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a network processor (NP), a field programmable gate array (FPGA) or other programmable logic devices, a gate circuit, a transistor, a discrete hardware component, etc. The communication interface of the embodiment of the present application can include a transceiver, a pin, a circuit, a bus, etc.

[0436] The above-mentioned electronic device can be a terminal, or other devices except the terminal, for example, a server, a network attached storage (NAS), etc.

[0437] The terminal can be a mobile phone, a tablet personal computer, a laptop computer, a notebook computer, a personal digital assistant (PDA), a palm computer, a netbook, an ultra-mobile personal computer (UMPC), a mobile Internet device (MID), an augmented reality (AR) device, a virtual reality (VR) device, a mixed reality (MR) device, a robot, a wearable device, a flight vehicle, a vehicle user equipment (VUE), a shipboard device, a pedestrian user equipment (PUE), a smart home (a home device with a wireless communication function, such as a refrigerator, a television, a washing machine, or furniture), a game console, a personal computer (PC), a teller machine, or a self-service machine, and the like. The wearable device includes a smart watch, a smart bracelet, a smart earphone, smart glasses, smart jewelry (a smart bracelet, a smart necklace, a smart ring, a smart necklace, a smart anklet, a smart necklace, and the like), a smart wristband, smart clothing, and the like. The vehicle-mounted device can also be referred to as a vehicle-mounted terminal, a vehicle-mounted controller, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip, or a vehicle-mounted unit, and the like. It should be noted that the specific type of the terminal is not limited in the embodiments of the present application.

[0438] The server can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server. The cloud server can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), or cloud computing services based on big data and artificial intelligence platforms.

[0439] For example, the electronic device can include, but is not limited to, a source device 100 or a destination device 110 as shown in FIG. 1.

[0440] Taking the electronic device as an example, FIG. 23 is a schematic diagram of the hardware structure of a terminal according to an embodiment of the present application.

[0441] The terminal 900 includes, but is not limited to, at least part of components such as a radio frequency unit 901, a network module 902, an audio output unit 903, an input unit 904, a sensor 905, a display unit 906, a user input unit 907, an interface unit 908, a memory 909, and a processor 910.

[0442] Those skilled in the art can understand that the terminal 900 can further include a power supply (such as a battery) for supplying power to each component, and the power supply can be logically connected to the processor 910 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The terminal structure shown in FIG. 23 does not constitute a limitation on the terminal, and the terminal can include more or fewer components than those shown, or combine certain components, or different component arrangements, which are not described here.

[0443] It should be understood that in the embodiments of the present application, the input unit 904 can include a graphics processor 9041 and a microphone 9042. The graphics processor 9041 processes image data of a still picture or a video obtained by an image acquisition device (such as a camera) in a video acquisition mode or an image acquisition mode, or can process obtained point cloud data. The display unit 906 can include a display panel 9061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 907 includes at least one of a touch panel 9071 and other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 can include two parts of a touch detection device and a touch controller. The other input devices 9072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), trackballs, mice, joysticks, etc., which are not described here.

[0444] In the embodiments of the present application, after the radio frequency unit 901 receives data from the opposite end, the radio frequency unit 901 can transmit the data to the processor 910 for processing. In addition, the radio frequency unit 901 can send data to the opposite end. Generally, the radio frequency unit 901 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, etc.

[0445] The memory 909 can be used to store software programs or instructions and various data. The memory 909 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 909 can include a volatile memory or a non-volatile memory. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 909 in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0446] The processor 910 can include one or more processing units; optionally, the processor 910 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 910.

[0447] As an implementation manner, the terminal 900 is a decoding end, the radio frequency unit 901 is configured to obtain a first reference point of a first reconstructed sub-grid, and the processor 910 is configured to determine first crack information of the first reconstructed sub-grid based on the first reference point; wherein the first crack information includes a boundary edge or a boundary point of the first reconstructed sub-grid, and the first crack information is used to determine a first crack region between the first reconstructed sub-grid and a second reconstructed sub-grid:

[0448] As another implementation manner, the terminal 900 is an encoding end, the radio frequency unit 901 is configured to acquire at least one boundary point of a first reconstructed sub-grid generated based on sub-grid partition, and the processor 910 is configured to determine a first reference point of the first reconstructed sub-grid based on the at least one boundary point; wherein the first reference point is used to determine first crack information of the first reconstructed sub-grid, the first crack information includes a boundary edge or a boundary point of the first reconstructed sub-grid, and the first crack information is used to determine a first crack region between the first reconstructed sub-grid and a second reconstructed sub-grid.

[0449] In the embodiment of the present application, the first reference point can assist the decoding end to determine a boundary edge or a boundary point used to form the first crack region of the first reconstructed sub-grid, so as to reduce the amount of data decoded by the decoding end, and further improve the grid decoding efficiency and the real-time performance of grid decoding.

[0450] It can be understood that the implementation processes of the implementation manners mentioned in the embodiment can refer to the related descriptions of the above-mentioned grid decoding method embodiments or the above-mentioned grid encoding method embodiments, and achieve the same or corresponding technical effects. To avoid repetition, they will not be described here.

[0451] The embodiment of the present application also provides a readable storage medium, the readable storage medium stores a program or instructions, the program or instructions are executed by a processor to realize each process of the above-mentioned grid decoding method embodiments or the above-mentioned grid encoding method embodiments, and can achieve the same technical effects. To avoid repetition, they will not be described here.

[0452] The processor is the processor in the terminal in the above-mentioned embodiments. The readable storage medium includes a computer readable storage medium, such as ROM, RAM, magnetic disk or optical disk, etc. In some examples, the readable storage medium can be a non-transitory readable storage medium.

[0453] The embodiment of the present application further provides a chip, the chip includes a processor and a communication interface, the communication interface and the processor are coupled, the processor is used to run a program or instructions, realizes each process of the above-mentioned grid decoding method embodiments or the above-mentioned grid encoding method embodiments, and can achieve the same technical effects. To avoid repetition, they will not be described here.

[0454] It should be understood that the chip mentioned in the embodiment of the present application can include a system-level chip (also known as a system chip, a chip system or a system-on-chip chip), and can also include a separate display chip, etc.

[0455] The embodiment of the present application further provides a computer program / product stored in a storage medium, which is executed by at least one processor to implement each process of the above-mentioned grid decoding method embodiment or the above-mentioned grid encoding method embodiment, and can achieve the same technical effects. To avoid repetition, details are not described herein.

[0456] The embodiment of the present application further provides a coding and decoding system, which comprises an encoding end device and a decoding end device. The encoding end device can be used to execute the steps of the above-mentioned grid encoding method, and the decoding end device can be used to execute the steps of the above-mentioned grid decoding method.

[0457] It should be noted that, in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles, or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or further include elements inherent to such processes, methods, articles, or devices. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article, or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to the order of performing the functions shown or discussed, and can also include performing the functions in a substantially simultaneous manner or in a reverse order, for example, the described method can be performed in an order different from that described, and various steps can also be added, omitted or combined. In addition, the features described with reference to certain examples can be combined in other examples.

[0458] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of computer software product and general hardware platform, of course, it can also be realized by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disc, optical disc, etc.), and includes a plurality of instructions for making the decoding end or the encoding end execute the method described in each embodiment of the present application.

[0459] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, which are only illustrative and not restrictive. Those skilled in the art can make many forms of embodiments under the inspiration of the present application without departing from the scope of the present application and the protection scope of the claims.

Claims

1. A trellis decoding method, wherein, The method comprises the following steps: The decoding end obtains a first reference point of a first reconstructed sub-grid; The decoding end determines first crack information of the first reconstructed sub-grid based on the first reference point; The first crack information comprises a boundary edge or a boundary point of the first reconstructed sub-grid, and the first crack information is used to determine a first crack region between the first reconstructed sub-grid and a second reconstructed sub-grid.

2. The method of claim 1, wherein, The decoding end obtains a first reference point of a first reconstructed sub-grid, which comprises the following steps: The decoding end decodes first indication information; The first indication information is used to indicate the first reference point.

3. The method of claim 1 or 2, wherein, Before the decoding end determines the first crack information of the first reconstructed sub-grid based on the first reference point, the method further comprises the following steps: The decoding end merges a plurality of reconstructed sub-grids to obtain a merged grid; the plurality of reconstructed sub-grids comprise the first reconstructed sub-grid and the second reconstructed sub-grid; The decoding end determines the first crack information of the first reconstructed sub-grid based on the first reference point, which comprises the following steps: The decoding end traverses a boundary edge or a boundary point of the merged grid based on the first reference point until a non-boundary edge or a non-boundary point of the merged grid to obtain the first crack information.

4. The method of claim 3, wherein, The first reference point is a starting point of the traversal of the boundary or the boundary point of the merged grid.

5. The method of claim 3 or 4, wherein, The decoding end traverses a boundary edge or a boundary point of the merged grid based on the first reference point until a non-boundary edge or a non-boundary point of the merged grid to obtain the first crack information, which comprises the following steps: The decoding end decodes second indication information; The decoding end traverses a boundary edge or a boundary point of the first reconstructed sub-grid in the merged grid based on a traversal order indicated by the second indication information until a non-boundary edge or a non-boundary point of the merged grid to obtain the first crack information.

6. The method of claim 5, wherein, The second indication information comprises at least one of the following: An index of a point traversed after the first reference point; An identifier used to indicate a traversal direction; An identifier indicating a relationship between a point traversed after the first reference point and a neighboring point of the first reference point.

7. The method of any one of claims 1 to 6, wherein, Before the decoding end obtains the first reference point of the first reconstructed sub-grid, the method further comprises the following steps: The decoding end decodes third indication information; The third indication information is used to indicate a number of crack regions between the first reconstructed sub-grid and the second reconstructed sub-grid.

8. The method of any one of claims 1 to 7, wherein, The method further comprises the following steps: The decoding end decodes fourth indication information; The fourth indication information is used to indicate the second reconstructed sub-grid.

9. The method of any one of claims 1 to 8, wherein, The decoding end obtains a first reference point of a first reconstructed sub-grid, which comprises the following steps: The decoding end decodes fifth indication information; In a case where the fifth indication information indicates that the decoding end determines the first crack information based on the first reference point, the decoding end obtains the first reference point.

10. A trellis encoding method, wherein, The method comprises the following steps: An encoding end obtains at least one boundary point of a first reconstructed sub-grid generated based on sub-grid division; The encoding end determines a first reference point of the first reconstructed sub-grid based on the at least one boundary point; The first reference point is used to determine first crack information of the first reconstructed sub-grid, the first crack information includes a boundary edge or a boundary point of the first reconstructed sub-grid, and the first crack information is used to determine a first crack region between the first reconstructed sub-grid and a second reconstructed sub-grid.

11. The method of claim 10, wherein, The method further includes: The encoding end encodes first indication information. The first indication information is used to indicate the first reference point.

12. The method of claim 10 or 11, wherein, The first reference point is a starting point of a boundary or a boundary point of a merged grid, the merged grid is a grid obtained by merging a plurality of reconstructed sub-grids, and the plurality of reconstructed sub-grids include the first reconstructed sub-grid.

13. The method of any one of claims 10 to 12, wherein, The method further includes: The encoding end encodes second indication information. The second indication information is used to indicate a traversal order.

14. The method of claim 13, wherein, The second indication information includes at least one of the following: An index of a point traversed after the first reference point; An identifier used to indicate a traversal direction; An identifier indicating a relationship between a point traversed after the first reference point and a neighboring point of the first reference point.

15. The method of any one of claims 10 to 14, wherein, The method further includes: The encoding end encodes third indication information. The third indication information is used to indicate a number of crack regions between the first reconstructed sub-grid and the second reconstructed sub-grid.

16. The method of any one of claims 10 to 15, wherein, The method further includes: The encoding end encodes fourth indication information. The fourth indication information is used to indicate the second reconstructed sub-grid.

17. The method of any one of claims 10 to 16, wherein, The method further includes: The encoding end encodes fifth indication information. The fifth indication information indicates that the decoding end determines the first crack information based on the first reference point.

18. A trellis decoding apparatus, wherein, It includes: An acquisition module is configured to acquire a first reference point of a first reconstructed sub-grid; A processing module is configured to determine first crack information of the first reconstructed sub-grid based on the first reference point; The first crack information includes a boundary edge or a boundary point of the first reconstructed sub-grid, and the first crack information is used to determine a first crack region between the first reconstructed sub-grid and a second reconstructed sub-grid.

19. The apparatus of claim 18, wherein, The acquisition of the first reference point of the first reconstructed sub-grid includes: Decoding first indication information; The first indication information is used to indicate the first reference point.

20. The apparatus of claim 18 or 19, wherein, Before the processing module determines the first crack information of the first reconstructed sub-grid based on the first reference point, the processing module is further configured to: Merge a plurality of reconstructed sub-grids to obtain a merged grid; the plurality of reconstructed sub-grids include the first reconstructed sub-grid and the second reconstructed sub-grid; The determination of the first crack information of the first reconstructed sub-grid based on the first reference point includes: Traversing a boundary edge or a boundary point of the merged grid based on the first reference point until a non-boundary edge or a non-boundary point of the merged grid to obtain the first crack information.

21. The apparatus of claim 20, wherein, The first reference point is a starting point of a boundary or a boundary point of the merged grid.

22. The apparatus of claim 20 or 21, wherein, The traversal of the boundary edge or the boundary point of the merged grid based on the first reference point until the non-boundary edge or the non-boundary point of the merged grid to obtain the first crack information includes: Decoding second indication information; Based on the traversal order indicated by the second indication information, a boundary edge or a boundary point of the first reconstructed sub-grid in the merged grid is traversed until a non-boundary edge or a non-boundary point of the merged grid, to obtain the first crack information.

23. A trellis encoding device, wherein, The method comprises: obtaining at least one boundary point of a first reconstructed sub-grid based on sub-grid division; determining a first reference point of the first reconstructed sub-grid based on the at least one boundary point; wherein the first reference point is used to determine first crack information of the first reconstructed sub-grid, the first crack information comprising a boundary edge or a boundary point of the first reconstructed sub-grid, and the first crack information is used to determine a first crack region between the first reconstructed sub-grid and a second reconstructed sub-grid.

24. The apparatus of claim 23, wherein, The processing module is further configured to: encode first indication information; wherein the first indication information is used to indicate the first reference point.

25. The apparatus of claim 23 or 24, wherein, The first reference point is a starting point for traversing a boundary or a boundary point of a merged grid, the merged grid being a grid obtained by merging a plurality of reconstructed sub-grids, and the plurality of reconstructed sub-grids comprising the first reconstructed sub-grid.

26. The apparatus of any one of claims 23-25, wherein, The processing module is further configured to: encode second indication information; wherein the second indication information is used to indicate a traversal order.

27. The apparatus of claim 26, wherein, The second indication information comprises at least one of: an index of a point traversed after the first reference point; an identifier used to indicate a traversal direction; an identifier indicating a relationship between a point traversed after the first reference point and a neighboring point of the first reference point.

28. An electronic device, comprising: A processor and a memory are included, the memory stores programs or instructions executable on the processor, and the programs or instructions are executed by the processor to implement steps of the grid decoding method according to any one of claims 1 to 9, or implement steps of the grid encoding method according to any one of claims 10 to 17.

29. A readable storage medium, wherein, A readable storage medium stores programs or instructions executable by a processor, and the programs or instructions are executed by the processor to implement steps of the grid decoding method according to any one of claims 1 to 9, or implement steps of the grid encoding method according to any one of claims 10 to 17.

30. A chip, wherein, The chip includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is configured to run programs or instructions to implement steps of the grid decoding method according to any one of claims 1 to 9, or implement steps of the grid encoding method according to any one of claims 10 to 17.

Images