Grid coding method, grid decoding method and related equipment

Abstract

The invention discloses a grid coding method, a grid decoding method and related equipment, and belongs to the technical field of computers.The grid coding method comprises the steps that a basic grid corresponding to a three-dimensional grid is obtained, subdivision processing is carried out based on the basic grid, and a subdivided grid is obtained; performing deformation operation based on the subdivided grid to obtain a deformed grid; displacement information to be coded is selected from the displacement information corresponding to the subdivision grid, the displacement information corresponding to the subdivision grid is used for representing the displacement of the vertex of the subdivision grid relative to the deformation grid, and the hierarchy number of the displacement information to be coded is smaller than or equal to the hierarchy number of the displacement information corresponding to the subdivision grid; and encoding the to-be-encoded displacement information to obtain a first encoding result.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and specifically relates to a grid encoding method, a grid decoding method and related equipment. Background Art

[0002] With the rapid development of multimedia technology, 3D models have become a new generation of digital media after audio, images, and videos. 3D mesh and point cloud are two commonly used ways to represent 3D models. Compared with traditional multimedia such as images and videos, 3D mesh models have stronger interactivity and realism, and are widely used.

[0003] In the related art, when the displacement information of the subdivided grid obtained based on the three-dimensional grid is encoded at the encoding end, all the displacement information of the subdivided grid is encoded, and the amount of data required to be encoded is large, and the encoding efficiency is low. Summary of the invention

[0004] The embodiments of the present application provide a grid encoding method, a grid decoding method and related equipment, which can solve the problem of low encoding efficiency.

[0005] In a first aspect, a grid coding method is provided, comprising:

[0006] Obtaining a basic grid corresponding to the three-dimensional grid, and performing subdivision processing based on the basic grid to obtain a subdivided grid;

[0007] Performing a deformation operation based on the subdivided grid to obtain a deformed grid;

[0008] Selecting displacement information to be encoded from the displacement information corresponding to the subdivided grid, the displacement information corresponding to the subdivided grid is used to represent the displacement of the vertices of the subdivided grid relative to the deformed grid, and the number of levels of the displacement information to be encoded is less than or equal to the number of levels of the displacement information corresponding to the subdivided grid;

[0009] The displacement information to be encoded is encoded to obtain a first encoding result.

[0010] In a second aspect, a grid decoding method is provided, comprising:

[0011] Decoding a first encoding result in a code stream corresponding to the three-dimensional grid to obtain decoded displacement information;

[0012] Decoding the third encoding result in the code stream to obtain a basic grid, and performing subdivision processing on the basic grid to obtain a subdivided grid;

[0013] Performing padding processing on the decoded displacement information to obtain padded displacement information, wherein the number of levels of the padded displacement information is greater than or equal to the number of levels of the decoded displacement information;

[0014] Reconstruction processing is performed based on the filled displacement information and the subdivided grid to obtain a reconstructed grid.

[0015] In a third aspect, a grid coding device is provided, comprising:

[0016] An acquisition module is used to acquire a basic grid corresponding to the three-dimensional grid, and perform subdivision processing based on the basic grid to obtain a subdivided grid;

[0017] A deformation module, used for performing a deformation operation based on the subdivided grid to obtain a deformed grid;

[0018] a selection module, configured to select displacement information to be encoded from the displacement information corresponding to the subdivided grid, wherein the displacement information corresponding to the subdivided grid is used to represent the displacement of the vertices of the subdivided grid relative to the deformed grid, and the number of levels of the displacement information to be encoded is less than or equal to the number of levels of the displacement information corresponding to the subdivided grid;

[0019] The encoding module is used to encode the displacement information to be encoded to obtain a first encoding result.

[0020] In a fourth aspect, a grid decoding device is provided, comprising:

[0021] A first decoding module, used for decoding a first encoding result in a code stream corresponding to the three-dimensional grid to obtain decoded displacement information;

[0022] A second decoding module is used to decode the third encoding result in the code stream to obtain a basic grid, and subdivide the basic grid to obtain a subdivided grid;

[0023] A filling module, used for performing filling processing on the decoded displacement information to obtain filled displacement information, wherein the number of levels of the filled displacement information is greater than or equal to the number of levels of the decoded displacement information;

[0024] The reconstruction module is used to perform reconstruction processing based on the filled displacement information and the subdivided grid to obtain a reconstructed grid.

[0025] In a fifth aspect, a terminal is provided, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect; or, the program or instruction, when executed by the processor, implements the steps of the method described in the second aspect.

[0026] In a sixth aspect, a terminal is provided, comprising a processor and a communication interface, wherein the processor is used to: obtain a basic grid corresponding to a three-dimensional grid, perform subdivision processing based on the basic grid to obtain a subdivided grid; perform deformation operations based on the subdivided grid to obtain a deformed grid; select displacement information to be encoded from the displacement information corresponding to the subdivided grid, the displacement information corresponding to the subdivided grid is used to characterize the displacement of the vertices of the subdivided grid relative to the deformed grid, and the number of levels of the displacement information to be encoded is less than or equal to the number of levels of the displacement information corresponding to the subdivided grid; encode the displacement information to be encoded to obtain a first encoding result.

[0027] In the seventh aspect, a terminal is provided, comprising a processor and a communication interface, wherein the processor is used to: decode a first coding result in a code stream corresponding to a three-dimensional grid to obtain decoded displacement information; decode a third coding result in the code stream to obtain a basic grid, and subdivide the basic grid to obtain a subdivided grid; fill the decoded displacement information to obtain filled displacement information, wherein the number of levels of the filled displacement information is greater than or equal to the number of levels of the decoded displacement information; and reconstruct based on the filled displacement information and the subdivided grid to obtain a reconstructed grid.

[0028] In an eighth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the grid encoding method as described in the first aspect are implemented, or when the program or instruction is executed by a processor, the steps of the grid decoding method as described in the second aspect are implemented.

[0029] In the ninth aspect, a chip is provided, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instructions to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the second aspect.

[0030] In the tenth aspect, a computer program / program product is provided, wherein the computer program / program product is stored in a non-volatile storage medium, and the program / program product is executed by at least one processor to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the second aspect.

[0031] In the embodiment of the present application, a base mesh corresponding to a three-dimensional mesh is obtained, and a subdivision process is performed based on the base mesh to obtain a subdivided mesh; a deformation operation is performed based on the subdivided mesh to obtain a deformed mesh; displacement information to be encoded is selected from the displacement information corresponding to the subdivided mesh, the displacement information corresponding to the subdivided mesh is used to characterize the displacement of the vertices of the subdivided mesh relative to the deformed mesh, and the number of levels of the displacement information to be encoded is less than or equal to the number of levels of the displacement information corresponding to the subdivided mesh; the displacement information to be encoded is encoded to obtain a first encoding result. In this way, by encoding only some levels of displacement information in the displacement information corresponding to the subdivided mesh, the amount of encoded data can be reduced and the encoding efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of V3C encoding in the related art;

[0033] Figure 2 It is a schematic diagram of V3C decoding in the related art;

[0034] Figure 3 is a schematic diagram of V-DMC coding in the related art;

[0035] Figure 4 is a schematic diagram of V-DMC decoding in the related art;

[0036] Figure 5 is a flow chart of a grid coding method provided in an embodiment of the present application;

[0037] Figure 6 It is one of the displacement coding schematic diagrams provided in the embodiment of the present application;

[0038] Figure 7 This is one of the displacement decoding schematic diagrams provided in the embodiment of the present application;

[0039] Figure 8 is a flow chart of a grid decoding method provided in an embodiment of the present application;

[0040] Fig. 9 This is the second displacement coding schematic diagram provided in the embodiment of the present application;

[0041] Fig.10 This is the second displacement decoding schematic diagram provided in the embodiment of the present application;

[0042] Fig.11 This is a third displacement decoding schematic diagram provided in an embodiment of the present application;

[0043] Fig.12 This is a third displacement coding schematic diagram provided in an embodiment of the present application;

[0044] Fig.13 This is a fourth displacement decoding schematic diagram provided in an embodiment of the present application;

[0045] Fig.14 This is a schematic diagram of a grid coding process provided by an embodiment of the present application;

[0046] Fig.15 It is a simplified schematic diagram of a grid provided in an embodiment of the present application;

[0047] Fig.16 is an example diagram of displacement calculation provided in an embodiment of the present application;

[0048] Fig.17 is a basic grid compression schematic diagram provided in an embodiment of the present application;

[0049] Fig.18 This is a schematic diagram of attribute graph conversion provided by an embodiment of the present application;

[0050] Fig.19 This is a schematic diagram of a grid decoding process provided by an embodiment of the present application;

[0051] Fig. 20 is a structural schematic diagram of a grid coding device provided in an embodiment of the present application;

[0052] Fig.21 It is a structural schematic diagram of a grid decoding device provided in an embodiment of the present application;

[0053] Fig. 22 It is a structural diagram of a communication device provided in an embodiment of the present application;

[0054] Fig.23 It is a structural diagram of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

[0056] The terms "first", "second", etc. of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of one type, and the number of objects is not limited, for example, the first object can be one or more. In addition, "or" in the present application represents at least one of the connected objects. For example, "A or B" covers three schemes, namely, Scheme 1: including A but not including B; Scheme 2: including B but not including A; Scheme 3: including both A and B. The character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0057] The term "indication" in this application can be a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, operations to be performed, or request results in the sent indication; an indirect indication can be understood as the receiver determining the corresponding information according to the indication sent by the sender, or making a judgment and determining the operation to be performed or the request result according to the judgment result.

[0058] The encoding and decoding end corresponding to the encoding and decoding method in the embodiment of the present application may be a terminal, which may also be referred to as a terminal device or a user terminal (User Equipment, UE). The terminal may be a mobile phone, a tablet computer (Tablet Personal Computer), a laptop computer (Laptop Computer) or a notebook computer, a personal digital assistant (Personal Digital Assistant, PDA), a handheld computer, a netbook, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device (Wearable Device) or a vehicle-mounted device (Vehicle User Equipment, VUE), a pedestrian terminal (Pedestrian User Equipment, PUE) and other terminal side devices, and the wearable device includes: a smart watch, a bracelet, a headset, glasses, etc. It should be noted that the specific type of the terminal is not limited in the embodiment of the present application.

[0059] For ease of understanding, some contents involved in the embodiments of the present application are described below:

[0060] 1. 3D Grid

[0061] In recent years, with the rapid development of multimedia technology, related research results have been rapidly industrialized and have become an indispensable part of people's lives. Three-dimensional models have become a new generation of digital media after audio, images, and videos. Three-dimensional grids are a commonly used way to represent three-dimensional models. Compared with traditional multimedia such as images and videos, three-dimensional grid models have stronger interactivity and realism, making them more and more widely used in various fields such as commerce, manufacturing, construction, education, medicine, entertainment, art, and military.

[0062] Although there are many ways to represent 3D meshes, triangular meshes are still the most common representation method. A 3D mesh can be considered to be composed of three basic elements: vertices, edges, and faces. Vertices are the most basic elements in a mesh, and they define a position in a 3D space. Edges are line segments connecting two vertices in a mesh. Faces can be considered as polygons formed by closed paths of edges. For a triangular mesh, each face is a triangle.

[0063] The information contained in the mesh is usually divided into three categories: geometric information, connection information, and attribute information. Geometric information refers to the position of each vertex of the mesh in three-dimensional space. Connection information describes the association between elements in the mesh, that is, the connection relationship between vertices. Attribute information is optional, and it can associate attributes to corresponding mesh elements (such as vertex color, normal vector, etc. can be associated with mesh vertices). Mesh parameterization can also be used to map the mesh from three-dimensional space to a two-dimensional plane area. This mapping relationship is usually described by a set of parameter coordinates, called UV coordinates or texture coordinates, which are associated with mesh vertices. This two-dimensional mapping can be used to represent high-resolution attribute information, such as textures, normal vectors, etc.

[0064] In almost all application fields that use 3D meshes (such as computational simulation, entertainment, medical imaging, digital cultural relics, computer design, e-commerce, etc.), as people's demand for higher visual effects of 3D mesh models increases, the models are becoming more and more complex, and the accuracy of the models is also increasing. Therefore, the amount of data required to represent the 3D mesh is also increasing accordingly. The above problems have led to the increasing complexity of the processing, visualization, transmission and storage of 3D meshes. 3D mesh compression can be regarded as a way to solve the above problems. It reduces the size of model data and is conducive to the processing, storage and transmission of 3D meshes. Therefore, it is necessary to propose an efficient and general 3D mesh compression algorithm.

[0065] Recently, the Moving Picture Expert Group (MPEG) of the International Organization for Standardization in the field of audio and video coding compression has begun to develop a compression standard for three-dimensional meshes, Video-based dynamic mesh coding (V-DMC or VDMC). This standard is specified based on the existing Visual Volumetric Video-based Coding (V3C) standard. The V3C standard provides a general method for compressing three-dimensional models, which can be presented in the form of point clouds, meshes or panoramic videos. Making the compression method of three-dimensional mesh models compatible with this standard is conducive to the promotion and applicability of this method. Therefore, it is of great significance to optimize the three-dimensional mesh encoding and decoding method in VDMC and combine the optimization method with the V3C standard. One possible optimization method is to optimize the displacement coding method. In the existing framework, the displacement is obtained by calculating the distance between the reconstructed mesh and the original mesh vertex, and its purpose is to improve the quality of the mesh. In the existing framework, the displacement is encoded using a video encoder, and providing multiple options for the encoding method of the displacement is conducive to improving the encoding performance.

[0066] 2. V3C Standard

[0067] The V3C standard provides a method for encoding and decoding various three-dimensional media through video or image coding technology. Specifically, it converts the three-dimensional media content from a three-dimensional representation to multiple two-dimensional representations (called V3C components) through projection and other methods before encoding, and then encodes the two-dimensional representation using existing video or image coding technology. V3C components mainly include occupancy components, geometric components, and attribute components. The occupancy component can indicate which areas in the two-dimensional representation are associated with the data in the three-dimensional representation; the geometric component represents information related to the position of the three-dimensional data in space, and the attribute component can provide attribute information corresponding to the vertex, such as material, texture, etc. In addition, the components also contain information on how to reconstruct the three-dimensional model through these components, which is called atlas information. An example diagram of the V3C standard is shown below. Figure 1 and Figure 2 shown.

[0068] The atlas information is used to associate all components, and additional information for reconstructing from 2D back to 3D is also included in the atlas components. The atlas consists of multiple basic units, called patches. Each patch represents an area in the available 2D components and contains the information needed to project the area back to 3D space.

[0069] 3. V-DMC

[0070] V-DMC is a standard developed by MPEG for compressing 3D meshes. Its main idea is to compress 3D meshes by using the existing V3C standard. Since 3D meshes have connection information that needs to be encoded, its specific encoding process is slightly different from V3C. It is necessary to expand the syntax and semantics of the V3C standard decoding end and the decoding operation to support the decoding and reconstruction of 3D meshes. Figure 3 and Figure 4 It is the current coding and decoding framework of V-DMC.

[0071] The overall framework of the encoding end is as follows Figure 3 As shown in the figure, for the input mesh, it is first simplified by the simplification module, and then the new texture coordinates are generated for the mesh through mesh parameterization. Then the parameterized mesh is subdivided and deformed, that is, new vertices are inserted on the mesh according to a specific subdivision method and the distance from the vertex of the subdivided mesh to the nearest neighbor of the input mesh is calculated, which is called displacement information. Subsequently, the mesh before subdivision, called the base mesh, is sent to the base mesh encoding module and compressed using the existing mesh encoder. In the inter-frame mode, the base mesh encoding module can generate a motion vector for each vertex of the base mesh according to the reference frame, and then only compress the motion vector.

[0072] The base mesh is reconstructed after encoding, and then the order of displacement is adjusted according to the vertex order of the reconstructed base mesh. Subsequently, the vertex displacement information after the order is adjusted is subjected to wavelet transformation, the transformed coefficients are quantized, and then the quantized coefficients are arranged into a two-dimensional image according to a specific scanning order, and the two-dimensional image is encoded using a video encoder. Then, the reconstructed displacement information is applied to the subdivided base mesh to obtain a reconstructed subdivided deformed mesh, and the mesh, the original input mesh and its corresponding texture map are input into the corresponding texture map conversion module to obtain the texture map corresponding to the reconstructed mesh, and the texture map is also encoded using a video encoder. The parameters used in the encoding process, such as the type of video encoder used, the type of mesh encoder, the transformation parameters, the quantization parameters, etc., are transmitted to the decoding end through auxiliary information.

[0073] The overall framework of the decoding end is as follows Figure 4As shown, for the received code stream, the decoding end first demultiplexes each part of the code stream to obtain the basic grid code stream, the displacement video code stream, the texture map video code stream and the auxiliary information code stream respectively. For the basic grid code stream, the grid decoder indicated by the auxiliary information is used to decode the basic grid. The displacement video code stream and the texture map video code stream are decoded by the video decoder. For the displacement part, after the video is decoded, the displacement needs to be taken out from the image through the displacement decoding module, and the steps such as inverse quantization and inverse transformation are performed, and then it is applied to the subdivided basic grid to obtain the deformed grid reconstructed by the decoding end. After decoding, the texture map is the texture map corresponding to the reconstructed deformed grid. The subsequent application or rendering module processes the reconstructed deformed grid and the decoded texture map as input.

[0074] The grid encoding method, grid decoding method and related equipment provided by the embodiments of the present application are described in detail below through some embodiments and their application scenarios in combination with the accompanying drawings.

[0075] See also Figure 5 , Figure 5 is a flowchart of a grid coding method provided in an embodiment of the present application, which can be applied to a coding end device, such as Figure 5 As shown, the grid coding method includes the following steps:

[0076] Step 101: Obtain a basic grid corresponding to a three-dimensional grid, and perform subdivision processing based on the basic grid to obtain a subdivided grid.

[0077] Among them, the three-dimensional grid can be mesh simplified to obtain the basic grid corresponding to the three-dimensional grid; or, the three-dimensional grid can be mesh simplified and mesh parameterized to obtain the basic grid corresponding to the three-dimensional grid; or, the basic grid corresponding to the three-dimensional grid can be obtained by other methods; and so on, this embodiment does not limit this.

[0078] In addition, the basic mesh may be subdivided to obtain a subdivided mesh. The subdivided process may be to add vertices to the edges constituting the basic mesh, and the basic mesh after adding the vertices is a subdivided mesh.

[0079] In one implementation, the grid may be represented in the form of a grid sequence, and processing of the grid may be considered as processing of the grid sequence.

[0080] Step 102: Perform a deformation operation based on the subdivided grid to obtain a deformed grid.

[0081] The deformation operation may be to deform the subdivided mesh into a mesh with the same shape as the three-dimensional mesh, and the mesh obtained after the deformation is the deformed mesh.

[0082] Step 103: Select displacement information to be encoded from the displacement information corresponding to the subdivided grid, wherein the displacement information corresponding to the subdivided grid is used to characterize the displacement of the vertices of the subdivided grid relative to the deformed grid, and the number of levels of the displacement information to be encoded is less than or equal to the number of levels of the displacement information corresponding to the subdivided grid.

[0083] The displacement information corresponding to the subdivided mesh can be used to characterize the displacement of the vertices of the subdivided mesh relative to the vertices of the deformed mesh. The displacement information can also be described as displacement data. For example, for each vertex of the subdivided mesh, a corresponding vertex can be found on the deformed mesh, and the displacement information corresponding to the subdivided mesh includes displacement information for characterizing the displacement of the vertex of each subdivided mesh relative to the corresponding vertex on the deformed mesh.

[0084] It should be noted that, due to the subdivision process, the displacement information includes multiple levels, and the more times the base grid is subdivided, the more levels of displacement information there are. The displacement information corresponding to the subdivided grid may include multiple levels of displacement information.

[0085] Among them, the displacement information corresponding to the subdivided grid may include displacement components of at least one dimension, and for the displacement components of each dimension of the at least one dimensional displacement components, the displacement information to be encoded corresponding to each dimension can be selected respectively, and for the displacement components of each dimension, the number of levels of the displacement information to be encoded corresponding to each dimension selected can be the same or different.

[0086] In one implementation, the selecting the displacement information to be encoded from the displacement information corresponding to the subdivided grid may include: determining the displacement information corresponding to the subdivided grid, the displacement information corresponding to the subdivided grid including displacement components of at least one dimension, and selecting the displacement information to be encoded corresponding to each dimension for the displacement components of each dimension of the at least one dimensional displacement components, respectively, the number of levels of the displacement information to be encoded corresponding to each dimension being less than or equal to the number of levels of the displacement components of each dimension.

[0087] For example, the displacement information corresponding to the subdivided grid may include: displacement components of the first dimension of N levels, displacement components of the second dimension of the N levels, and displacement components of the third dimension of the N levels. The displacement information to be encoded corresponding to the first dimension may include: displacement components of the first dimension of M1 levels among the displacement components of the first dimension of the N levels; the displacement information to be encoded corresponding to the second dimension may include: displacement components of the second dimension of M2 levels among the displacement components of the second dimension of the N levels; the displacement information to be encoded corresponding to the third dimension may include: displacement components of the third dimension of M3 levels among the displacement components of the third dimension of the N levels, N is a positive integer, M1, M2 and M3 are all integers greater than or equal to 0, and M1, M2 and M3 are all less than or equal to N.

[0088] It should be noted that the first dimension may be a first coordinate axis dimension, such as an x-axis dimension; the second dimension may be a second coordinate axis dimension, such as a y-axis dimension; and the third dimension may be a third coordinate axis dimension, such as a z-axis dimension.

[0089] Step 104: Encode the displacement information to be encoded to obtain a first encoding result.

[0090] The encoding of the displacement information to be encoded to obtain the first encoding result may be encoding the selected displacement information to be encoded to obtain the first encoding result.

[0091] In addition, the encoding of the displacement information to be encoded may be video encoding of the displacement information to be encoded, or may be entropy encoding of the displacement information to be encoded, etc. This embodiment does not limit the specific encoding method for encoding the displacement information to be encoded.

[0092] In the related art, the same level is used for the displacement components of each dimension, but the actual displacement data is not necessarily evenly distributed in each dimension. In this case, if the same level of subdivision is used for each dimension, bits for encoding the displacement data will be wasted. In this embodiment, it is allowed to encode only part of the level data of the displacement components of each dimension, and then the displacement data recovery operation is performed on each dimension separately at the decoding end. For each displacement dimension i, only the first n i (n i >=0) levels of data are encoded, and Nn i At the decoding end, for each dimension i of the decoded displacement data, the discarded Nn can be replaced by zero padding, for example. i The data of each level is filled in, corresponding one by one to the mesh vertices on the decoding end.

[0093] In the embodiment of the present application, a base mesh corresponding to a three-dimensional mesh is obtained, and a subdivision process is performed based on the base mesh to obtain a subdivided mesh; a deformation operation is performed based on the subdivided mesh to obtain a deformed mesh; displacement information to be encoded is selected from the displacement information corresponding to the subdivided mesh, the displacement information corresponding to the subdivided mesh is used to characterize the displacement of the vertices of the subdivided mesh relative to the deformed mesh, and the number of levels of the displacement information to be encoded is less than or equal to the number of levels of the displacement information corresponding to the subdivided mesh; the displacement information to be encoded is encoded to obtain a first encoding result. In this way, by encoding only some levels of displacement information in the displacement information corresponding to the subdivided mesh, the amount of encoded data can be reduced and the encoding efficiency can be improved.

[0094] Optionally, the displacement information corresponding to the subdivided grid includes displacement information of multiple levels, and the selecting the displacement information to be encoded from the displacement information corresponding to the subdivided grid includes:

[0095] Determine a target level among the multiple levels, wherein the target level is determined based on displacement information representing a displacement of a first preset value among the displacement information of the multiple levels;

[0096] The displacement information to be encoded is selected from the displacement information corresponding to the subdivided grid based on the target level.

[0097] The first preset value may be 0.

[0098] The displacement information may include a displacement component of at least one dimension, and for the displacement component of each dimension, a target level corresponding to each dimension may be determined respectively.

[0099] In one embodiment, the displacement components of at least one dimension include at least the displacement components of the target dimension, the displacement information corresponding to the subdivided grid includes at least the displacement components of the target dimension of multiple levels, and the number of displacement components representing the displacement of the first preset value in the displacement components of the target dimension of the target level meets a preset condition. The selecting the displacement information to be encoded from the displacement information corresponding to the subdivided grid based on the target level may be selecting the displacement information to be encoded corresponding to the target dimension from the displacement components of the target dimension of the multiple levels based on the target level. By way of example, the target dimension may be the first dimension, the second dimension, or the third dimension.

[0100] In addition, the target level is determined based on the displacement information of the multiple levels whose characterizing displacement as the first preset value, and may include: the ratio of the number of displacement information of the target level whose characterizing displacement as the first preset value to the total number of displacement information of the target level satisfies a preset condition; or, may include: the difference between the number of displacement information of the target level whose characterizing displacement as the first preset value and the total number of displacement information of the target level satisfies a preset condition; or, may include: the ratio of the number of displacement information of the target level whose characterizing displacement as the first preset value to the number of displacement information of the target level whose characterizing displacement as not the first preset value satisfies the preset condition; or, may include: the difference between the number of displacement information of the target level whose characterizing displacement as the first preset value and the number of displacement information of the target level whose characterizing displacement as not the first preset value satisfies the preset condition; and so on. This embodiment does not limit this.

[0101] It should be noted that, since the sum of the number of displacement information whose displacement is the first preset value and the number of displacement information whose displacement is not the first preset value is the total number of displacement information, that is, the number of displacement information whose displacement is the first preset value and the number of displacement information whose displacement is not the first preset value have a corresponding relationship, wherein the target level is determined based on the displacement information of the multiple levels whose displacement is characterized by the first preset value, which can also be described as: the target level is determined based on the displacement information of the multiple levels whose displacement is characterized by the displacement not being the first preset value. The target level is determined based on the displacement information of the multiple levels whose displacement is characterized by the displacement not being the first preset value, which can include: the ratio of the number of displacement information of the target level whose displacement is not the first preset value to the total number of displacement information of the target level satisfies the preset condition; or the difference between the number of displacement information of the target level whose displacement is not the first preset value and the total number of displacement information of the target level satisfies the preset condition; etc., which are not limited in this embodiment.

[0102] It should be noted that the target level can be determined starting from the first level among the multiple levels. When the target level is not the first level, the displacement information to be encoded includes the displacement information from the first level to the level before the target level; or, the target level can be determined starting from the last level among the multiple levels, and the displacement information to be encoded includes the displacement information from the first level among the multiple levels to the target level.

[0103] In this implementation, a target level among the multiple levels is determined, wherein the target level is determined based on displacement information representing a displacement of a first preset value among the displacement information of the multiple levels; and displacement information to be encoded is selected from the displacement information corresponding to the subdivided grid based on the target level. In this way, the displacement information to be encoded can be determined from the displacement information of multiple levels based on the number of displacement information representing a displacement of the first preset value, and the displacement information of the level carrying a larger amount of information can be selected for encoding, thereby effectively reducing the amount of encoded data and improving encoding efficiency.

[0104] Optionally, a ratio of the number of displacement information representing a displacement of a first preset value in the displacement information of the target level to the total number of displacement information of the target level meets a preset condition.

[0105] The target level corresponding to the displacement component of each dimension may be determined respectively for the displacement component of each dimension, thereby supporting the selection of displacement information of different levels for encoding for displacement components of different dimensions.

[0106] In one embodiment, for the target dimension, the ratio of the number of displacement components representing a displacement of a first preset value in the displacement components of the target dimension of the target level to the total number of displacement components of the target dimension of the target level meets a preset condition.

[0107] In this implementation, the ratio of the number of displacement information representing the displacement of the first preset value in the displacement information of the target layer to the total number of displacement information of the target layer meets the preset conditions, thereby supporting the selection of layers with a relatively large proportion of non-zero displacement information for displacement encoding and discarding layers with a relatively small proportion of non-zero displacement information, which can effectively reduce the amount of encoded data and improve encoding efficiency.

[0108] Optionally, determining a target level among the multiple levels includes:

[0109] Determining a target level from a first level among the multiple levels, wherein a ratio of the number of displacement information representing a displacement of a first preset value in the displacement information of the target level to the total number of displacement information of the target level is greater than the first preset ratio;

[0110] In a case where the target level is not the first level, the displacement information to be encoded includes displacement information from the first level to a level before the target level.

[0111] The first preset value may be 0. The first preset ratio may be 80%, 85%, 90%, 95%, etc., and this embodiment does not limit the first preset ratio. In addition, when the target level is the first level, the displacement information to be encoded may be empty.

[0112] In addition, the target level corresponding to the displacement component of each dimension can be determined for each displacement component of each dimension, so as to support the selection of displacement information of different levels for encoding for displacement components of different dimensions. It should be noted that in the process of determining the target level corresponding to the displacement components of each dimension, the displacement components of each dimension can be set with a corresponding first preset ratio, and the first preset ratios corresponding to the displacement components of different dimensions can be the same or different.

[0113] In one embodiment, the ratio of the number of displacement components representing a displacement of a first preset value in the displacement components of the target dimension of the target level to the total number of displacement components of the target dimension of the target level is greater than the first preset ratio; when the target level is not the first level, the displacement information to be encoded corresponding to the target dimension includes the displacement components of the target dimension from the first level to the level before the target level.

[0114] It should be noted that the ratio of the number of displacement information of the target level that represents displacement as the first preset value to the total number of displacement information of the target level is greater than the first preset ratio. It can also be described as: the ratio of the number of displacement information of the target level that represents displacement not as the first preset value to the total number of displacement information of the target level is less than or equal to the first preset ratio.

[0115] For example, starting from the first level of displacement information, the number k of non-zero displacement components of the target dimension in the level can be counted, and the proportion of k in the level can be calculated. If the proportion is less than or equal to the first preset ratio, the displacement components of the target dimension of the level and all levels after the level are removed, and only the displacement components of the target dimension before the level are encoded; if the proportion does not meet the requirement of being less than or equal to the first preset ratio, the same statistical calculation is performed on the next level until a level that meets the conditions is found or all subdivided levels are traversed.

[0116] In this implementation, the target level is determined starting from the first level among the multiple levels, wherein the ratio of the number of displacement information representing the displacement as the first preset value in the displacement information of the target level to the total number of displacement information of the target level is greater than the first preset ratio; when the target level is not the first level, the displacement information to be encoded includes the displacement information of the level before the first level to the target level. In this way, the level selection can be performed starting from the first level, and the displacement information of the target level and subsequent levels carrying less information can be discarded, which can effectively reduce the amount of encoded data and improve the encoding efficiency.

[0117] Optionally, determining a target level among the multiple levels includes:

[0118] Determining a target level from the last level of the multiple levels, wherein a ratio of the number of displacement information representing a displacement of a first preset value in the displacement information of the target level to the total number of displacement information of the target level is less than or equal to a second preset ratio;

[0119] The displacement information to be encoded includes displacement information from a first level among the multiple levels to the target level.

[0120] The first preset value may be 0. The second preset ratio may be 5%, 8%, 10%, 15%, etc. This embodiment does not limit the second preset ratio.

[0121] In addition, the target level corresponding to the displacement component of each dimension can be determined for each displacement component of each dimension, so as to support the selection of displacement information of different levels for encoding for displacement components of different dimensions. It should be noted that in the process of determining the target level corresponding to the displacement component of each dimension, the displacement component of each dimension can be set with a corresponding second preset ratio, and the second preset ratios corresponding to the displacement components of different dimensions can be the same or different.

[0122] In one embodiment, the ratio of the number of displacement components representing a displacement of a first preset value in the displacement components of the target dimension of the target level to the total number of displacement components of the target dimension of the target level is less than or equal to a second preset ratio.

[0123] It should be noted that the ratio of the number of displacement information of the target level that represents displacement of the first preset value to the total number of displacement information of the target level is less than or equal to the second preset ratio. It can also be described as: the ratio of the number of displacement information of the target level that represents displacement not of the first preset value to the total number of displacement information of the target level is greater than the second preset ratio.

[0124] For example, the displacement component of the target dimension is statistically calculated from the last level forward, the number k of non-zero displacement components of the target dimension in the level is counted, and the proportion of k in the level is calculated. If the proportion of the last level is greater than the second preset ratio, the forward traversal is stopped. If the proportion of the last level meets the requirements (less than or equal to the second preset ratio), the displacement component of the target dimension of the level is removed, and the same statistical calculation and proportion judgment are continued for the previous level until a level that does not meet the requirements is encountered or all subdivided levels are traversed.

[0125] In this implementation, the target level is determined starting from the last level of the multiple levels, wherein the ratio of the number of displacement information representing the displacement as the first preset value in the displacement information of the target level to the total number of displacement information of the target level is less than or equal to the second preset ratio; the displacement information to be encoded includes the displacement information from the first level of the multiple levels to the target level. In this way, the level selection can be started from the last level, and the displacement information of the levels after the target level that carry less information is discarded, which can effectively reduce the amount of encoded data and improve the encoding efficiency.

[0126] Optionally, the displacement information corresponding to the subdivided grid includes: displacement components of a first dimension of N levels, displacement components of a second dimension of the N levels, and displacement components of a third dimension of the N levels;

[0127] The displacement information to be encoded includes: the displacement components of the first dimension of M1 levels among the displacement components of the first dimension of the N levels, the displacement components of the second dimension of M2 levels among the displacement components of the second dimension of the N levels, and the displacement components of the third dimension of M3 levels among the displacement components of the third dimension of the N levels, N is a positive integer, M1, M2 and M3 are all integers greater than or equal to 0, and M1, M2 and M3 are all less than or equal to N.

[0128] The selected displacement information to be encoded includes: the displacement components of the first dimension of the M1 levels, the displacement components of the second dimension of the M2 levels, and the displacement components of the third dimension of the M3 levels.

[0129] In this implementation, the displacement information to be encoded includes the displacement components of the first dimension of M1 levels among the displacement components of the first dimension of the N levels, the displacement components of the second dimension of M2 levels among the displacement components of the second dimension of the N levels, and the displacement components of the third dimension of M3 levels among the displacement components of the third dimension of the N levels, thereby supporting the selection of displacement components of different levels for displacement components of each dimensionality for displacement encoding, which can improve the flexibility of displacement encoding while improving the encoding efficiency.

[0130] Optionally, the code stream corresponding to the three-dimensional grid includes the first encoding result and the second encoding result, and the second encoding result includes the encoding result of the first flag bit;

[0131] When the first flag bit is a second preset value, the second encoding result also includes an encoding result of the first parameter; or,

[0132] When the first flag bit is a third preset value, the second encoding result also includes an encoding result of the first parameter, an encoding result of the second parameter, and an encoding result of the third parameter.

[0133] The first parameter is used to characterize the M1 levels, the second parameter is used to characterize the M2 levels, and the third parameter is used to characterize the M3 levels.

[0134] In addition, the second preset value and the third preset value are different values. For example, the second preset value may be 0, and the third preset value may be 1; or, the second preset value may be 1, and the third preset value may be 0. This embodiment does not limit the second preset value and the third preset value.

[0135] It should be noted that the displacement component of the first dimension may be a displacement component of higher importance. In some scenarios, it may be allowed to encode only the displacement component of the first dimension. Through the encoding result of the first flag bit carried in the bitstream, the decoding end can determine the dimension of the encoded displacement component. For example, the displacement component of the first dimension may be a normal component of the displacement.

[0136] In this implementation, when the first flag is the second preset value, the second encoding result also includes the encoding result of the first parameter; when the first flag is the third preset value, the second encoding result also includes the encoding result of the first parameter, the encoding result of the second parameter, and the encoding result of the third parameter. Thus, the decoding end can determine the displacement component of the encoding through the first flag; and through the parameter information included in the second encoding result, the decoding end can determine the level of the displacement information selected by the encoding.

[0137] Optionally, encoding the displacement information to be encoded to obtain a first encoding result includes:

[0138] Packing the displacement information to be encoded into a video frame;

[0139] Performing padding processing on the arranged video frames to obtain padded video frames;

[0140] The padded video frame is encoded to obtain a first encoding result.

[0141] The arranged video frames may be filled based on a preset video filling value, which may be a predetermined value known to the decoding end.

[0142] In this implementation, the displacement information to be encoded is arranged in a video frame; the arranged video frame is padded to obtain a padded video frame; the padded video frame is encoded to obtain a first encoding result. In this way, when the displacement information is encoded using a video encoding method, the video padding process can make the resolution of each frame in the video the same, and the pixel point data can be arranged in a rectangle.

[0143] Optionally, encoding the displacement information to be encoded to obtain a first encoding result includes:

[0144] Perform entropy coding on the displacement information to be encoded to obtain a first coding result.

[0145] In this implementation, entropy coding is performed on the displacement information to be encoded to obtain a first coding result, and the encoding of the displacement information can be achieved through entropy coding.

[0146] As a specific embodiment, Figure 6 and Figure 7 As shown, an embodiment of the present application proposes a new encoding and decoding method for dynamic mesh displacement, which allows encoding of only part of the hierarchical data of each dimensional component of the displacement, and then performing displacement data recovery operations on each dimension separately at the decoding end. Displacement is an important part of the dynamic mesh encoding and decoding process, which stores the distance information from the vertices of the subdivided mesh obtained after the base mesh is subdivided to the nearest neighbor point of the original input mesh in the normal direction. It is a three-dimensional data, and the x, y, and z components can be used to represent its three-dimensional data respectively. Due to the existence of the subdivision operation, the displacement has multiple subdivision levels and a base level. The base level comes first, and the subdivision level comes later. The more times the base mesh is subdivided, the more subdivision levels of the displacement. Assuming that the displacement has N (N>=1) levels, for each dimension i of the displacement, only the first n levels can be encoded at the encoding end. i (n i>=0) levels of data are encoded, and Nn i The decoder uses a method such as zero padding to replace the discarded Nn with data of each dimension i of the decoded displacement data. i The data of each level is filled in, corresponding one by one to the mesh vertices on the decoding end.

[0147] like Figure 6 As shown, at the encoding end, the displacement data is first input into the pre-processing module, which includes all operations before encoding except data selection, such as transformation, quantization, etc.; then each dimensional component of the pre-processed displacement data is input into the corresponding level selection module, and these modules will output the corresponding possible x, y, z component labeling data according to different selection methods during the selection process; finally, the selected displacement level data is input into the encoding module for encoding to obtain the displacement code stream. There are many specific encoding methods for this module, such as video encoding, entropy encoding, etc. Figure 7 As shown, at the decoding end, the displacement code stream is decoded by a decoding method corresponding to the encoding method; then each dimensional component of the decoded data is input into the corresponding data filling module, and these modules fill each dimensional component of the decoded partial displacement data into data with complete length according to the x, y, and z component marking data that may be received, and finally restore the three-dimensional displacement data with complete length; finally, the filled displacement data is input into the post-processing module to obtain the displacement data. The operation of the post-processing module is the reverse implementation of the operation of the pre-processing module at the encoding end, such as inverse quantization and inverse transformation.

[0148] See also Figure 8 , Figure 8 is a flowchart of a grid decoding method provided by an embodiment of the present application, which can be applied to a decoding end device, such as Figure 8 As shown, the grid decoding method includes the following steps:

[0149] Step 201: Decode a first encoding result in a code stream corresponding to a three-dimensional grid to obtain decoded displacement information;

[0150] Step 202: Decode the third encoding result in the code stream to obtain a basic grid, and subdivide the basic grid to obtain a subdivided grid;

[0151] Step 203: performing padding processing on the decoded displacement information to obtain padded displacement information, wherein the number of levels of the padded displacement information is greater than or equal to the number of levels of the decoded displacement information;

[0152] Step 204: Reconstruct the grid based on the filled displacement information and the subdivided grid to obtain a reconstructed grid.

[0153] Optionally, decoding the first encoding result in the code stream corresponding to the three-dimensional grid to obtain decoded displacement information includes:

[0154] Decoding a first encoding result in a code stream corresponding to the three-dimensional grid to obtain a video frame;

[0155] Decoded displacement information is obtained from the video frame, where the decoded displacement information is displacement information in the video frame excluding a preset video filling value.

[0156] Optionally, the method further comprises:

[0157] Decoding the second encoding result in the code stream to obtain a target parameter, where the target parameter is used to represent the number of levels of displacement information;

[0158] The decoding of the first encoding result in the code stream corresponding to the three-dimensional grid to obtain the decoded displacement information includes:

[0159] Decoding a first encoding result in a code stream corresponding to the three-dimensional grid to obtain a video frame;

[0160] Decoded displacement information is obtained from the video frame based on the target parameter.

[0161] Optionally, decoding the first encoding result in the code stream corresponding to the three-dimensional grid to obtain decoded displacement information includes:

[0162] The first encoding result in the code stream corresponding to the three-dimensional grid is entropy decoded to obtain decoded displacement information.

[0163] Optionally, performing padding processing on the decoded displacement information to obtain the padded displacement information includes:

[0164] The decoded displacement information is filled based on the number of vertices of the subdivided mesh to obtain filled displacement information.

[0165] Optionally, the filled displacement information includes N levels of displacement components of the first dimension, N levels of displacement components of the second dimension, and N levels of displacement components of the third dimension;

[0166] The decoded displacement information includes the displacement components of the first dimension of M1 levels among the displacement components of the first dimension of the N levels, the displacement components of the second dimension of M2 levels among the displacement components of the second dimension of the N levels, and the displacement components of the third dimension of M3 levels among the displacement components of the third dimension of the N levels, N is a positive integer, M1, M2 and M3 are all integers greater than or equal to 0, and M1, M2 and M3 are all less than or equal to N.

[0167] Optionally, the method further comprises:

[0168] Decoding the second encoding result in the code stream to obtain a first flag bit and a target parameter;

[0169] Wherein, when the first flag bit is a second preset value, the target parameter includes the first parameter; or,

[0170] When the first flag bit is a third preset value, the target parameter includes a first parameter, a second parameter and a third parameter.

[0171] The first parameter is used to characterize the M1 levels, the second parameter is used to characterize the M2 levels, and the third parameter is used to characterize the M3 levels.

[0172] The mesh encoding and decoding method is described below through several specific embodiments:

[0173] It should be noted that for different displacement coding and decoding methods, the processing methods of the encoding end and the decoding end have different implementation processes. The displacement coding and decoding process will be described below according to different coding and decoding methods.

[0174] Embodiment 1:

[0175] In this embodiment, the displacement information is encoded and decoded based on a video encoding and decoding method.

[0176] Encoding side:

[0177] like Fig. 9 As shown, the displacement data is input into the pre-processing module, which performs a series of pre-processing operations on the original displacement data, such as transformation and quantization; then, each dimensional component of the pre-processed displacement data is input into the corresponding level selection module, and the level selection module selects the level by judging whether the proportion of the number of non-zero corresponding components of the displacement in the level is less than or equal to the set threshold. Among them, when different dimensional components are selected for level, the setting of their thresholds is independent of each other. Level selection includes the following two methods:

[0178] The first method: starting from the first level of displacement data, count the number k of non-zero displacement corresponding components in this level, and calculate the proportion of k in this level. If the proportion is less than or equal to the set threshold, remove the displacement corresponding components of this level and all levels after this level, and only encode the displacement corresponding components before this level; if the proportion does not meet the requirements, continue to perform the same statistical calculation on the next level until a level that meets the conditions is found or all subdivided levels are traversed.

[0179] The second method: Start from the last level and continue to perform statistical calculations. If the proportion of the last level does not meet the requirements, stop traversing forward. If the proportion of the last level meets the requirements, remove the corresponding component of the displacement of the level, and continue to perform the same statistical calculations and proportion judgments on the previous level until a level that does not meet the requirements is encountered or all subdivided levels are traversed.

[0180] It should be noted that you can choose to mark the number of levels selected for each dimensional component and pass it to the decoding end; or you can choose not to mark the number of levels selected for each dimensional component, and only need to fill each dimensional component of the displacement information at the decoding end according to the number of vertices of the decoded grid. After the level selection, the selected displacement data can be arranged into the YUV image according to a predetermined rule. In the video arrangement process, in order to ensure that the resolution of each frame in the YUV video is the same and the pixel data can be arranged into a rectangle, the YUV video can be filled. The video filling value to be filled can be an agreed value known to the decoding end.

[0181] Use a video encoder to encode the YUV video to obtain a displacement code stream.

[0182] Decoding side:

[0183] Depending on whether the encoding end marks the number of selected levels of each dimensional component of the displacement, the displacement decoding method based on video decoding has different implementation methods. Two possible implementation methods are described below.

[0184] (1) Do not mark the number of selected levels

[0185] like Fig.10 As shown in the figure, the displacement decoding process of the selected level number of each dimensional component of the unmarked displacement is as follows: for the YUV video obtained by video decoding, all image pixel values ​​are extracted in the order in which the encoder arranges the displacement data into the video. Among them, the extracted image pixel value contains two parts of data: one is the displacement data selected by the encoder; the other is the value of the video filling at the encoder, which is a value agreed upon between the encoder and the decoder. After extracting the pixel value data, the video filling part in the pixel value data can be removed according to the video filling value agreed upon with the encoder, thereby obtaining the displacement data selected by the encoder.

[0186] According to the number of vertices of the mesh obtained after the decoded mesh is subdivided, each dimensional component of the displacement data is filled with zeros until the number of data is the same as the number of vertices of the mesh obtained after the decoded mesh is subdivided. At this time, the restored complete displacement data can be obtained. It should be noted that the number of vertices of the mesh obtained after the decoded mesh is subdivided can be obtained in other modules of the dynamic mesh decoding framework, such as the V-DMC framework, which can be obtained by subdividing the decoded basic mesh. Then, the restored complete displacement data is input into the post-processing module to obtain the original displacement data and complete the decoding of the displacement. The operation of the post-processing module is the reverse implementation of the operation of the pre-processing module on the encoding end, such as inverse quantization and inverse transformation.

[0187] (2) Mark the number of selected levels

[0188] like Fig.11 As shown, the displacement decoding process of marking the number of selected levels of each dimensional component of the displacement is as follows: Since the number of levels of displacement and the number of displacement data in each level are exactly the same as the decoded basic grid after subdivision, the number of displacement data selected by the encoder in each displacement dimensional component can be obtained according to the number of selected levels of each dimensional component in the decoded basic grid after subdivision. According to the number of displacement data of each dimensional component of the displacement, the data of each selected displacement dimensional component is extracted from the YUV video obtained by video decoding in the order in which the displacement data is arranged into the video by the encoder. After obtaining the data of each dimensional component of the displacement, zero padding is performed at the end of the data of each dimensional component according to the number of vertices of the grid after subdivision of the decoding grid, until the number of data is the same as the number of vertices; then the padded complete displacement data is input into the post-processing module to obtain the original displacement data, and the decoding of the displacement is completed. The operation of the post-processing module is the reverse implementation of the operation of the pre-processing module of the encoder, such as inverse quantization and inverse transformation.

[0189] Embodiment 2:

[0190] In this embodiment, the displacement information is encoded and decoded based on an entropy coding method.

[0191] Encoding side:

[0192] When entropy coding is selected as the displacement coding method, the displacement coding process is as follows Fig.12As shown. First, the displacement data is input into the pre-processing module, which will perform a series of pre-processing operations on the original displacement data, such as transformation, quantization, etc.; then, for each dimensional component of the pre-processed displacement data, it is input into the corresponding level selection module respectively. The level selection module selects the level by judging whether the proportion of the number of non-zero corresponding components of the displacement in the level is less than or equal to the set threshold. Among them, when different dimensional components are selected for level, the setting of their thresholds is independent of each other. Level selection includes the following two methods:

[0193] The first method: starting from the first level of displacement data, count the number k of non-zero displacement corresponding components in this level, and calculate the proportion of k in this level. If the proportion is less than or equal to the set threshold, remove the displacement corresponding components of this level and all levels after this level, and only encode the displacement corresponding components before this level; if the proportion does not meet the requirements, perform the same statistical calculation on the next level until a level that meets the conditions is found or all subdivided levels are traversed.

[0194] The second method: Start from the last level and continue to perform statistical calculations. If the proportion of the last level does not meet the requirements, stop traversing forward. If the proportion of the last level meets the requirements, remove the corresponding component of the displacement of the level, and continue to perform the same statistical calculations and proportion judgments on the previous level until a level that does not meet the requirements is encountered or all subdivided levels are traversed.

[0195] After the level selection, the selected displacement data can be directly entropy coded. The specific entropy coding method is not limited here, for example, context-based adaptive variable length coding (CAVLC), context-based adaptive binary coding (CABAC), etc. can be used.

[0196] Decoding end

[0197] like Fig.13As shown, for the received displacement code stream, the displacement code stream is first entropy decoded to obtain each dimension component of the selected displacement, and then zero padding is performed at the end of each dimension component of the displacement according to the number of vertices of the mesh after the decoded mesh is subdivided, until the number of data is the same as the number of vertices. It should be noted that the number of vertices of the mesh after the decoded mesh is subdivided can be obtained in other modules of the dynamic mesh decoding framework, such as the V-DMC framework, which can be obtained by subdividing the decoded basic mesh. Then, the restored complete displacement data is input into the post-processing module to obtain the original displacement data, and the decoding of the displacement is completed. The operation of the post-processing module is the reverse implementation of the operation of the pre-processing module on the encoding end, such as inverse quantization and inverse transformation.

[0198] Embodiment 3:

[0199] In this embodiment, the displacement information is encoded and decoded based on the V-DMC framework.

[0200] Encoding side:

[0201] The application of the embodiment of the present application at the encoding end is mainly in the displacement encoding process. Fig.14 As shown, the grid coding method performed by the coding end includes the following process:

[0202] (1) Mesh simplification

[0203] Mesh simplification is to simplify the current input mesh to a basic mesh with relatively few points and faces, and keep the shape of the original mesh as much as possible. The key of mesh simplification is the simplification operation and the corresponding error measurement. A feasible mesh simplification operation is Fig.15 As shown in the figure, the vertices at both ends of the edge are merged into one vertex and the connection between the two vertices is deleted. This process is repeated in the entire mesh according to certain rules to reduce the number of faces and vertices of the mesh to the target value.

[0204] During the simplification process, you can select an error metric to optimize the simplified result. For example, you can select the sum of the coefficients of the equations of all adjacent faces of a vertex as the error metric of the vertex, and the error metric of the corresponding edge is the sum of the error metrics of the two vertices on the edge. In other words, the error generated by merging an edge is the sum of the distances of all adjacent planes from the merged vertex to the original two vertices of the edge.

[0205] After determining the simplification operation and the corresponding error metric, the mesh simplification process begins iteratively. First, the vertex error of the initial mesh is calculated to obtain the error of each edge. Then each edge is arranged from small to large error, and the edge with the smallest error is selected for merging each time. At the same time, the position of the merged vertex is calculated, and the error of all edges related to the merged vertex is updated. That is, the order of edge arrangement is updated to ensure that each iteration is based on the global error metric. Through iteration, the mesh faces are simplified to the number required to meet lossy coding.

[0206] (2) Mesh parameterization

[0207] Texture coordinates may be regenerated based on the reconstructed base mesh of the current frame. This step requires generating texture coordinates for each attribute map of the input mesh. If multiple attribute maps have similar characteristics, they can share the same texture coordinates. There are many ways to generate texture coordinates, including mesh parameterization. There are many algorithms for parameterizing meshes, such as the Isocharts algorithm, which uses spectral analysis to achieve stretch-driven 3D mesh parameterization, UV-unwrapping, slicing, and packing the 3D mesh into a 2D texture domain.

[0208] (3) Subdivision deformation

[0209] The subdivision and deformation modules are optional modules that can improve the quality of the mesh reconstructed at the decoding end. This module can be omitted when the quality of the basic mesh can meet the application requirements.

[0210] The basic idea of ​​subdivision and deformation module is as follows Fig.16 As shown in , the same concept is applied to the input 3D mesh to generate displacement vector information. Fig.16 In the module, the input 2D curve (represented by a 2D polyline), called the "original" curve, is first downsampled to generate a basic curve / polyline, called the "simplified" curve. The subdivision scheme is then applied to the simplified polyline to generate the "subdivided" curve. The subdivided polyline is then deformed to obtain a better approximation of the original curve. That is, a geometric displacement vector is calculated for each vertex of the subdivided mesh so that the shape of the subdivided curve is as close to the shape of the original curve as possible. These geometric displacement vectors are the geometric displacement vector information (vertex displacement) output by this module. The same deformation process is also applied to the attribute information corresponding to the vertex to obtain the corresponding attribute displacement vector.

[0211] (4) Compression of base mesh

[0212] The basic mesh compression module compresses the basic mesh information output by the subdivision deformation module. There are two main modes of basic mesh compression, namely intra-frame mode and inter-frame mode. Fig.17As shown in the figure. In intra mode, the input of the basic mesh compression module is a 3D mesh, which contains geometric coordinates, connection relationships, and attribute information associated with vertices. In inter mode, the input of the basic mesh compression module is motion vectors and their identifiers, as well as possible intra-frame sub-meshes. After encoding, the encoded mesh needs to be reconstructed so that it can be provided to subsequent modules for processing.

[0213] Among them, the grid coding in intra-frame mode can use the existing grid encoder to encode the input basic grid, such as Draco, etc. The grid encoder type is encoded through auxiliary information and passed to the decoding end;

[0214] In inter-frame mode, motion vectors and identifiers need to be encoded. The motion vector identifier is an array consisting of numbers greater than or equal to 0. The array size is the same as the number of vertices of the base grid of the reference frame corresponding to the temporal motion vector identifier. The array can be directly encoded using an existing entropy coding algorithm, such as CABAC.

[0215] (5) Reconstruct the base mesh

[0216] After the base grid is encoded, the base grid is reconstructed. For a grid encoded in intra-frame mode, the encoded grid is decoded to obtain a reconstructed grid.

[0217] For the basic grid information encoded in the inter-frame mode, it is necessary to reconstruct the inter-frame sub-grid according to the motion vector and the identifier. If there is an intra-frame sub-grid in the inter-frame mode, the reconstructed inter-frame sub-grid is merged with the intra-frame sub-grid to obtain the reconstructed basic grid.

[0218] (6) Encoding vertex displacement

[0219] To encode the vertex displacement, first adjust the order of vertex displacement according to the reconstructed basic mesh; then perform wavelet transform and quantization on the displacement to obtain the quantized wavelet transform coefficient; then input the quantized wavelet transform coefficient into the data truncation module; then arrange the wavelet transform coefficient of the current frame into a two-dimensional image, and finally form a YUV video, which is encoded with the help of a video encoder to obtain the displacement code stream. Encoding the vertex displacement includes the following process:

[0220] a. Adjust the displacement order

[0221] The order of vertex displacements is adjusted according to the reconstructed base mesh. The reconstructed base mesh is first subdivided, and then the order of displacements is adjusted to be the same as the subdivided base mesh vertex order. Optionally, the coordinate system of the vertex displacements can be converted from a Cartesian coordinate system to a local coordinate system.

[0222] b. Wavelet transform

[0223] A transformation can be applied to the displacement vector to reduce the correlation between its data. An optional transformation is a linear wavelet transform, and its prediction process is defined as follows:

[0224]

[0225] Where v is the newly inserted midpoint on the edge (v1, v2), Signal(v), Signal(v1) and Signal(v2) are the displacement vectors corresponding to vertices v, v1 and v2 respectively. The displacement vector of vertex v is predicted and then updated. The update process is defined as follows:

[0226]

[0227] where v * is the set of all adjacent vertices of vertex v. The transformed displacement vector is called wavelet coefficient.

[0228] c. Coefficient quantization

[0229] The transformed displacement vector, i.e., the wavelet coefficient, can be quantified in a variety of ways. One quantization formula is as follows:

[0230] disp[v].d[k]=floor(disp[v].d[k]*scale[k])

[0231]

[0232] Where disp[v] represents the transformed value of the displacement vector of the vth vertex, d[k] represents the kth value of the displacement vector, and floor represents rounding down. scale[k] is an intermediate parameter. bitDepthPosition represents the bit depth of the geometric position of the current mesh vertex, and qp[k] represents the quantization parameter of the kth coefficient. As mentioned above, after the displacement vector is converted to the coordinate system, its normal component has a more significant effect on the quality than the tangential component, so a larger quantization parameter can be used for the tangential component.

[0233] At the same time, according to the characteristics of wavelet transform, different quantization parameters can be used for the newly generated vertices and the original vertices. That is, for the subdivided vertices, the quantization parameter update formula is as follows:

[0234] scale[k]=scale[k]*lodScale[k]

[0235] Among them, lodScale[k] represents the coefficient of the quantization parameter of the current subdivision level.

[0236] d. Level selection

[0237] For the quantized wavelet transform coefficients, before arranging them into a two-dimensional image, the x, y, and z three-dimensional components of the quantized wavelet transform coefficients are respectively subjected to a level selection operation. The level selection operation selects the level by judging whether the proportion of the number of non-zero corresponding components in the level of the quantized wavelet transform coefficients in the level is less than or equal to the set threshold. Specifically, starting from the first level of the quantized wavelet transform coefficients, the number k of non-zero corresponding components in the level is counted, and the proportion of k in the level is calculated. If the proportion is less than or equal to the set threshold, the corresponding components of the wavelet transform coefficients of the level and all levels after the level are removed, and only the corresponding components of the wavelet transform coefficients before the level are encoded; if the proportion does not meet the requirements, the same statistical calculation is performed on the next level until a level that meets the conditions is found or all subdivided levels are traversed. Alternatively, statistical calculations are performed continuously from the last level forward. If the proportion of the last level does not meet the requirements, the forward traversal is stopped; if the proportion of the last level meets the requirements, the corresponding components of the wavelet transform coefficients of this level are removed, and the same statistical calculations and proportion judgments are performed on the previous level until a level that does not meet the requirements is encountered or all subdivided levels are traversed.

[0238] It should be noted that the number of levels selected for each dimensional component can be marked and stored in the auxiliary information and transmitted to the decoding end; or the number of levels selected for each dimensional component can be not marked, and the displacement information only needs to be filled in at the decoding end according to the number of decoded mesh vertices.

[0239] e. Two-dimensional arrangement

[0240] After the displacement data is truncated through hierarchical selection, the displacement data can be arranged on the two-dimensional image as follows:

[0241] Traverse the wavelet coefficients in order from low frequency to high frequency;

[0242] For each coefficient, determine the index of the NxM pixel block where it is located (e.g., N=M=16), and store it in the raster scan order of the block;

[0243] The position of the corresponding NxM pixel block on the image is calculated according to the Morton order.

[0244] It should be noted that the block arrangement is not limited here, and other arrangement schemes can also be used, such as zigzag order, raster order, etc. The encoder can explicitly specify the corresponding arrangement scheme in the bitstream. In order to ensure that the resolution of each frame of the generated YUV video is the same and the number of pixels can be arranged in a rectangle, the video filling operation can be performed, and the video filling value can be an agreed value known by the decoder.

[0245] f. Video Compression

[0246] After arranging the wavelet coefficients on a two-dimensional image, they can be directly encoded using a video encoder to obtain a displacement code stream.

[0247] (7) Reconstruction of deformed mesh

[0248] When encoding the displacement vector, it is necessary to obtain the reconstructed displacement vector value. For the method of obtaining the quantized wavelet transform coefficients from the reconstructed two-dimensional image, the selection level of each dimensional component of the displacement can be divided into the following two methods according to whether the level is marked when selecting the level:

[0249] The first method: unmark the number of selected levels

[0250] In this way, the number of levels for selecting each dimensional component of the unmarked displacement. When obtaining the quantized wavelet transform coefficients, firstly, all the pixel data in the two-dimensional image are taken out according to the arrangement method during encoding; the extracted pixel data contains two values, one is the quantized wavelet transform coefficient selected by the level selection module, and the other is the video filling value in the two-dimensional arrangement process. After obtaining the pixel data, the video filling part in the pixel data can be removed according to the video filling value to obtain the data of each dimensional component of the quantized wavelet transform coefficient selected by the level selection module. After that, the zero-filling operation is performed on each dimensional component of the selected quantized wavelet transform coefficient according to the number of vertices after the subdivision of the reconstructed basic mesh, until the number of data of each dimensional component selected is the same as the number of vertices after the subdivision of the reconstructed basic mesh, and the quantized wavelet transform coefficient can be obtained.

[0251] The second method: mark the number of selected levels

[0252] In this way, the number of levels selected for each dimensional component of the displacement is marked. When obtaining the quantized wavelet transform coefficients, the number of levels of the quantized wavelet transform coefficients and the number of data in each level are exactly the same as those of the subdivided reconstructed basic grid. Therefore, in the subdivided reconstructed basic grid, the number of each dimensional component data of the quantized wavelet transform coefficients selected by the level selection module can be obtained according to the number of levels selected for each dimensional component. Then, according to the number of each dimensional component data of the wavelet transform coefficients, the selected data is taken out in the two-dimensional image according to the arrangement method during encoding. Then, according to the number of vertices of the subdivided reconstructed basic grid, zero padding operations are performed at the end of each dimensional component data taken out until the number of data is the same as the number of vertices, and the quantized wavelet transform coefficients can be obtained.

[0253] The quantized wavelet transform coefficients are dequantized and inversely transformed to obtain the displacement vector consistent with the decoding end. After obtaining the reconstructed geometric displacement vector, the subdivided reconstructed basic grid is obtained according to the corresponding geometric displacement vector to obtain the reconstructed subdivided deformed grid, which is passed to the attribute graph conversion module for attribute graph conversion.

[0254] (8) Attribute Graph Conversion

[0255] The attribute map is transformed according to the input original mesh, the input original attribute map and the reconstructed deformed mesh, such as Fig.18 shown.

[0256] The steps of property graph conversion are as follows:

[0257] a. Calculate the texture coordinates of each pixel on the attribute map to be generated, such as the texture coordinates corresponding to pixel A(i,j) is P(u,v);

[0258] b. Determine whether the texture coordinate is within a certain triangle face after the parameterization of the subdivided deformed mesh;

[0259] c. If the texture coordinate does not belong to any triangle, the pixel is marked as an empty pixel, which can be filled with a filling algorithm later;

[0260] d. If the texture coordinate belongs to a triangle, then:

[0261] Mark the pixel as filled;

[0262] Calculate the center of gravity coordinates of the texture in the current triangle surface according to the texture coordinates;

[0263] According to the barycentric coordinates and the corresponding triangular face, the two-dimensional texture coordinates are mapped to three-dimensional geometric coordinates, that is, mapped to the points on the subdivided deformation grid corresponding to the texture coordinates, such as Fig.18 As shown in M(x,y,z);

[0264] Find the point closest to the three-dimensional coordinate on the input original grid, such as Fig.18 Medium ′ (x, y, z);

[0265] The three-dimensional coordinates are calculated according to the triangular face where they are located, and their center coordinates are mapped to two dimensions to calculate their texture coordinates, i.e. P ′ (u ′ ,v ′ );

[0266] The texture coordinates are used to sample the original attribute map input to obtain the value A at the corresponding pixel position. ′ (i ′ ,j ′ );

[0267] Assign this value to the corresponding pixel A(i,j) on the attribute map to be generated.

[0268] (9) Attribute Graph Compression

[0269] After obtaining the converted attribute map, the empty pixels therein can be filled using existing filling algorithms (such as the Push-Pull algorithm). Then, existing video encoders such as H.264 / AVC, H.265 / HEVC, and H.266 / VVC can be used to encode the empty pixels to obtain the bitstream of the output attribute map. In addition, color space conversion and chroma subsampling operations can be selectively applied to enable video encoding to obtain better rate-distortion performance, such as color space conversion from RGB 444 to YUV420. In the case of multiple attribute maps, the texture coordinates corresponding to each attribute map need to be identified through auxiliary information during encoding.

[0270] (10) Auxiliary Information Coding

[0271] During the encoding process, there are some alternative schemes in each module, such as the type of grid encoder, the type of video encoder, the grid subdivision scheme, the spatial displacement vector transformation scheme, etc., and different schemes are allowed to be used in the encoding framework. Therefore, the selected scheme can be passed to the decoding end to guide the correct decoding.

[0272] The auxiliary information includes the type of static grid encoder, the type of video encoder, the grid subdivision scheme, the number of grid iterations, the geometric displacement vector transformation scheme, the coefficient arrangement scheme, the number of levels selected for the displacement x, y, and z components, or the color conversion scheme, etc. For the grid part encoded using the inter-frame coding mode, it also includes the corresponding reference frame list, etc.

[0273] After all modules are encoded, the basic grid part bitstream, texture coordinate part bitstream, displacement vector video bitstream, attribute map video bitstream and auxiliary information bitstream are mixed to obtain the encoded bitstream finally output by the encoding end.

[0274] Decoding side:

[0275] like Fig.19 As shown, the grid decoding method performed by the decoding end includes the following process:

[0276] (1) Auxiliary Information Decoding

[0277] The auxiliary information is used to indicate the type of static grid encoder, the type of video encoder, grid subdivision scheme, grid iteration number, geometric displacement vector transformation scheme, coefficient arrangement scheme, displacement truncation mark, or color conversion scheme, etc. For the grid part encoded using the inter-frame coding mode, it also includes the corresponding reference frame list, etc. The auxiliary information can be used to guide the decoding end to perform correct decoding, and the information is represented by the corresponding syntax parameters after decoding.

[0278] (2) Basic grid decoding

[0279] The decoding of the base grid can be divided into intra-frame mode and inter-frame mode. The base grid decoding is only responsible for decoding the information representing the base grid. For the intra-frame mode, the output of the base grid decoding is a three-dimensional grid, which contains geometric information and connection relationships, as well as possible attribute information and texture coordinate information. For the inter-frame mode, the base grid decoding is responsible for decoding the motion vector and identifier as well as the possible intra-frame sub-grid, and the output information is the motion vector, identifier and intra-frame sub-grid.

[0280] The decoding of the basic grid information in the intra-frame mode is also the decoding of the basic grid. The basic grid can be decoded using the corresponding grid decoder through the static grid decoder type indicated by the auxiliary information;

[0281] In inter-frame mode, the motion vector, identifier and possible intra-frame sub-grid are decoded. For the intra-frame sub-grid, the grid decoder indicated by the auxiliary information can be used for decoding. The motion vector identifier is obtained by entropy decoding, and the motion vector is obtained by inverse prediction and inverse quantization after entropy decoding.

[0282] (3) Segmentation

[0283] The subdivision module operation is the same as the encoding end subdivision operation, and the auxiliary information is used to indicate the number of subdivision iterations of different areas of the current basic grid.

[0284] (4) Displacement vector video decoding

[0285] The displacement vector video code stream is decoded according to the video decoder type indicated in the auxiliary information, including geometric displacement vector video and possible displacement vector video with different attributes.

[0286] (5) Displacement vector decoding

[0287] Depending on whether the number of levels selected for each dimension component of the displacement is marked, there are two ways to decode the displacement vector:

[0288] The first method: unmark the number of selected levels

[0289] In this way, the number of levels for selecting each dimensional component of the unmarked displacement. When obtaining the quantized wavelet transform coefficients, firstly, all the pixel data in the two-dimensional image are taken out according to the arrangement mode during encoding; the extracted pixel data contains two values, one is the quantized wavelet transform coefficients selected by the level selection module of the encoding end, and the other is the video filling value in the two-dimensional arrangement process of the encoding end, which is a value agreed upon between the encoding end and the decoding end. After obtaining the pixel data, the video filling part in the pixel data can be removed according to the video filling value to obtain the data of each dimensional component of the quantized wavelet transform coefficients selected by the level selection module of the encoding end. Then, the zero padding operation is performed on each dimensional component of the selected quantized wavelet transform coefficients according to the number of vertices after the decoding basic grid is subdivided, until the number of data of each dimensional component selected is the same as the number of vertices after the decoding basic grid is subdivided, and the quantized wavelet transform coefficients can be obtained. Then, the corresponding displacement vector can be restored by performing inverse quantization, inverse transformation and other operations on the quantized wavelet transform coefficients.

[0290] The second method: Mark the number of selected levels

[0291] In this way, the number of levels for selecting each dimensional component of the displacement is marked. When obtaining the quantized wavelet transform coefficients, the number of levels of the quantized wavelet transform coefficients and the number of data in each level are exactly the same as the subdivided decoding basic grid. Therefore, in the subdivided decoding basic grid, the number of each dimensional component data of the quantized wavelet transform coefficients selected by the encoding end level selection module can be obtained according to the number of selected levels of each dimensional component. Then, according to the number of each dimensional component data of the wavelet transform coefficient, the selected data is taken out in the two-dimensional image according to the arrangement method during encoding. Then, according to the number of vertices of the subdivided decoding basic grid, zero padding operations are performed at the end of each dimensional component data taken out, until the number of data is the same as the number of vertices, and the quantized wavelet transform coefficients can be obtained. Then, the quantized wavelet transform coefficients are subjected to inverse quantization, inverse transformation and other operations to restore the corresponding displacement vector.

[0292] (6) Reconstruct the deformed mesh

[0293] After the base mesh and displacement vectors are decoded and reconstructed, the deformed mesh is reconstructed based on these two parts. The corresponding displacement vector is added to each vertex of the subdivided mesh, and the calculation method is as follows:

[0294] deformedmesh[i].v[k]=subdivmesh[i].v[k]+displacement[k]

[0295] Among them, subdivmesh[i].v[k] is the geometric coordinates of the kth vertex after the base mesh is subdivided in the current frame (index is i), displacement[k] is the spatial displacement vector corresponding to the kth vertex, and deformedmesh[i].v[k] is the geometric coordinates of the kth vertex after subdivision and deformation in the current frame.

[0296] In addition, the attribute displacement vector is applied to the corresponding attribute value in the same way. Taking texture coordinates as an example, the calculation formula is as follows:

[0297] deformedmesh[i].vt[k]=subdivmesh[i].vt[k]+attdisplacement[k]

[0298] Among them, subdivmesh[i].vt[k] represents the k-th texture coordinate after the base mesh is subdivided in the current frame, attdisplacement[k] represents the k-th value of the attribute displacement vector, and deformedmesh[i].vt[k] represents the value of the k-th texture coordinate after subdivision and deformation in the current frame.

[0299] (7) Attribute Graph Decoding

[0300] The attribute graph decoder is responsible for decoding the attribute graph bitstream. The attribute graph is decoded using the video decoder indicated in the auxiliary information. An optional color space conversion is performed on it to obtain the attribute graph that matches the input attribute graph of the encoder. Figure 1 The image format is consistent, and the attribute map of the final decoded output is obtained. For multiple attribute map bitstreams, after decoding, each attribute map is matched with the corresponding texture coordinates according to the identifier.

[0301] After each module is processed, the deformed mesh and the corresponding attribute map reconstructed by the decoder are finally obtained. Subsequent applications use the reconstructed deformed mesh and attribute map as input for processing.

[0302] Embodiment 4:

[0303] In this embodiment, the syntax structure designed for the V-DMC codec framework is used as an example to illustrate the syntax structure of the code stream corresponding to the three-dimensional grid:

[0304] Table 1: Atlas sequence parameter set vdmc extension RBSP syntax

[0305]

[0306]

[0307] Wherein, Atlas sequence parameter set vdmc extension RBSP syntax refers to Atlas sequence parameter set vdmc extension network raw byte sequence payload (RBSP) syntax. Descriptor refers to a descriptor symbol.

[0308] Wherein, asps_vdmc_ext_subdivision_method indicates the subdivision method of the current mesh sequence. Table 2 describes the corresponding relationship between the subdivision method and asps_vdmc_ext_subdivision_method.

[0309] Table 2

[0310] asps_vdmc_ext_subdivision_method Name of subdivision method 0 NONE 1 MIDPOINT

[0311] Among them, Name of subdivision method refers to the name of the subdivision method.

[0312] Where asps_vdmc_ext_subdivision_iteration_count indicates the number of mesh subdivision iterations. If it does not exist, the asps_vdmc_ext_subdivision_iteration_count value is considered to be 0.

[0313] Wherein, asps_vdmc_ext_displacement_coordinate_system indicates the coordinate system identifier of the current grid sequence. Table 3 describes the correspondence between the supported coordinate systems and asps_vdmc_ext_displacement_coordinate_system.

[0314] Table 3

[0315] asps_vdmc_ext_displacement_coordinate_system Name of displacement coordinate system 0 CANNONICAL 1 LOCAL

[0316] Among them, Name of displacement coordinate system refers to the name of the displacement coordinate system.

[0317] Wherein, asps_vdmc_ext_transform_method indicates the wavelet transform identifier applied to the displacement. Table 4 describes the correspondence between the supported wavelet transform methods and asps_vdmc_ext_transform_method.

[0318] Table 4

[0319]

[0320] Among them, Name of transform method refers to the name of the transformation method.

[0321] Among them, asps_vdmc_ext_num_attribute_video represents the number of attributes sent through the video sub-bitstream.

[0322] asps_vdmc_ext_attribute_type_id[i] indicates the attribute type of the attribute video data unit with index i.

[0323] asps_vdmc_ext_attribute_frame_width[i] indicates the atlas frame width of the attribute video data unit with index i.

[0324] asps_vdmc_ext_attribute_frame_height[i] represents the atlas frame height of the attribute video data unit with index i.

[0325] asps_vdmc_ext_attribute_transform_method[i] applies to the transform identifier of the attribute with index i in the attribute video data unit.

[0326] Table 5 describes the correspondence between the supported transformation methods and asps_vdmc_ext_attribute_transform_method.

[0327] Table 5

[0328] asps_vdmc_ext_attribute_transform_method Name of transform method 0 NONE 1 LINEAR_LIFTING

[0329] Among them, Name of transform method refers to the name of the transformation method.

[0330] asps_vdmc_ext_packing_method equal to 0 indicates that the displacement component samples are arranged in ascending order, and asps_vdmc_ext_packing_method equal to 1 indicates that the displacement component samples are arranged in descending order.

[0331] asps_vdmc_ext_1d_displacement_flag equal to 1 indicates that only the normal component of the displacement is present in the displacement vector video. The other two components are inferred to be 0. asps_vdmc_ext_1d_displacement_flag equal to 0 indicates that all 3 components of the displacement are present in the displacement vector video.

[0332] Table 6: Atlas frame parameter set vdmc extension RBSP syntax

[0333]

[0334] Among them, Atlas frame parameter set vdmc extension RBSP syntax refers to Atlas frame parameter set vdmc extended RBSP syntax.

[0335] Among them, afps_vdmc_ext_overriden_flag is equal to 1, indicating that the following parameters exist in the atlas frame parameter set extension:

[0336] afps_vdmc_ext_subdivision_enable_flag;

[0337] afps_vdmc_ext_displacement_coordinate_system_enable_flag;

[0338] afps_vdmc_ext_transform_method_enable_flag;

[0339] afps_vdmc_ext_transform_parameters_enable_flag;

[0340] afps_vdmc_ext_packing_block_size_enable_flag.

[0341] Among them, afps_vdmc_ext_subdivision_enable_flag is equal to 1, which indicates that the following parameters exist in the atlas frame parameter set extension: afps_vdmc_ext_subdivision_method and afps_vdmc_ext_subdivision_iteration_count. When afps_vdmc_ext_subdivision_enable_flag does not exist, its value is inferred to be 0.

[0342] Among them, afps_vdmc_ext_displacement_coordinate_system_enable_flag is equal to 1, indicating that afps_vdmc_ext_displacement_coordinate_system exists in the atlas frame parameter set extension.

[0343] When afps_vdmc_ext_displacement_coordinate_system_enable_flag is not present, its value is inferred to be 0.

[0344] Among them, afps_vdmc_ext_transform_method_enable_flag is equal to 1 to indicate that afps_vdmc_ext_transform_method exists in the atlas frame parameter set extension. When afps_vdmc_ext_transform_method_enable_flag does not exist, its value is inferred to be equal to 0.

[0345] afps_vdmc_ext_transform_parameters_enable_flag equal to 1 indicates the presence of vdmc_lifting_transform_parameters in the atlas frame parameter set extension. When afps_vdmc_ext_transform_parameters_enable_flag is not present, its value is inferred to be equal to 0.

[0346] Wherein, afps_vdmc_ext_subdivision_method indicates the subdivision method of the current frame. When afps_vdmc_ext_subdivision_method does not exist, afps_vdmc_ext_subdivision_method is equal to asps_vdmc_ext_subdivision_method.

[0347] afps_vdmc_ext_subdivision_iteration_count indicates the number of iterations of the subdivision. afps_vdmc_ext_subdivision_iteration_count is equal to 0 when afps_vdmc_ext_subdivision_method is equal to 0 and afps_vdmc_ext_subdivision_iteration_count is not present.

[0348] When afps_vdmc_ext_subdivision_method is not 0 and afps_vdmc_ext_subdivision_iteration_count does not exist, afps_vdmc_ext_subdivision_iteration_count is equal to asps_vdmc_ext_subdivision_iteration_count.

[0349] afps_vdmc_ext_displacement_coordinate_system indicates the coordinate system identifier of the current frame grid displacement. When afps_vdmc_ext_displacement_coordinate_system does not exist, afps_vdmc_ext_displacement_coordinate_system is equal to asps_vdmc_ext_displacement_coordinate_system.

[0350] afps_vdmc_ext_transform_method indicates an identifier applied to displacement transform. When afps_vdmc_ext_transform_method is not present, afps_vdmc_ext_transform_method is equal to asps_vdmc_ext_transform_method.

[0351] afps_ext_disp_x_lod_enable_flag indicates the identifier of the number of layers transmitted by the current frame displacement x component. Table 7 describes the correspondence between the number of layers transmitted and afps_ext_disp_x_lod_enable_flag. The maximum value of afps_ext_disp_x_lod_enable_flag depends on afps_vdmc_ext_subdivision_iteration_count.

[0352] Table 7

[0353] afps_ext_disp_x_lod_enable_flag 0 Indicates that all levels of the x component are passed 1 Represents the first layer that passes the x component 2 Represents the first two layers that pass the x component … …

[0354] Among them, afps_ext_disp_y_lod_enable_flag indicates the identifier of the number of layers transmitted by the current frame displacement y component. Table 8 describes the correspondence between the number of transmitted layers and afps_ext_disp_y_lod_enable_flag. The maximum value of afps_ext_disp_y_lod_enable_flag depends on afps_vdmc_ext_subdivision_iteration_count.

[0355] Table 8

[0356] afps_ext_disp_y_lod_enable_flag 0 Indicates that all levels of the y component are passed 1 Represents the first layer that passes the y component 2 Represents the first two layers that pass the y component … …

[0357] Among them, afps_ext_disp_z_lod_enable_flag indicates the identifier of the number of levels transmitted by the current frame displacement z component. Table 9 describes the correspondence between the number of transmitted levels and afps_ext_disp_z_lod_enable_flag. The maximum value of afps_ext_disp_z_lod_enable_flag depends on afps_vdmc_ext_subdivision_iteration_count.

[0358] Table 9

[0359] afps_ext_disp_z_lod_enable_flag 0 Indicates that all levels of the z component are passed 1 Represents the first layer that passes the z component 2 Represents the first two layers that pass the z component … …

[0360] It should be noted that the grid coding method provided in the embodiment of the present application can be executed by a grid coding device, or a control module in the grid coding device for executing the grid coding method. In the embodiment of the present application, the grid coding device provided in the embodiment of the present application is described by taking the grid coding method executed by the grid coding device as an example.

[0361] See also Fig. 20 , Fig. 20 is a structural diagram of a grid coding device provided in an embodiment of the present application, such as Fig. 20 As shown, the grid coding device 300 includes:

[0362] An acquisition module 301 is used to acquire a basic grid corresponding to a three-dimensional grid, and perform subdivision processing based on the basic grid to obtain a subdivided grid;

[0363] A deformation module 302, configured to perform a deformation operation based on the subdivided grid to obtain a deformed grid;

[0364] A selection module 303 is used to select displacement information to be encoded from the displacement information corresponding to the subdivided grid, the displacement information corresponding to the subdivided grid is used to represent the displacement of the vertices of the subdivided grid relative to the deformed grid, and the number of levels of the displacement information to be encoded is less than or equal to the number of levels of the displacement information corresponding to the subdivided grid;

[0365] The encoding module 304 is used to encode the displacement information to be encoded to obtain a first encoding result.

[0366] Optionally, the displacement information corresponding to the subdivided grid includes displacement information of multiple levels, and the selection module includes:

[0367] A determining unit, configured to determine a target level among the multiple levels, wherein the target level is determined based on displacement information representing a displacement of a first preset value among the displacement information of the multiple levels;

[0368] A selection unit is used to select displacement information to be encoded from the displacement information corresponding to the subdivided grid based on the target level.

[0369] Optionally, a ratio of the number of displacement information representing a displacement of a first preset value in the displacement information of the target level to the total number of displacement information of the target level meets a preset condition.

[0370] Optionally, the determining unit is specifically configured to:

[0371] Determining a target level from a first level among the multiple levels, wherein a ratio of the number of displacement information representing a displacement of a first preset value in the displacement information of the target level to the total number of displacement information of the target level is greater than the first preset ratio;

[0372] In a case where the target level is not the first level, the displacement information to be encoded includes displacement information from the first level to a level before the target level.

[0373] Optionally, the determining unit is specifically configured to:

[0374] Determine a target level from the last level of the multiple levels, wherein a ratio of the number of displacement information representing a displacement of a first preset value in the displacement information of the target level to the total number of displacement information of the target level is less than or equal to a second preset ratio;

[0375] The displacement information to be encoded includes displacement information from a first level among the multiple levels to the target level.

[0376] Optionally, the displacement information corresponding to the subdivided grid includes: displacement components of a first dimension of N levels, displacement components of a second dimension of the N levels, and displacement components of a third dimension of the N levels;

[0377] The displacement information to be encoded includes: the displacement components of the first dimension of M1 levels among the displacement components of the first dimension of the N levels, the displacement components of the second dimension of M2 levels among the displacement components of the second dimension of the N levels, and the displacement components of the third dimension of M3 levels among the displacement components of the third dimension of the N levels, N is a positive integer, M1, M2 and M3 are all integers greater than or equal to 0, and M1, M2 and M3 are all less than or equal to N.

[0378] Optionally, the code stream corresponding to the three-dimensional grid includes the first encoding result and the second encoding result, and the second encoding result includes the encoding result of the first flag bit;

[0379] When the first flag bit is a second preset value, the second encoding result also includes an encoding result of the first parameter; or,

[0380] When the first flag bit is a third preset value, the second encoding result also includes an encoding result of the first parameter, an encoding result of the second parameter, and an encoding result of the third parameter.

[0381] The first parameter is used to characterize the M1 levels, the second parameter is used to characterize the M2 levels, and the third parameter is used to characterize the M3 levels.

[0382] Optionally, the encoding module is specifically used for:

[0383] Arranging the displacement information to be encoded into a video frame;

[0384] Performing padding processing on the arranged video frames to obtain padded video frames;

[0385] The padded video frame is encoded to obtain a first encoding result.

[0386] Optionally, the encoding module is specifically used for:

[0387] Perform entropy coding on the displacement information to be encoded to obtain a first coding result.

[0388] The grid coding device in the embodiment of the present application can be a device, a device or electronic device with an operating system, or a component, an integrated circuit, or a chip in a terminal. The device or electronic device can be a mobile terminal or a non-mobile terminal. Exemplarily, the mobile terminal can include but is not limited to the types of terminals listed above, and the non-mobile terminal can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personalcomputer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0389] The grid coding device provided in the embodiment of the present application can achieve Figure 5 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0390] It should be noted that the grid decoding method provided in the embodiment of the present application can be executed by a grid decoding device, or a control module in the grid decoding device for executing the grid decoding method. In the embodiment of the present application, the grid decoding device provided in the embodiment of the present application is described by taking the grid decoding method executed by the grid decoding device as an example.

[0391] See also Fig.21 , Fig.21 is a structural diagram of a grid decoding device provided in an embodiment of the present application, such as Fig.21 As shown, the grid decoding device 400 includes:

[0392] A first decoding module 401 is used to decode a first encoding result in a code stream corresponding to a three-dimensional grid to obtain decoded displacement information;

[0393] A second decoding module 402 is used to decode the third encoding result in the code stream to obtain a basic grid, and to subdivide the basic grid to obtain a subdivided grid;

[0394] A filling module 403 is used to perform filling processing on the decoded displacement information to obtain filled displacement information, wherein the number of levels of the filled displacement information is greater than or equal to the number of levels of the decoded displacement information;

[0395] The reconstruction module 404 is used to perform reconstruction processing based on the filled displacement information and the subdivided grid to obtain a reconstructed grid.

[0396] Optionally, the first decoding module is specifically used to:

[0397] Decoding a first encoding result in a code stream corresponding to the three-dimensional grid to obtain a video frame;

[0398] Decoded displacement information is obtained from the video frame, where the decoded displacement information is displacement information in the video frame excluding a preset video filling value.

[0399] Optionally, the device further comprises:

[0400] A third decoding module, used for decoding the second encoding result in the code stream to obtain a target parameter, where the target parameter is used to represent the number of levels of displacement information;

[0401] The first decoding module is specifically used for:

[0402] Decoding a first encoding result in a code stream corresponding to the three-dimensional grid to obtain a video frame;

[0403] Decoded displacement information is obtained from the video frame based on the target parameter.

[0404] Optionally, the first decoding module is specifically used to:

[0405] The first encoding result in the code stream corresponding to the three-dimensional grid is entropy decoded to obtain decoded displacement information.

[0406] Optionally, the filling module is specifically used for:

[0407] The decoded displacement information is filled based on the number of vertices of the subdivided mesh to obtain filled displacement information.

[0408] Optionally, the filled displacement information includes N levels of displacement components of the first dimension, N levels of displacement components of the second dimension, and N levels of displacement components of the third dimension;

[0409] The decoded displacement information includes the displacement components of the first dimension of M1 levels among the displacement components of the first dimension of the N levels, the displacement components of the second dimension of M2 levels among the displacement components of the second dimension of the N levels, and the displacement components of the third dimension of M3 levels among the displacement components of the third dimension of the N levels, N is a positive integer, M1, M2 and M3 are all integers greater than or equal to 0, and M1, M2 and M3 are all less than or equal to N.

[0410] Optionally, the device further comprises:

[0411] A fourth decoding module, used for decoding the second encoding result in the code stream to obtain a first flag bit and a target parameter;

[0412] Wherein, when the first flag bit is a second preset value, the target parameter includes the first parameter; or,

[0413] When the first flag bit is a third preset value, the target parameter includes a first parameter, a second parameter and a third parameter.

[0414] The first parameter is used to characterize the M1 levels, the second parameter is used to characterize the M2 levels, and the third parameter is used to characterize the M3 levels.

[0415] The grid decoding device in the embodiment of the present application can be a device, a device or electronic device with an operating system, or a component, an integrated circuit, or a chip in a terminal. The device or electronic device can be a mobile terminal or a non-mobile terminal. Exemplarily, the mobile terminal can include but is not limited to the types of terminals listed above, and the non-mobile terminal can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personalcomputer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0416] The grid decoding device provided in the embodiment of the present application can achieve Figure 8 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0417] Alternatively, if Fig. 22 As shown, the embodiment of the present application further provides a communication device 500, including a processor 501 and a memory 502, and the memory 502 stores a program or instruction that can be run on the processor 501. For example, when the communication device 500 is an encoding end device, the program or instruction is executed by the processor 501 to implement the various steps of the above-mentioned grid encoding method embodiment, and can achieve the same technical effect. When the communication device 500 is a decoding end device, the program or instruction is executed by the processor 501 to implement the various steps of the above-mentioned grid decoding method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0418] The embodiment of the present application also provides a terminal, including a processor and a communication interface, wherein the processor is used to: obtain a basic grid corresponding to a three-dimensional grid, perform subdivision processing based on the basic grid to obtain a subdivided grid; perform deformation operation based on the subdivided grid to obtain a deformed grid; select displacement information to be encoded from the displacement information corresponding to the subdivided grid, the displacement information corresponding to the subdivided grid is used to characterize the displacement of the vertices of the subdivided grid relative to the deformed grid, and the number of levels of the displacement information to be encoded is less than or equal to the number of levels of the displacement information corresponding to the subdivided grid; encode the displacement information to be encoded to obtain a first encoding result. Alternatively, the processor is used to: decode the first encoding result in the code stream corresponding to the three-dimensional grid to obtain the decoded displacement information; decode the third encoding result in the code stream to obtain the basic grid, and perform subdivision processing on the basic grid to obtain a subdivided grid; perform filling processing on the decoded displacement information to obtain the filled displacement information, and the number of levels of the filled displacement information is greater than or equal to the number of levels of the decoded displacement information; perform reconstruction processing based on the filled displacement information and the subdivided grid to obtain a reconstructed grid.

[0419] This terminal embodiment corresponds to the above-mentioned grid encoding method or grid decoding method embodiment, and each implementation process and implementation method of the above-mentioned grid encoding method or grid decoding method embodiment can be applied to this terminal embodiment and can achieve the same technical effect. Specifically, Fig.23 A schematic diagram of the hardware structure of a terminal for implementing an embodiment of the present application.

[0420] The terminal 600 includes but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609 and at least some of the components of a processor 610.

[0421] Those skilled in the art will appreciate that the terminal 600 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 610 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.23 The terminal structure shown in the figure does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0422] It should be understood that in the embodiment of the present application, the input unit 604 may include a graphics processing unit (GPU) 6041 and a microphone 6042, and the GPU 6041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 606 may include a display panel 6061, and the display panel 6061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 607 includes a touch panel 6071 and at least one of other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include two parts: a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0423] In the embodiment of the present application, after receiving downlink data from the network side device, the RF unit 601 can transmit the data to the processor 610 for processing; in addition, the RF unit 601 can send uplink data to the network side device. Generally, the RF unit 601 includes but is not limited to an antenna, an amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.

[0424] The memory 609 can be used to store software programs or instructions and various data. The memory 609 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instruction required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 609 may include a volatile memory or a non-volatile memory, or the memory 609 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 609 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0425] The processor 610 may include one or more processing units; optionally, the processor 610 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 610.

[0426] Wherein, when the terminal is a coding end device:

[0427] The processor 610 is configured to:

[0428] Obtaining a basic grid corresponding to the three-dimensional grid, and performing subdivision processing based on the basic grid to obtain a subdivided grid;

[0429] Performing a deformation operation based on the subdivided grid to obtain a deformed grid;

[0430] Selecting displacement information to be encoded from the displacement information corresponding to the subdivided grid, the displacement information corresponding to the subdivided grid is used to represent the displacement of the vertices of the subdivided grid relative to the deformed grid, and the number of levels of the displacement information to be encoded is less than or equal to the number of levels of the displacement information corresponding to the subdivided grid;

[0431] The displacement information to be encoded is encoded to obtain a first encoding result.

[0432] Optionally, the displacement information corresponding to the subdivided grid includes displacement information of multiple levels, and the processor 610 is specifically configured to:

[0433] Determine a target level among the multiple levels, wherein the target level is determined based on displacement information representing a displacement of a first preset value among the displacement information of the multiple levels;

[0434] The displacement information to be encoded is selected from the displacement information corresponding to the subdivided grid based on the target level.

[0435] Optionally, a ratio of the number of displacement information representing a displacement of a first preset value in the displacement information of the target level to the total number of displacement information of the target level meets a preset condition.

[0436] Optionally, the processor 610 is specifically configured to:

[0437] Determining a target level from a first level among the multiple levels, wherein a ratio of the number of displacement information representing a displacement of a first preset value in the displacement information of the target level to the total number of displacement information of the target level is greater than the first preset ratio;

[0438] In a case where the target level is not the first level, the displacement information to be encoded includes displacement information from the first level to a level before the target level.

[0439] Optionally, the processor 610 is specifically configured to:

[0440] Determine a target level from the last level of the multiple levels, wherein a ratio of the number of displacement information representing a displacement of a first preset value in the displacement information of the target level to the total number of displacement information of the target level is less than or equal to a second preset ratio;

[0441] The displacement information to be encoded includes displacement information from a first level among the multiple levels to the target level.

[0442] Optionally, the displacement information corresponding to the subdivided grid includes: displacement components of a first dimension of N levels, displacement components of a second dimension of the N levels, and displacement components of a third dimension of the N levels;

[0443] The displacement information to be encoded includes: the displacement components of the first dimension of M1 levels among the displacement components of the first dimension of the N levels, the displacement components of the second dimension of M2 levels among the displacement components of the second dimension of the N levels, and the displacement components of the third dimension of M3 levels among the displacement components of the third dimension of the N levels, N is a positive integer, M1, M2 and M3 are all integers greater than or equal to 0, and M1, M2 and M3 are all less than or equal to N.

[0444] Optionally, the code stream corresponding to the three-dimensional grid includes the first encoding result and the second encoding result, and the second encoding result includes the encoding result of the first flag bit;

[0445] When the first flag bit is a second preset value, the second encoding result also includes an encoding result of the first parameter;

[0446] When the first flag bit is a third preset value, the second encoding result also includes an encoding result of the first parameter, an encoding result of the second parameter, and an encoding result of the third parameter.

[0447] The first parameter is used to characterize the M1 levels, the second parameter is used to characterize the M2 levels, and the third parameter is used to characterize the M3 levels.

[0448] Optionally, the processor 610 is specifically configured to:

[0449] Arranging the displacement information to be encoded into a video frame;

[0450] Performing padding processing on the arranged video frames to obtain padded video frames;

[0451] The padded video frame is encoded to obtain a first encoding result.

[0452] Optionally, the processor 610 is specifically configured to:

[0453] Perform entropy coding on the displacement information to be encoded to obtain a first coding result.

[0454] Wherein, when the terminal is a decoding end device:

[0455] The processor 610 is configured to:

[0456] Decoding a first encoding result in a code stream corresponding to the three-dimensional grid to obtain decoded displacement information;

[0457] Decoding the third encoding result in the code stream to obtain a basic grid, and performing subdivision processing on the basic grid to obtain a subdivided grid;

[0458] Performing padding processing on the decoded displacement information to obtain padded displacement information, wherein the number of levels of the padded displacement information is greater than or equal to the number of levels of the decoded displacement information;

[0459] Reconstruction processing is performed based on the filled displacement information and the subdivided grid to obtain a reconstructed grid.

[0460] Optionally, the processor 610 is specifically configured to:

[0461] Decoding a first encoding result in a code stream corresponding to the three-dimensional grid to obtain a video frame;

[0462] Decoded displacement information is obtained from the video frame, where the decoded displacement information is displacement information in the video frame excluding a preset video filling value.

[0463] Optionally, the processor 610 is specifically configured to:

[0464] Decoding the second encoding result in the code stream to obtain a target parameter, where the target parameter is used to represent the number of levels of displacement information;

[0465] Decoding a first encoding result in a code stream corresponding to the three-dimensional grid to obtain a video frame;

[0466] Decoded displacement information is obtained from the video frame based on the target parameter.

[0467] Optionally, the processor 610 is specifically configured to:

[0468] The first encoding result in the code stream corresponding to the three-dimensional grid is entropy decoded to obtain decoded displacement information.

[0469] Optionally, performing padding processing on the decoded displacement information to obtain the padded displacement information includes:

[0470] The decoded displacement information is filled based on the number of vertices of the subdivided mesh to obtain filled displacement information.

[0471] Optionally, the filled displacement information includes N levels of displacement components of the first dimension, N levels of displacement components of the second dimension, and N levels of displacement components of the third dimension;

[0472] The decoded displacement information includes the displacement components of the first dimension of M1 levels among the displacement components of the first dimension of the N levels, the displacement components of the second dimension of M2 levels among the displacement components of the second dimension of the N levels, and the displacement components of the third dimension of M3 levels among the displacement components of the third dimension of the N levels, N is a positive integer, M1, M2 and M3 are all integers greater than or equal to 0, and M1, M2 and M3 are all less than or equal to N.

[0473] Optionally, the processor 610 is specifically configured to:

[0474] Decoding the second encoding result in the code stream to obtain a first flag bit and a target parameter;

[0475] Wherein, when the first flag bit is a second preset value, the target parameter includes a first parameter;

[0476] When the first flag bit is a third preset value, the target parameter includes a first parameter, a second parameter and a third parameter.

[0477] The first parameter is used to characterize the M1 levels, the second parameter is used to characterize the M2 levels, and the third parameter is used to characterize the M3 levels.

[0478] Specifically, the terminal of the embodiment of the present application also includes: instructions or programs stored in the memory 609 and executable on the processor 610, and the processor 610 calls the instructions or programs in the memory 609 to execute Fig.16 or Fig.17 The methods executed by the modules shown achieve the same technical effects, and therefore will not be described here in detail to avoid repetition.

[0479] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned grid encoding method embodiment are implemented, or when the program or instruction is executed by a processor, the various processes of the above-mentioned grid decoding method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0480] The processor is the processor in the terminal described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0481] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned grid encoding method embodiment, or to implement the various processes of the above-mentioned grid decoding method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0482] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0483] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0484] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0485] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A grid coding method, characterized in that: include: Obtaining a basic grid corresponding to the three-dimensional grid, and performing subdivision processing based on the basic grid to obtain a subdivided grid; Performing a deformation operation based on the subdivided grid to obtain a deformed grid; Selecting displacement information to be encoded from the displacement information corresponding to the subdivided grid, the displacement information corresponding to the subdivided grid is used to represent the displacement of the vertices of the subdivided grid relative to the deformed grid, and the number of levels of the displacement information to be encoded is less than or equal to the number of levels of the displacement information corresponding to the subdivided grid; The displacement information to be encoded is encoded to obtain a first encoding result.

2. The method according to claim 1, characterized in that The displacement information corresponding to the subdivided grid includes displacement information of multiple levels, and the selecting the displacement information to be encoded from the displacement information corresponding to the subdivided grid includes: Determine a target level among the multiple levels, wherein the target level is determined based on displacement information representing a displacement of a first preset value among the displacement information of the multiple levels; The displacement information to be encoded is selected from the displacement information corresponding to the subdivided grid based on the target level.

3. The method according to claim 2, characterized in that The ratio of the number of displacement information representing a displacement of a first preset value in the displacement information of the target level to the total number of displacement information of the target level meets a preset condition.

4. The method according to claim 2 or 3, characterized in that: The determining of a target level among the multiple levels comprises: Determining a target level from a first level among the multiple levels, wherein a ratio of the number of displacement information representing a displacement of a first preset value in the displacement information of the target level to the total number of displacement information of the target level is greater than the first preset ratio; In a case where the target level is not the first level, the displacement information to be encoded includes displacement information from the first level to a level before the target level.

5. The method according to claim 2 or 3, characterized in that: The determining of a target level among the multiple levels comprises: Determine a target level from the last level of the multiple levels, wherein a ratio of the number of displacement information representing a displacement of a first preset value in the displacement information of the target level to the total number of displacement information of the target level is less than or equal to a second preset ratio; The displacement information to be encoded includes displacement information from a first level among the multiple levels to the target level.

6. The method according to any one of claims 1 to 5, characterized in that The displacement information corresponding to the subdivided grid includes: displacement components of the first dimension of N levels, displacement components of the second dimension of the N levels, and displacement components of the third dimension of the N levels; The displacement information to be encoded includes: the displacement components of the first dimension of M1 levels among the displacement components of the first dimension of the N levels, the displacement components of the second dimension of M2 levels among the displacement components of the second dimension of the N levels, and the displacement components of the third dimension of M3 levels among the displacement components of the third dimension of the N levels, N is a positive integer, M1, M2 and M3 are all integers greater than or equal to 0, and M1, M2 and M3 are all less than or equal to N.

7. The method according to claim 6, characterized in that The code stream corresponding to the three-dimensional grid includes the first encoding result and the second encoding result, and the second encoding result includes the encoding result of the first flag bit; When the first flag bit is a second preset value, the second encoding result also includes an encoding result of the first parameter; or, When the first flag bit is a third preset value, the second encoding result also includes an encoding result of the first parameter, an encoding result of the second parameter, and an encoding result of the third parameter. The first parameter is used to characterize the M1 levels, the second parameter is used to characterize the M2 levels, and the third parameter is used to characterize the M3 levels.

8. The method according to any one of claims 1 to 7, characterized in that The encoding of the displacement information to be encoded to obtain a first encoding result includes: Arranging the displacement information to be encoded into a video frame; Performing padding processing on the arranged video frames to obtain padded video frames; The padded video frame is encoded to obtain a first encoding result.

9. The method according to any one of claims 1 to 7, characterized in that: The encoding of the displacement information to be encoded to obtain a first encoding result includes: Perform entropy coding on the displacement information to be encoded to obtain a first coding result.

10. A grid decoding method, characterized in that: include: Decoding a first encoding result in a code stream corresponding to the three-dimensional grid to obtain decoded displacement information; Decoding the third encoding result in the code stream to obtain a basic grid, and performing subdivision processing on the basic grid to obtain a subdivided grid; Performing padding processing on the decoded displacement information to obtain padded displacement information, wherein the number of levels of the padded displacement information is greater than or equal to the number of levels of the decoded displacement information; Reconstruction processing is performed based on the filled displacement information and the subdivided grid to obtain a reconstructed grid.

11. The method according to claim 10, characterized in that The decoding of the first encoding result in the code stream corresponding to the three-dimensional grid to obtain the decoded displacement information includes: Decoding a first encoding result in a code stream corresponding to the three-dimensional grid to obtain a video frame; Decoded displacement information is obtained from the video frame, where the decoded displacement information is displacement information in the video frame excluding a preset video filling value.

12. The method according to claim 10, characterized in that The method further comprises: Decoding the second encoding result in the code stream to obtain a target parameter, where the target parameter is used to represent the number of levels of displacement information; The decoding of the first encoding result in the code stream corresponding to the three-dimensional grid to obtain the decoded displacement information includes: Decoding a first encoding result in a code stream corresponding to the three-dimensional grid to obtain a video frame; Decoded displacement information is obtained from the video frame based on the target parameter.

13. The method according to claim 10, characterized in that The decoding of the first encoding result in the code stream corresponding to the three-dimensional grid to obtain the decoded displacement information includes: The first encoding result in the code stream corresponding to the three-dimensional grid is entropy decoded to obtain decoded displacement information.

14. The method according to any one of claims 10 to 13, characterized in that: The step of performing padding processing on the decoded displacement information to obtain the padded displacement information includes: The decoded displacement information is filled based on the number of vertices of the subdivided mesh to obtain filled displacement information.

15. The method according to any one of claims 10 to 14, characterized in that The filled displacement information includes N levels of displacement components of the first dimension, N levels of displacement components of the second dimension, and N levels of displacement components of the third dimension; The decoded displacement information includes the displacement components of the first dimension of M1 levels among the displacement components of the first dimension of the N levels, the displacement components of the second dimension of M2 levels among the displacement components of the second dimension of the N levels, and the displacement components of the third dimension of M3 levels among the displacement components of the third dimension of the N levels, N is a positive integer, M1, M2 and M3 are all integers greater than or equal to 0, and M1, M2 and M3 are all less than or equal to N.

16. The method according to claim 15, characterized in that The method further comprises: Decoding the second encoding result in the code stream to obtain a first flag bit and a target parameter; Wherein, when the first flag bit is a second preset value, the target parameter includes the first parameter; or, When the first flag bit is a third preset value, the target parameter includes a first parameter, a second parameter and a third parameter. The first parameter is used to characterize the M1 levels, the second parameter is used to characterize the M2 levels, and the third parameter is used to characterize the M3 levels.

17. A grid coding device, characterized in that: The device comprises: An acquisition module is used to acquire a basic grid corresponding to the three-dimensional grid, and perform subdivision processing based on the basic grid to obtain a subdivided grid; A deformation module, used for performing a deformation operation based on the subdivided grid to obtain a deformed grid; a selection module, configured to select displacement information to be encoded from the displacement information corresponding to the subdivided grid, wherein the displacement information corresponding to the subdivided grid is used to represent the displacement of the vertices of the subdivided grid relative to the deformed grid, and the number of levels of the displacement information to be encoded is less than or equal to the number of levels of the displacement information corresponding to the subdivided grid; The encoding module is used to encode the displacement information to be encoded to obtain a first encoding result.

18. A grid decoding device, characterized in that: The device comprises: A first decoding module, used for decoding a first encoding result in a code stream corresponding to the three-dimensional grid to obtain decoded displacement information; A second decoding module is used to decode the third encoding result in the code stream to obtain a basic grid, and subdivide the basic grid to obtain a subdivided grid; A filling module, used for performing filling processing on the decoded displacement information to obtain filled displacement information, wherein the number of levels of the filled displacement information is greater than or equal to the number of levels of the decoded displacement information; The reconstruction module is used to perform reconstruction processing based on the filled displacement information and the subdivided grid to obtain a reconstructed grid.

19. A terminal, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the grid encoding method as described in any one of claims 1 to 9; or, the program or instruction, when executed by the processor, implements the steps of the grid decoding method as described in any one of claims 10 to 16.

20. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, the steps of the grid encoding method as described in any one of claims 1 to 9 are implemented, or when the programs or instructions are executed by the processor, the steps of the grid decoding method as described in any one of claims 10 to 16 are implemented.

Images