Coding processing method, decoding processing method and related equipment

Abstract

The present application discloses an encoding processing method, a decoding processing method, and related devices, belonging to the field of three-dimensional mesh coding technology. The encoding processing method of an embodiment of the present application includes: determining a basic mesh code stream based on a mesh to be encoded; performing displacement encoding on a first vertex displacement to obtain a displacement code stream, wherein the first vertex displacement is obtained by adjusting the displacement order of a reconstructed basic mesh obtained by reconstructing the basic mesh code stream based on the displacement information, wherein the displacement information is obtained by subdividing and deforming a first mesh, and the first mesh is obtained by mesh simplification and mesh parameterization of the mesh to be encoded; generating a target code stream based on a texture map to be encoded corresponding to the mesh to be encoded, the basic mesh code stream, and the displacement code stream, wherein the target code stream includes target identification information, and the target identification information is used to indicate whether the encoding method of the displacement code stream is a video encoding method or an entropy encoding method.

Description

Technical Field

[0001] The present application belongs to the field of three-dimensional grid coding technology, and specifically relates to an encoding processing method, a decoding processing method and related equipment. Background Art

[0002] During 3D mesh encoding, displacement is calculated by calculating the distance between the vertices of the reconstructed mesh and the original mesh. This distance represents the distance from the vertex of the reconstructed mesh to the nearest neighbor on the original input mesh, aiming to improve mesh quality. Currently, displacement encoding is typically performed using a fixed encoding scheme, such as a video encoder. Consequently, existing displacement encoding techniques lack flexibility. Summary of the Invention

[0003] The embodiments of the present application provide an encoding processing method, a decoding processing method and related devices, which can solve the problem of poor flexibility of displacement coding.

[0004] In a first aspect, a coding processing method is provided, which is applied to a coding end and includes:

[0005] Determine a basic grid code stream based on the grid to be coded;

[0006] performing displacement encoding on a first vertex displacement to obtain a displacement code stream, wherein the first vertex displacement is obtained by adjusting a displacement order of a reconstructed basic mesh obtained by reconstructing the basic mesh code stream according to displacement information, the displacement information being obtained based on subdividing and deforming a first mesh, and the first mesh being obtained based on mesh simplification and mesh parameterization of the mesh to be encoded;

[0007] A target code stream is generated based on the texture map to be encoded corresponding to the grid to be encoded, the basic grid code stream and the displacement code stream, wherein the target code stream includes target identification information, and the target identification information is used to indicate that the encoding method of the displacement code stream is a video encoding method or an entropy encoding method.

[0008] In a second aspect, a decoding processing method is provided, which is applied to a decoding end and includes:

[0009] receiving a target code stream, the target code stream including a displacement code stream and target identification information, the target identification information being used to indicate whether a coding mode of the displacement code stream is a video coding mode or an entropy coding mode;

[0010] determining a target decoding mode according to the target identification information;

[0011] The displacement code stream is decoded according to the target decoding mode to obtain a third vertex displacement.

[0012] In a third aspect, a coding processing device is provided, which is applied to a coding end and includes:

[0013] A first processing module, configured to determine a basic grid code stream based on the grid to be coded;

[0014] a first encoding module configured to perform displacement encoding on a first vertex displacement to obtain a displacement code stream, wherein the first vertex displacement is obtained by adjusting a displacement order of a reconstructed basic mesh obtained by reconstructing the basic mesh code stream according to the displacement information, the displacement information being obtained by subdividing and deforming a first mesh, and the first mesh being obtained by mesh simplification and mesh parameterization of the mesh to be encoded;

[0015] A generation module is used to generate a target code stream based on the texture map to be encoded corresponding to the grid to be encoded, the basic grid code stream and the displacement code stream, wherein the target code stream includes target identification information, and the target identification information is used to indicate that the encoding method of the displacement code stream is a video encoding method or an entropy encoding method.

[0016] In a fourth aspect, a decoding processing device is provided, which is applied to a decoding end, including:

[0017] A receiving module, configured to receive a target code stream, wherein the target code stream includes a displacement code stream and target identification information, wherein the target identification information is used to indicate whether the encoding mode of the displacement code stream is a video encoding mode or an entropy encoding mode;

[0018] A determination module, configured to determine a target decoding mode according to the target identification information;

[0019] The first decoding module is configured to decode the displacement code stream according to the target decoding mode to obtain a third vertex displacement.

[0020] In a fifth aspect, a coding processing method is provided, which is applied to a coding end and includes:

[0021] Determine a basic grid code stream based on the grid to be coded;

[0022] Displacement coding is performed on the first vertex displacement to obtain a displacement code stream, where the displacement coding method is an entropy coding method.

[0023] In a sixth aspect, a coding processing method is provided, which is applied to a decoding end, including:

[0024] Get the target stream;

[0025] Perform entropy decoding on the target code stream to obtain vertex displacements.

[0026] In a seventh aspect, a coding processing device is provided, applied to a coding end, including:

[0027] A second processing module is used to determine a basic grid code stream based on the grid to be encoded;

[0028] The second encoding module is used to perform displacement encoding on the first vertex displacement to obtain a displacement code stream, where the displacement encoding method is an entropy encoding method.

[0029] In an eighth aspect, a coding processing device is provided, applied to a decoding end, comprising:

[0030] Acquisition module, used to obtain the target code stream;

[0031] The second decoding module is configured to perform entropy decoding on the target code stream to obtain vertex displacements.

[0032] In the ninth aspect, an electronic device is provided, which terminal includes a processor and a memory, the memory storing a program or instruction that can be run on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect, or, when executed by the processor, implements the steps of the method described in the second aspect, or, when executed by the processor, implements the steps of the method described in the fifth aspect, or, when executed by the processor, implements the steps of the method described in the sixth aspect.

[0033] In a tenth aspect, an electronic device is provided, comprising a processor and a communication interface, wherein:

[0034] When the electronic device is an encoding end, the processor is configured to determine a basic mesh bitstream based on a mesh to be encoded; perform displacement encoding on a first vertex displacement to obtain a displacement bitstream, wherein the first vertex displacement is obtained by adjusting a displacement order of a reconstructed basic mesh obtained based on the basic mesh bitstream according to displacement information, wherein the displacement information is obtained based on subdivision and deformation processing of a first mesh, wherein the first mesh is obtained based on mesh simplification and mesh parameterization of the mesh to be encoded; and generate a target bitstream based on a texture map to be encoded corresponding to the mesh to be encoded, the basic mesh bitstream, and the displacement bitstream, wherein the target bitstream includes target identification information, wherein the target identification information is used to indicate whether an encoding mode of the displacement bitstream is a video encoding mode or an entropy encoding mode.

[0035] When the electronic device is a decoding end, the communication interface is used to receive a target code stream, the target code stream including a displacement code stream and target identification information, the target identification information being used to indicate whether the encoding mode of the displacement code stream is a video encoding mode or an entropy encoding mode; the processor is used to determine a target decoding mode based on the target identification information; and decode the displacement code stream according to the target decoding mode to obtain a third vertex displacement;

[0036] or,

[0037] When the electronic device is an encoding end, the processor is configured to determine a basic grid code stream based on the grid to be encoded; perform displacement encoding on the first vertex displacement to obtain a displacement code stream, wherein the displacement encoding method is an entropy encoding method;

[0038] When the electronic device is a decoding end, the processor is configured to obtain a target code stream; and perform entropy decoding on the target code stream to obtain vertex displacements.

[0039] In an eleventh aspect, a video encoding and decoding system is provided, comprising: an encoding end device and a decoding end device, wherein the encoding end device can be used to perform the steps of the trellis encoding method as described in the first aspect, and the decoding end device can be used to perform the steps of the trellis decoding method as described in the second aspect;

[0040] Alternatively, the encoding end device can be used to perform the steps of the grid encoding method as described in the fifth aspect, and the decoding end device can be used to perform the steps of the grid decoding method as described in the sixth aspect.

[0041] In the twelfth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the second aspect are implemented, or the steps of the method described in the fifth aspect are implemented, or the steps of the method described in the sixth aspect are implemented.

[0042] In the ninth aspect, a chip is provided, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps of the method described in the first aspect, or the steps of the method described in the second aspect, or the steps of the method described in the fifth aspect, or the steps of the method described in the sixth aspect.

[0043] In the tenth aspect, a computer program / program product is provided, which is stored in a storage medium and is executed by at least one processor to implement the steps of the method described in the first aspect, or the steps of the method described in the second aspect, or the steps of the method described in the fifth aspect, or the steps of the method described in the sixth aspect.

[0044] In the embodiment of the present application, since target identification information is set in the target code stream to indicate the encoding method of the displacement code stream, different displacement encoding methods can be used for displacement encoding according to different requirements, thereby improving the flexibility of displacement encoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a traditional coding framework diagram;

[0046] Figure 2 It is a traditional decoding framework diagram;

[0047] Figure 3 This is a flowchart of the encoding processing method provided in an embodiment of the present application;

[0048] Figure 4 This is an example diagram of a grid simplification operation in the encoding processing method provided in an embodiment of the present application;

[0049] Figure 5 This is an example diagram of subdivision processing in the encoding processing method provided in an embodiment of the present application;

[0050] Figure 6 This is an example diagram of a coding framework in the coding processing method provided in an embodiment of the present application;

[0051] Figure 7 Schematic diagram of the decoding process provided by the embodiment of the present application;

[0052] Figure 8 This is an example diagram of a decoding framework in the decoding processing method provided in an embodiment of the present application;

[0053] Figure 9 is a structural diagram of the encoding processing device provided in an embodiment of the present application;

[0054] Figure 10 is a structural diagram of a decoding processing device provided in an embodiment of the present application;

[0055] Figure 11 is a structural diagram of a communication device provided in an embodiment of the present application;

[0056] Figure 12 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the accompanying drawings in the embodiments of this application to clearly describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0058] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and do not limit the number of objects, for example, the first object can be one or more. In addition, "or" in the specification and claims represents at least one of the connected objects, for example, "A or B" covers three options, namely, Option 1: including A and excluding B; Option 2: including B and excluding A; Option 3: including both A and B. The character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0059] The term "instruction" in the specification and claims of this application can be either an explicit instruction or an implicit instruction. An explicit instruction can be understood as the sender explicitly informing the recipient of the required operation or requested result in the instruction sent; an implicit instruction can be understood as the recipient making a judgment based on the instruction sent by the sender and determining the required operation or requested result based on the judgment result.

[0060] The encoding and decoding end corresponding to the grid encoding and decoding method in the embodiment of the present application can be a terminal, which can also be called a terminal device or user equipment (UE). The terminal can be a mobile phone, a tablet computer (Tablet Personal Computer), a laptop computer (Laptop Computer) or a notebook computer, a personal digital assistant (PDA), a handheld computer, a netbook, an ultra-mobile personal computer (UMPC), a mobile Internet device (Mobile Internet Device, MID), a wearable device (Wearable Device) or a vehicle-mounted device (VUE), a pedestrian terminal (PUE) and other terminal-side devices. Wearable devices include: smart watches, bracelets, headphones, glasses, etc. It should be noted that the specific type of the terminal is not limited in the embodiment of the present application.

[0061] For ease of understanding, some of the contents involved in the embodiments of this application are described below:

[0062] 1. Visual Volumetric Video-based Coding (V3C) standard

[0063] The V3C standard provides a method for encoding and decoding various three-dimensional media through video or image coding technology. Specifically, it converts the three-dimensional media content from a three-dimensional representation into multiple two-dimensional representations (called V3C components) through projection and other methods before encoding, and then encodes the two-dimensional representation using existing video or image coding technology. V3C components mainly include occupancy components, geometric components, and attribute components. The occupancy component can indicate which areas in the two-dimensional representation are associated with the data of the three-dimensional representation; the geometric component represents information related to the position of the three-dimensional data in space, and the attribute component can provide attribute information corresponding to the vertex, such as material and texture. In addition, the components also contain information on how to reconstruct the three-dimensional model through these components, which is called atlas information.

[0064] Atlas information is used to link all components, and additional information for reconstructing 3D from 2D is also included in the atlas components. An atlas consists of multiple basic units, called patches. Each patch represents a region of the available 2D components and contains the information needed to project that region back into 3D space.

[0065] 2. Video-based dynamic mesh coding (VDMC).

[0066] VDMC is a standard developed by the Moving Picture Experts Group (MPEG) for compressing 3D meshes. Its main idea is to compress 3D meshes by using the existing V3C standard. Since 3D meshes have connection information that needs to be encoded, its specific encoding process is slightly different from V3C. The syntax and semantics of the V3C standard decoding end and the decoding operation need to be extended to support the decoding and reconstruction of 3D meshes. VDMC-related encoding and decoding frameworks are as follows: Figure 1 and Figure 2 shown.

[0067] The overall framework of the encoding end is as follows Figure 1As shown, the input mesh is first simplified by the simplification module. Mesh parameterization is then used to generate new texture coordinates for the mesh. The parameterized mesh is then subdivided and deformed. This involves inserting new vertices according to a specific subdivision method and calculating the distances between the vertices of the subdivided mesh and their nearest neighbors on the input mesh, which is referred to as displacement information. The parameterized mesh is then adjusted based on the displacement information, i.e., the vertex positions of the pre-deformed mesh. This adjusted mesh, called the base mesh, is then fed into the base mesh encoding module, where it is compressed using an existing mesh encoder. In inter-frame mode, motion vectors are also generated for each vertex of the base mesh based on a reference frame. The base mesh module only compresses the motion vectors. After encoding, the base mesh is reconstructed, and the displacement order is adjusted based on the vertex order of the reconstructed base mesh. The reordered vertex displacement information is then first subjected to a wavelet transform, and the transformed coefficients (also known as wavelet coefficients) are quantized. These quantized coefficients are then arranged into a two-dimensional image according to a specific scan order, and the 2D image is encoded using a video encoder. The reconstructed displacement information is then applied to the subdivided base mesh to produce a reconstructed, deformed mesh. This mesh, along with the original input mesh and its corresponding texture map, is then fed into a texture map conversion module to produce a texture map corresponding to the reconstructed mesh. This texture map is also encoded using the video encoder. Parameters used in the encoding process, such as the video encoder type, mesh encoder type, transform parameters, and quantization parameters, are passed to the decoder via auxiliary information.

[0068] The overall framework of the decoding end is as follows Figure 2 As shown, for the received bitstream, the decoding end first demultiplexes each part of the bitstream to obtain the base mesh bitstream, displacement bitstream, texture map bitstream and auxiliary information bitstream. For the base mesh bitstream, the mesh decoder indicated by the auxiliary information is used to decode the base mesh. The displacement bitstream and texture map bitstream are decoded by the video decoder. For the displacement part, after the video is decoded, the displacement needs to be extracted from the image through the displacement decoding module, and dequantization, inverse transformation and other steps are performed. Then, it is applied to the subdivided base mesh to obtain the deformed mesh reconstructed by the decoding end. After decoding, the texture map is the texture map corresponding to the reconstructed deformed mesh. The subsequent application or rendering module processes the reconstructed deformed mesh and the decoded texture map as input.

[0069] The following describes in detail the encoding processing method provided in the embodiments of the present application through some embodiments and their application scenarios in combination with the accompanying drawings.

[0070] Reference Figure 3 , the embodiment of the present application provides a coding processing method, such as Figure 3 As shown, the encoding processing method includes:

[0071] Step 301, determining a basic grid code stream based on the grid to be coded;

[0072] Step 302: Displacement encoding is performed on the first vertex displacement to obtain a displacement code stream, wherein the first vertex displacement is obtained by adjusting the displacement order of a reconstructed basic mesh obtained by reconstructing the basic mesh code stream according to the displacement information, wherein the displacement information is obtained by subdividing and deforming the first mesh, and the first mesh is obtained by mesh simplification and mesh parameterization of the mesh to be encoded;

[0073] Step 303: Generate a target bitstream based on the texture map to be encoded corresponding to the mesh to be encoded, the basic mesh bitstream and the displacement bitstream, wherein the target bitstream includes target identification information, and the target identification information is used to indicate whether the encoding method of the displacement bitstream is a video encoding method or an entropy encoding method.

[0074] In the embodiment of the present application, determining a basic grid code stream based on the grid to be encoded may include the following process:

[0075] Performing grid simplification and grid parameterization processing on the grid to be coded to obtain a first grid;

[0076] performing subdivision processing and deformation processing on the first mesh to obtain displacement information and a second mesh obtained by adjusting vertex positions of the first mesh;

[0077] The second grid is compressed and coded to obtain a basic grid code stream.

[0078] Optionally, mesh simplification is to simplify the currently input mesh to be encoded into a basic mesh with relatively few points and faces, and to keep the shape of the original mesh as much as possible. The focus of mesh simplification is the simplification operation and the corresponding error metric. A feasible mesh simplification operation is as follows: Figure 4 As shown in the figure, the vertices at both ends of the edge are merged into a single vertex and the connection between the two vertices is deleted. This process is repeated throughout the mesh according to a certain rule to reduce the number of faces and vertices of the mesh to the target value.

[0079] During the simplification process, a specific error metric can be selected to optimize the simplified result. For example, the error metric for a vertex can be the sum of the coefficients of the equations of all adjacent faces. The error metric for an edge can be the sum of the error metrics of the two vertices on the edge. In other words, the error resulting from merging an edge is the sum of the distances from the merged vertex to all adjacent faces of the original two vertices on the edge.

[0080] After determining the simplification operation and the corresponding error metric, the mesh simplification process begins iteratively. First, the vertex errors of the initial mesh are calculated to obtain the error for each edge. Edges are then sorted from smallest to largest error, and the edge with the smallest error is merged each time. Simultaneously, the positions of the merged vertices are calculated, and the errors of all edges associated with the merged vertices are updated. This means that the order of edge arrangement is updated to ensure that each iteration is based on a global error metric. Through iteration, the mesh faces are simplified to the number required for lossy encoding.

[0081] Mesh parameterization primarily involves regenerating texture coordinates for the simplified mesh to obtain the primary mesh. The specific mesh parameterization algorithm can be configured based on actual needs. For example, the Isocharts algorithm uses spectral analysis to implement stretch-driven 3D mesh parameterization, performing UV unwrapping, tiling, and packing the 3D mesh into a 2D texture domain.

[0082] Subdivision is applied to the input 3D mesh to generate displacement vector information. The input 2D curve (represented by a 2D polyline), called the "original" curve, is first downsampled to generate a basic curve / polyline, called the "simplified" curve. The subdivision scheme is then applied to the simplified polyline to generate the "subdivided" curve. The subdivided polyline is then deformed to obtain a better approximation of the original curve. That is, a geometric displacement vector is calculated for each vertex of the subdivided mesh so that the shape of the subdivided curve is as close as possible to the shape of the original curve. These geometric displacement vectors are the geometric displacement vector information output by this module. The same deformation process is also applied to the attribute information corresponding to the vertex to obtain the corresponding attribute displacement vector.

[0083] The subdivision deformation process takes the parameterized mesh as input. This step first subdivides the input mesh. The subdivision scheme can be any. One possible scheme is the midpoint subdivision scheme, which subdivides each triangle into four subtriangles in each subdivision process iteration, such as Figure 5 As shown in Figure 2, a new vertex is introduced in the middle of each edge, and the subdivision of geometric information and attribute information is performed independently because the connection relationship between geometric information and attribute information is usually different.

[0084] Optionally, calculate the midpoint v of the newly introduced edge (v1,v2) 12 Position Pos(v 12 ) is shown in formula (1):

[0085]

[0086] Among them, Pos(v1) is the geometric coordinate of vertex v1, and Pos(v2) is the geometric coordinate of vertex v2.

[0087] For each point in the subdivided mesh, we find the nearest neighbor of each point on the original input mesh (including points on the original mesh surface). This can be accelerated using data structures such as kdTree. We calculate the distance between each vertex on the subdivided mesh and the geometric coordinates of its nearest neighbor on the original input mesh to obtain the displacement vector of each vertex's geometric coordinates. This module passes the generated displacement vector to subsequent modules for encoding.

[0088] For the generated displacement vectors, which are in the same global coordinate system as the input mesh, one possible optimization method is to transform them into a local coordinate system, where the local coordinate system of each vertex is defined by the normal vector of the vertex on the subdivided mesh. The advantage of this method is that the normal component of the geometric displacement vector has a more significant impact on the quality of the reconstructed mesh than the two tangential components, so a larger quantization parameter can be set for the tangential component.

[0089] The compression encoding process of the basic mesh can be understood as inputting the basic mesh (i.e., the above-mentioned second mesh) into the basic mesh compression module for compression encoding. There are two main different modes for this compression encoding, namely, intra-frame mode and inter-frame mode. In intra-frame mode, the basic mesh compression module encodes and reconstructs the input three-dimensional mesh using the existing static mesh encoder. In inter-frame mode, the basic mesh compression module calculates the motion vector between the input mesh vertex and the reference frame mesh vertex, and then encodes the motion vector and reconstructs the current frame basic mesh based on the reconstructed motion vector and the reference frame mesh. Among them, the basic mesh compression module outputs the compressed code stream, and passes the reconstructed basic mesh as an output to the displacement sequence adjustment module, which performs vertex displacement adjustment on the reconstructed basic mesh based on the displacement information to obtain the first vertex displacement.

[0090] For displacement coding, two coding methods are provided in the embodiment of the present application, namely video coding method and entropy coding method.

[0091] Optionally, in an embodiment of the present application, any video encoder can be used to encode the first vertex displacement, in which case the type information of the video encoder needs to be encoded in the auxiliary information. The first vertex displacement can also be encoded using any entropy coding algorithm, such as context-based adaptive binary arithmetic coding (CABAC). At the same time, since the statistical characteristics of different levels of displacement and different components may be different, assigning different entropy coding contexts to each subdivision level or different components may achieve better performance.

[0092] Optionally, the target identification information may implicitly or explicitly indicate the encoding method of the bit-shifted code stream.

[0093] In the embodiment of the present application, since target identification information is set in the target code stream to indicate the encoding method of the displacement code stream, different displacement encoding methods can be used for displacement encoding according to different requirements, thereby improving the flexibility of displacement encoding.

[0094] Optionally, in some embodiments, when the encoding mode is an entropy encoding mode, performing displacement encoding on the first vertex displacement to obtain a displacement code stream includes any one of the following:

[0095] Performing displacement processing on the first vertex displacement to obtain a second vertex displacement, and inputting the second vertex displacement into an entropy encoder for displacement encoding to obtain the displacement code stream;

[0096] The first vertex displacement is input into an entropy encoder for displacement coding to obtain the displacement code stream.

[0097] In the embodiment of the present application, the above-mentioned displacement processing may include wavelet transform and coefficient quantization. It should be understood that when the displacement coding method is a video coding method, the corresponding displacement processing also includes two-dimensional arrangement, that is, arranging the coefficients after quantization into a two-dimensional image, and finally encoding the two-dimensional image using a video encoder to obtain a displacement code stream.

[0098] Optionally, an optional arrangement is as follows: traverse the wavelet coefficients in order from low frequency to high frequency.

[0099] Optionally, for each coefficient, determine the index of the NxM pixel block (e.g., N=M=16) in which the coefficient is located, where the coefficients should be stored in the pixel block in raster scan order;

[0100] Optionally, the position of the corresponding NxM pixel block on the image may be calculated according to the Morton order.

[0101] It should be understood that the embodiments of the present application are not limited to the arrangement mode, and other arrangements may also be used, such as zigzag order, raster order, etc. The encoder may explicitly specify the corresponding arrangement mode in the bitstream.

[0102] It should be noted that in the embodiment of the present application, bit shifting can be performed or bit shifting encoding can be performed directly without bit shifting. Bit shifting can improve the compression efficiency of subsequent bit shifting encoding, thereby reducing the size of the target code stream.

[0103] Optionally, in some embodiments, generating a target bitstream based on the to-be-encoded texture map corresponding to the to-be-encoded mesh, the basic mesh bitstream, and the displacement bitstream includes:

[0104] Encoding the auxiliary information to obtain an auxiliary information code stream, wherein the auxiliary information is used to assist a decoding end in decoding;

[0105] Performing texture conversion on the to-be-encoded texture map corresponding to the to-be-encoded grid and the to-be-encoded grid to obtain a target texture map, and performing compression encoding on the target texture map to obtain a texture map bitstream;

[0106] The basic grid code stream, the displacement code stream, the auxiliary information code stream and the texture map code stream are mixed to obtain the target code stream.

[0107] In this embodiment of the present application, displacement reconstruction can be performed based on the displacement code stream to obtain a reconstructed displacement. A deformed mesh can be reconstructed based on the reconstructed displacement and the reconstructed base mesh to obtain a reconstructed deformed mesh. Then, a texture map conversion can be performed on a texture map to be encoded based on the reconstructed deformed mesh and the mesh to be encoded to obtain a target texture map. The target texture map is then compressed and encoded to obtain a texture map code stream. Finally, the base mesh code stream, the displacement code stream, the auxiliary information code stream, and the texture map code stream are mixed to obtain the target code stream.

[0108] Optionally, in some embodiments, the auxiliary information includes at least one of the following: first coding information corresponding to the basic mesh code stream, second coding information corresponding to the texture map code stream, coding method of the displacement code stream, and processing parameters for displacement processing of the first vertex displacement.

[0109] In an embodiment of the present application, the first encoding information may include information such as an encoder type, the second encoding information may include information such as an encoder type, the encoding method of the displacement code stream may be a video encoding method or an entropy encoding method, and the processing parameter may include at least one of a transform parameter, a quantization parameter, and an alignment parameter.

[0110] Optionally, a transformation may be applied to the displacement vector to reduce the correlation between its data. An optional transformation is a linear wavelet transform, and its prediction process is defined as shown in Equation 2:

[0111]

[0112] Where v is the newly inserted midpoint on the edge (v1, v2), Signal(v) is the displacement vector corresponding to vertex v, Signal(v1) is the displacement vector corresponding to vertex v1, and Signal(v2) is the displacement vector corresponding to vertex v2. The displacement vector of vertex v is predicted and then updated. The update process is defined as shown in Equation 3:

[0113]

[0114] where v * is the set of all adjacent vertices of vertex v. The transformed displacement vector is called wavelet coefficient.

[0115] Optionally, the transformed displacement vector, i.e., the wavelet coefficient, may be quantized. There are multiple quantization methods. One method is shown in Equations 4 and 5:

[0116] disp[v].d[k]=floor(disp[v].d[k]*scale[k]) (4);

[0117]

[0118] Where disp[v] represents the transformed value of the displacement vector at the vth vertex, d[k] represents the kth value of the displacement vector, and floor indicates rounding down. bitDepthPosition represents the bit depth of the current mesh vertex's geometric position, and qp[k] represents the quantization parameter for the kth coefficient. As mentioned earlier, after transforming the coordinate system of the displacement vector, its normal component has a more significant impact on quality than its tangential component, so a larger quantization parameter can be used for the tangential component.

[0119] At the same time, according to the characteristics of wavelet transform, different quantization parameters can be used for the newly generated vertices and the original vertices after subdivision processing. That is, for the vertices after subdivision processing, the quantization parameter update is shown in Equation 6:

[0120] disp[v].d[k]=floor(disp[v].d[k]*scale[k]) (6);

[0121] Among them, lodScale[k] represents the coefficient of the quantization parameter of the current subdivision level.

[0122] Optionally, in some embodiments, after performing displacement decoding to obtain the displacement, a displacement vector consistent with the decoding end can be obtained by inverse quantization (i.e., inverse coefficient quantization) and inverse transformation (i.e., inverse wavelet quantization). After obtaining the reconstructed geometric displacement vector, the reconstructed basic grid is subdivided and the reconstructed deformed grid after subdivision processing and deformation processing is obtained according to the corresponding displacement vector, and is passed to the texture map conversion module. The texture map conversion module performs texture map conversion based on the input original grid (i.e., the grid to be encoded), the input original texture map (i.e., the texture map to be encoded), and the reconstructed deformed grid. Specifically, the texture map conversion may include the following steps:

[0123] Calculate the texture coordinates of each pixel on the target texture map to be generated, such as the texture coordinates corresponding to pixel A(i,j) are P(u,v).

[0124] Determine whether the texture coordinate is within a certain triangle face after the parameterization of the subdivided deformed mesh;

[0125] If the texture coordinate does not belong to any triangle, the pixel is marked as an empty pixel and can be filled with a filling algorithm later;

[0126] If the texture coordinate belongs to a triangle, perform the target operation.

[0127] The target operations may include:

[0128] Mark the pixel as filled;

[0129] Calculate the center of gravity coordinates of the texture in the current triangle according to the texture coordinates;

[0130] According to the barycentric coordinates and the corresponding triangular face, the two-dimensional texture coordinates are mapped to three-dimensional geometric coordinates, that is, mapped to the point on the subdivided deformed grid corresponding to the texture coordinates, as shown by M(x, y, z) in the figure;

[0131] Find the point closest to the three-dimensional coordinate on the input original grid, as shown in the figure M'(x,y,z);

[0132] Calculate the barycentric coordinates of the three-dimensional coordinates according to the triangle face they are on and map them to two dimensions to calculate their texture coordinates, i.e. P′(u′,v′);

[0133] The texture coordinates are used to sample the input original texture map to obtain the value A′(i′, j′) of the corresponding pixel position;

[0134] Assign this value to the corresponding pixel A(i,j) on the target texture map to be generated.

[0135] Optionally, after obtaining the converted texture map, empty pixels therein can be filled using a relevant filling algorithm (such as the Push-Pull algorithm). This can then be encoded using existing video encoders such as H.264 / AVC, H.265 / HEVC, and H.266 / VVC to produce an output texture map bitstream. Furthermore, color space conversion and chroma subsampling can be selectively applied to achieve better rate-distortion performance in video encoding, such as color space conversion from RGB 444 to YUV420.

[0136] It should be noted that the location of the target identification information within the target bitstream can be set based on actual needs. For example, in some embodiments, the target bitstream includes an auxiliary information bitstream determined based on auxiliary information, and the target identification information is part of the auxiliary information bitstream, which is used to assist a decoding end in decoding. In other words, the target bitstream includes an auxiliary information bitstream, and the auxiliary information bitstream includes the target identification information.

[0137] In an embodiment of the present application, the V3C parameter set can be extended based on the V3C syntax structure, which is used to specify common parameters in the sequence. Parameters indicating the grid displacement encoding method are defined in the parameter set, as shown in Table 1 below.

[0138] Table 1:

[0139]

[0140] Among them, vps_ext_disp_video_codec_id[j] indicates the type of displacement encoder used for the grid corresponding to the atlas with index j (corresponding to one or more three-dimensional grid frames). If its value is 0, it means that the displacement uses entropy coding, that is, the displacement code stream needs to be decoded by an entropy decoder; a value greater than 0 indicates that a video encoder is used, and the specific type of video encoder is specified by the corresponding value.

[0141] Optionally, in some embodiments, the target identification information is information for indicating functions and algorithms that the decoding end needs to support.

[0142] In the embodiments of the present application, the target identification information can be understood as a profile identifier (ID). This target identification information is carried in the header field of the target bitstream. In other words, different profile IDs can implicitly indicate the encoding method of the displacement bitstream. Specifically, the method for indicating the displacement encoding method by using a profile ID is shown in Table 2.

[0143] Table 2:

[0144] profile_tier_level(){ Descriptor ptl_tier_flag u(1) ptl_profile_codec_group_idc u(7) ptl_profile_toolset_idc u(8) ptl_profile_reconstruction_idc u(8) ptl_reserved_zero_16bits u(16) ptl_max_decodes_idc u(4) ptl_reserved_0xfff_12bits u(12) ptl_level_idc u(8) ptl_num_sub_profiles u(6) ptl_extended_sub_profile_flag u(1) for(i=0;i<ptl_num_sub_profiles;i++) ptl_sub_profile_idc[i] u(v) ptl_toolset_constraints_present_flag u(1) if(ptl_toolset_constraints_present_flag) profile_toolset_constraints_information() }

[0145] ptl_profile_toolset_idc is defined in profile_tier_level() in the V3C parameter set and indicates the decoding tools supported by the bitstream that references this V3C parameter set. Taking Table 3 as an example, if its value is 0, it indicates that the vps_ext_disp_video_code_flag parameter can only take the value 0. If its value is 1, it indicates that the vps_ext_disp_video_code_flag parameter can take values ​​greater than 0, indicating that different encoders are supported.

[0146] Table 3:

[0147]

[0148]

[0149] Optionally, in some embodiments, the target code stream includes an auxiliary information code stream, and the auxiliary information code stream includes at least one of the following:

[0150] First field information, where the first field information is used to indicate the number of first vertices;

[0151] Second field information, where the second field information is used to indicate whether the absolute value of the kth component of the first displacement is equal to 0;

[0152] The third field information, wherein the second field information is used to indicate whether the kth component of the first displacement is greater than 0;

[0153] Fourth field information, the third field information is used to indicate whether the absolute value of the kth component of the first displacement is greater than 1;

[0154] Fifth field information, the fourth field information is used to represent the value of the kth component of the first displacement minus the first preset value;

[0155] The first vertex represents a vertex corresponding to a first mesh in a coding unit, the first displacement represents the vertex displacement of the vth vertex in the first vertices or the displacement of the vth vertex in the first vertices after shifting, and k and v are both positive integers.

[0156] Optionally, the size of the first preset value can be set according to actual needs. For example, in some embodiments, the first preset value can be 2.

[0157] In this embodiment, the V3C structure uses patches as the basic data description unit. A subgrid (i.e., the first grid described above) corresponds to a patch. A tile is an independently coded unit. A tile can contain one or more patches, i.e., a tile can correspond to one or more subgrids. Subgrids are obtained by dividing the original input grid. Therefore, the displacements corresponding to vertices in a subgrid are also described in terms of patches. Examples of possible syntax structures are shown in Table 4 below.

[0158] Table 4:

[0159]

[0160]

[0161] Among them, pdu_vertex_count_minus1[tileID][patchIdx] represents the number of vertices of the sub-mesh corresponding to the patch indexed by patchIdx in the tile indexed by tileID;

[0162] pdu_disp_abs_gt0[tileID][patchIdx][v][k] indicates whether the absolute value of the k-th component of the displacement (or coefficient after transformation, quantization, etc.) corresponding to the v-th vertex of the sub-mesh corresponding to the patch indexed by patchIdx in the tile indexed by tileID is equal to 0. When its value is 1, it means that its value is greater than 0, and when its value is 0, it means that it is equal to 0;

[0163] pdu_disp_sign[tileID][patchIdx][v][k] indicates whether the k-th component of the displacement (or coefficient after transformation, quantization, etc.) corresponding to the v-th vertex of the sub-mesh corresponding to the patch indexed by patchIdx in the tile indexed by tileID is greater than 0. Its value is 1 if the value is greater than 0; its value is 0 if the value is less than 0. If not specified, its value is 1;

[0164] pdu_disp_abs_gt1[tileID][patchIdx][v][k] indicates whether the absolute value of the k-th component of the displacement (or coefficient after transformation, quantization, etc.) corresponding to the v-th vertex of the sub-mesh corresponding to the patch indexed by patchIdx in the tile indexed by tileID is greater than 1. A value of 1 indicates that the value is greater than 1, and a value of 0 indicates that the value is less than 1. If not specified, the value is 0;

[0165] pdu_disp_abs_rem[tileID][patchIdx][v][k] represents the value of the k-th component of the displacement (or the coefficient after transformation, quantization, etc.) corresponding to the v-th vertex of the sub-mesh corresponding to the patch indexed by patchIdx in the tile indexed by tileID, minus 2. If not specified, its value is 0.

[0166] In this embodiment of the present application, pdu_vertex_count_minus1[tileID][patchIdx] can be understood as the above-mentioned first field information; pdu_disp_abs_gt0[tileID][patchIdx][v][k] can be understood as the above-mentioned second field information; pdu_disp_sign[tileID][patchIdx][v][k] can be understood as the above-mentioned third field information; pdu_disp_abs_gt1[tileID][patchIdx][v][k] can be understood as the above-mentioned fourth field information; pdu_disp_abs_rem[tileID][patchIdx][v][k] can be understood as the above-mentioned fifth field information.

[0167] It should be noted that, in the embodiment of the present application, the final displacement (or the corresponding quantization coefficient) is obtained by the above parameters: vectexDisp[tileID][patchIdx][v][k] = (pdu_disp_sign[tileID][patchIdx][v][k]?1:-1)*(pdu_disp_abs_gt0[tileID][patchIdx][v][k]+pdu_disp_abs_gt1[tileID][patchIdx][v][k]+pdu_disp_abs_rem[tileID][patchIdx][v][k]).

[0168] Optionally, in some embodiments, the target code stream includes an auxiliary information code stream, and the auxiliary information code stream includes at least one of the following:

[0169] First field information, where the first field information is used to indicate the number of first vertices;

[0170] Sixth field information, the sixth field information is used to indicate the number of iterations of subdivision processing of a first grid in a coding unit;

[0171] seventh field information, where the seventh field information is used to indicate the number of second vertices;

[0172] Eighth field information, the eighth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 0;

[0173] Ninth field information, the ninth field information is used to indicate whether the kth component of the second displacement is greater than 0;

[0174] The tenth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 1;

[0175] 11th field information, the 11th field information is used for the value of the kth component of the second shift minus the second preset value;

[0176] Among them, the first vertex represents a vertex corresponding to a first mesh in a coding unit, the second vertex represents a vertex generated by the j-th subdivision processing of a first mesh in a coding unit, the second displacement represents the vertex displacement of the v-th vertex in the second vertex or the displacement of the vertex displacement of the v-th vertex in the second vertex after shifting, and j, k and v are all positive integers.

[0177] Optionally, the size of the second preset value can be set according to actual needs. For example, in some embodiments, the second preset value can be 2.

[0178] In the embodiment of the present application, considering that the displacement is generated through the subdivision and deformation processes, each subdivision process will generate new vertices, and the displacement of the newly generated vertices may have different statistical characteristics from the displacement of the original vertices. During entropy coding, considering the statistical characteristics of the displacements at different levels, different entropy coding methods and different context information can be used according to the level. In addition to dividing by level, the three components of the displacement can also be encoded using different entropy encoders according to their respective statistical characteristics.

[0179] Alternatively, an entropy coding method for displacements at different subdivision levels is shown in Table 5.

[0180] Table 5:

[0181]

[0182] Among them, pdu_vertex_count_minus1[tileID][patchIdx] represents the number of vertices of the sub-mesh corresponding to the patch indexed by patchIdx in the tile indexed by tileID;

[0183] pdu_subdiv_iteration_count[tileID][patchIdx] represents the number of subdivision iterations of the sub-grid corresponding to the patch indexed by patchIdx in the tile indexed by tileID;

[0184] pdu_sub_vertex_count_minus1[tileID][patchIdx][j] represents the number of new vertices generated by the j-th subdivision of the sub-mesh corresponding to the patch indexed by patchIdx in the tile indexed by tileID;

[0185] pdu_disp_abs_gt0[tileID][patchIdx][j][v][k] indicates whether the absolute value of the k-th component of the displacement (or coefficient after transformation, quantization, etc.) corresponding to the v-th vertex in the newly generated vertex after the j-th subdivision of the submesh corresponding to the patch indexed by patchIdx in the tile indexed by tileID is greater than 0. When its value is 1, it indicates that its value is greater than 0, and when its value is 0, it indicates that it is equal to 0;

[0186] pdu_disp_sign[tileID][patchIdx][j][v][k] indicates whether the k-th component of the displacement (or coefficient after transformation, quantization, etc.) corresponding to the v-th vertex in the j-th subdivision of the submesh corresponding to the patch indexed by patchIdx in the tile indexed by tileID is greater than 0. A value of 1 indicates that the value is greater than 0; a value of 0 indicates that the value is less than 0. If not specified, the value is 1.

[0187] pdu_disp_abs_gt1[tileID][patchIdx][j][v][k] indicates whether the absolute value of the k-th component of the displacement (or coefficient after transformation, quantization, etc.) corresponding to the v-th vertex in the j-th subdivision of the submesh corresponding to the patch indexed by patchIdx in the tile indexed by tileID is greater than 1. A value of 1 indicates that the value is greater than 1, and a value of 0 indicates that the value is less than 1. If not specified, the value is 0;

[0188] pdu_disp_abs_rem[tileID][patchIdx][j][v][k] represents the value minus 2 of the k-th component of the displacement (or transformed, quantized coefficient, etc.) corresponding to the v-th vertex in the newly generated vertex from the j-th subdivision of the sub-mesh corresponding to the patch indexed as patchIdx in the tile indexed as tileID. If not specified, its value is 0.

[0189] In the embodiment of the present application, pdu_vertex_count_minus1[tileID][patchIdx] can be understood as the above-mentioned first field information; pdu_subdiv_iteration_count[tileID][patchIdx] can be understood as the above-mentioned sixth field information; pdu_sub_vertex_count_minus1[tileID][patchIdx][j] can be understood as the above-mentioned seventh field information; pdu_disp_abs_gt0[tileID][patchIdx][j][v][k] can be understood as the above-mentioned eighth field information; pdu_disp_sign[tileID][patchIdx][j][v][k] can be understood as the above-mentioned ninth field information; pdu_disp_abs_gt1[tileID][patchIdx][j][v][k] can be understood as the above-mentioned tenth field information; and pdu_disp_abs_rem[tileID][patchIdx][j][v][k] can be understood as the above-mentioned eleventh field information.

[0190] It should be noted that, in the embodiment of the present application, the encoding frame of the encoding end is as follows Figure 6 As shown, the dotted box can represent an optional encoding process, and the specific implementation of each encoding process can refer to the above embodiment and will not be repeated here.

[0191] Optionally, an embodiment of the present application further provides an encoding processing method, applied to an encoding end, comprising:

[0192] Determine a basic grid code stream based on the grid to be coded;

[0193] Displacement coding is performed on the first vertex displacement to obtain a displacement code stream, where the displacement coding method is an entropy coding method.

[0194] In the embodiment of the present application, the efficiency of displacement coding can be improved by using a quotient coding method to encode the displacement code stream. Specifically, the encoding process can also include texture image encoding and auxiliary information encoding. In other words, the method further includes:

[0195] Encoding the auxiliary information to obtain an auxiliary information code stream;

[0196] The texture image to be encoded is encoded to obtain a texture image bit stream.

[0197] The encoding process of the texture image encoding and the auxiliary information encoding may refer to the above embodiment and will not be described in detail here.

[0198] Optionally, since the V3C structure uses patches as the basic data description unit, a subgrid (i.e., the first grid mentioned above) corresponds to a patch, and a tile is an independently coded and decodable unit. A tile can contain one or more patches, that is, a tile can correspond to one or more subgrids, and the subgrids are obtained by dividing the original input grid. Therefore, in the embodiment of the present application, the displacement corresponding to the vertices in the subgrid is also described in units of patches, and an example of a possible grammatical structure is shown in Table 4 above. That is, in one embodiment, the above-mentioned auxiliary information code stream may include at least one of the following:

[0199] First field information, where the first field information is used to indicate the number of first vertices;

[0200] Second field information, where the second field information is used to indicate whether the absolute value of the kth component of the first displacement is equal to 0;

[0201] third field information, the third field information being used to indicate whether the absolute value of the kth component of the first displacement is greater than 0;

[0202] Fourth field information, the fourth field information is used to indicate whether the absolute value of the kth component of the first displacement is greater than 1;

[0203] Fifth field information, where the fifth field information is used to represent a value of the kth component of the first displacement minus a first preset value;

[0204] The first vertex represents a vertex corresponding to the first mesh in a coding unit, the first displacement represents the vertex displacement of the vth vertex in the first vertices or the displacement of the vth vertex in the first vertices after shifting, and k and v are both positive integers.

[0205] Optionally, considering that the displacement is generated through the subdivision processing and deformation processing process, each subdivision processing will generate new vertices, and the displacement of the newly generated vertices may have different statistical characteristics from the displacement of the original vertices. During entropy coding, considering the statistical characteristics between displacements at different levels, different entropy coding methods and different context information can be used according to the level. In addition to dividing by level, the three components of the displacement can also be encoded using different entropy encoders according to their respective statistical characteristics. An entropy coding method for displacements at different subdivision levels is shown in Table 5. That is, in one embodiment, the above-mentioned auxiliary information code stream may include at least one of the following:

[0206] First field information, where the first field information is used to indicate the number of first vertices;

[0207] Sixth field information, the sixth field information is used to indicate the number of iterations of subdivision processing of a first grid in a coding unit;

[0208] seventh field information, where the seventh field information is used to indicate the number of second vertices;

[0209] Eighth field information, the eighth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 0;

[0210] Ninth field information, the ninth field information is used to indicate whether the kth component of the second displacement is greater than 0;

[0211] The tenth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 1;

[0212] 11th field information, the 11th field information is used for the value of the kth component of the second shift minus the second preset value;

[0213] Among them, the first vertex represents a vertex corresponding to a first mesh in a coding unit, the second vertex represents a vertex generated by the j-th subdivision processing of a first mesh in a coding unit, the second displacement represents the vertex displacement of the v-th vertex in the second vertex or the displacement of the vertex displacement of the v-th vertex in the second vertex after shifting, and j, k and v are all positive integers.

[0214] Optionally, an embodiment of the present application further provides a coding processing method, applied to a decoding end, comprising:

[0215] Get the target stream;

[0216] Perform entropy decoding on the target code stream to obtain vertex displacements.

[0217] Optionally, since the V3C structure uses patch as the basic data description unit, a subgrid (i.e., the first grid mentioned above) corresponds to a patch, and a tile is an independently coded and decodable unit. A tile can contain one or more patches, that is, a tile can correspond to one or more subgrids, and the subgrid is obtained by dividing the original input grid. Therefore, in an embodiment of the present application, the displacement corresponding to the vertex in the subgrid is also described in units of patch, and an example of a possible grammatical structure is shown in Table 4 above. That is, in one embodiment, the target code stream also includes an auxiliary information code stream, and the above auxiliary information code stream may include at least one of the following:

[0218] First field information, where the first field information is used to indicate the number of first vertices;

[0219] Second field information, where the second field information is used to indicate whether the absolute value of the kth component of the first displacement is equal to 0;

[0220] third field information, the third field information being used to indicate whether the absolute value of the kth component of the first displacement is greater than 0;

[0221] Fourth field information, the fourth field information is used to indicate whether the absolute value of the kth component of the first displacement is greater than 1;

[0222] Fifth field information, where the fifth field information is used to represent a value of the kth component of the first displacement minus a first preset value;

[0223] The first vertex represents a vertex corresponding to the first mesh in a coding unit, the first displacement represents the vertex displacement of the vth vertex in the first vertices or the displacement of the vth vertex in the first vertices after shifting, and k and v are both positive integers.

[0224] Optionally, considering that the displacement is generated through the subdivision processing and deformation processing process, each subdivision processing will generate new vertices, and the displacement of the newly generated vertices may have different statistical characteristics from the displacement of the original vertices. During entropy coding, considering the statistical characteristics between displacements at different levels, different entropy coding methods and different context information can be used according to the level. In addition to the division by level, the three components of the displacement can also be encoded using different entropy encoders according to their respective statistical characteristics. An entropy coding method for displacements at different subdivision levels is shown in Table 5. That is, in one embodiment, the target code stream also includes an auxiliary information code stream, and the auxiliary information code stream may include at least one of the following:

[0225] First field information, where the first field information is used to indicate the number of first vertices;

[0226] Sixth field information, the sixth field information is used to indicate the number of iterations of subdivision processing of a first grid in a coding unit;

[0227] seventh field information, where the seventh field information is used to indicate the number of second vertices;

[0228] Eighth field information, the eighth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 0;

[0229] Ninth field information, the ninth field information is used to indicate whether the kth component of the second displacement is greater than 0;

[0230] The tenth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 1;

[0231] 11th field information, the 11th field information is used for the value of the kth component of the second shift minus the second preset value;

[0232] Among them, the first vertex represents a vertex corresponding to a first mesh in a coding unit, the second vertex represents a vertex generated by the j-th subdivision processing of a first mesh in a coding unit, the second displacement represents the vertex displacement of the v-th vertex in the second vertex or the displacement of the vertex displacement of the v-th vertex in the second vertex after shifting, and j, k and v are all positive integers.

[0233] Optionally, the embodiment of the present application further provides a decoding processing method, such as Figure 7 The decoding processing method includes:

[0234] Step 701: Receive a target bitstream, where the target bitstream includes a displacement bitstream and target identification information, where the target identification information is used to indicate whether the encoding method of the displacement bitstream is a video encoding method or an entropy encoding method.

[0235] Step 702, determining a target decoding method according to the target identification information;

[0236] Step 703: Decode the displacement code stream according to the target decoding method to obtain the third vertex displacement.

[0237] In the embodiment of the present application, the target indication information may indicate the encoding mode explicitly or implicitly. Since the encoding mode is indicated by the target identification information, the decoding end may use the corresponding decoding mode to obtain the vertex displacement, thereby improving the flexibility of displacement decoding.

[0238] Optionally, in some embodiments, when the target decoding mode is an entropy decoding mode, decoding the displacement code stream according to the target decoding mode to obtain the third vertex displacement includes any one of the following:

[0239] Inputting the displacement code stream into an entropy decoder for decoding to obtain a fourth vertex displacement, and performing displacement reconstruction processing on the fourth vertex displacement to obtain the third vertex displacement;

[0240] The displacement code stream is input into an entropy decoder for decoding to obtain the third vertex displacement.

[0241] In the embodiment of the present application, the above-mentioned displacement reconstruction process can be understood as the inverse processing of the above-mentioned displacement process, that is, the displacement reconstruction process includes inverse coefficient quantization and inverse wavelet transform.

[0242] Optionally, in some embodiments, the target code stream further includes a basic grid code stream and an auxiliary information code stream, and the method further includes:

[0243] Decoding the auxiliary information code stream to obtain auxiliary information;

[0244] Decoding the basic grid code stream based on the auxiliary information to obtain a reconstructed basic grid;

[0245] Subdividing the reconstructed basic grid based on the auxiliary information to obtain a reconstructed subdivided grid;

[0246] The reconstructed subdivision mesh is deformed based on the third vertex displacement to obtain a target decoding mesh.

[0247] Optionally, the auxiliary information includes at least one of the following: first coding information corresponding to the basic mesh code stream, a coding mode of the displacement code stream, and processing parameters for performing displacement processing on the first vertex displacement;

[0248] The first vertex displacement is used to determine the displacement code stream.

[0249] In the embodiment of the present application, the reconstructed base mesh can be subdivided based on the subdivision processing scheme, number of iterations, and other information in the auxiliary information to obtain a reconstructed subdivided mesh. After obtaining the reconstructed subdivided mesh, a deformed mesh can be reconstructed based on the reconstructed subdivided mesh and the third vertex displacement information to obtain a reconstructed deformed mesh, thereby obtaining a target decoded mesh.

[0250] Optionally, inputting the displacement code stream into an entropy decoder for decoding to obtain a fourth vertex displacement, and performing displacement reconstruction processing on the fourth vertex displacement to obtain the third vertex displacement includes:

[0251] In a case where the auxiliary information includes processing parameters for performing displacement processing on the first vertex displacement, inputting the displacement code stream into an entropy decoder for decoding to obtain a fourth vertex displacement;

[0252] Performing displacement reconstruction processing on the fourth vertex displacement according to the processing parameters to obtain the third vertex displacement;

[0253] The first vertex displacement is used to determine the displacement code stream.

[0254] Optionally, inputting the displacement code stream into an entropy decoder for decoding to obtain the third vertex displacement includes:

[0255] In a case where the auxiliary information does not include processing parameters for performing displacement processing on the first vertex displacement, inputting the displacement code stream into an entropy decoder for decoding to obtain the third vertex displacement;

[0256] The first vertex displacement is used to determine the displacement code stream.

[0257] Optionally, the displacement processing includes wavelet transform and coefficient quantization.

[0258] Optionally, the target code stream includes an auxiliary information code stream determined based on auxiliary information, the target identification information is part of the auxiliary information code stream, and the auxiliary information is used to assist a decoding end in decoding.

[0259] Optionally, the target identification information is information for indicating functions and algorithms that the decoding end needs to support.

[0260] Optionally, the target code stream further includes an auxiliary information code stream, and the auxiliary information code stream includes at least one of the following:

[0261] First field information, where the first field information is used to indicate the number of first vertices;

[0262] Second field information, where the second field information is used to indicate whether the absolute value of the kth component of the first displacement is equal to 0;

[0263] third field information, the third field information being used to indicate whether the absolute value of the kth component of the first displacement is greater than 0;

[0264] Fourth field information, the fourth field information is used to indicate whether the absolute value of the kth component of the first displacement is greater than 1;

[0265] Fifth field information, where the fifth field information is used to represent a value of the kth component of the first displacement minus a first preset value;

[0266] The first vertex represents a vertex corresponding to a first mesh in a coding unit, the first displacement represents the vertex displacement of the vth vertex in the first vertices or the displacement of the vth vertex in the first vertices after shifting, and k and v are both positive integers.

[0267] Optionally, the target code stream further includes an auxiliary information code stream, and the auxiliary information code stream includes at least one of the following:

[0268] First field information, where the first field information is used to indicate the number of first vertices;

[0269] Sixth field information, the sixth field information is used to indicate the number of iterations of subdivision processing of a first grid in a coding unit;

[0270] seventh field information, where the seventh field information is used to indicate the number of second vertices;

[0271] Eighth field information, the eighth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 0;

[0272] Ninth field information, the ninth field information is used to indicate whether the kth component of the second displacement is greater than 0;

[0273] The tenth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 1;

[0274] 11th field information, the 11th field information is used for the value of the kth component of the second shift minus the second preset value;

[0275] Among them, the first vertex represents a vertex corresponding to a first mesh in a coding unit, the second vertex represents a vertex generated by the j-th subdivision processing of a first mesh in a coding unit, the second displacement represents the vertex displacement of the v-th vertex in the second vertex or the displacement of the vertex displacement of the v-th vertex in the second vertex after shifting, and j, k and v are all positive integers.

[0276] It should be noted that, in the embodiment of the present application, the decoding framework of the decoding end is as follows Figure 8 As shown, the dotted box can represent an optional decoding process, and the decoding process is described in detail below.

[0277] 1. Auxiliary information decoding.

[0278] The decoding end first determines the decoding scheme based on the auxiliary information, which mainly includes the displacement coding method, indicating whether the displacement is encoded by the video encoder or the entropy encoder; the static grid encoder type, which guides the decoding end to use the corresponding static grid decoder; the video encoder type, which guides the decoding end to use the corresponding video decoder; the subdivision processing scheme, that is, the scheme for subdivision processing of the basic grid in the reconstructed deformed grid, and the subdivision processing scheme of the encoder and decoder should be consistent; there are also optional spatial domain displacement transformation schemes, coefficient arrangement schemes, etc.

[0279] 2. Base Mesh Decoding. Base mesh decoding can be divided into intra-frame mode and inter-frame mode. In intra-frame mode, this module decodes the input base mesh bitstream using the decoder corresponding to the static mesh encoder indicated by the auxiliary information. The decoded output is a 3D mesh containing geometric information, connectivity, and texture coordinate information. In inter-frame mode, this module is responsible for decoding the motion vectors corresponding to the vertices and then reconstructing the base mesh of the current frame based on the reference frame.

[0280] 3. Displacement decoding. During displacement decoding, the decoding method for the displacement needs to be determined based on the auxiliary information identifier. If the auxiliary information indicates that the displacement information is encoded by the video encoder, the decoding end calls the corresponding video decoder to decode the displacement code stream; if the auxiliary information indicates that the displacement information is encoded by the entropy encoder, the entropy decoder is directly used for decoding.

[0281] For the decoded displacement information, displacement reconstruction is also required to obtain the displacement vectors corresponding to the reconstructed subdivision mesh vertices. The displacement reconstruction operation mainly uses the quantization parameters and transform parameters indicated by the auxiliary information to perform inverse quantization and inverse transformation on the decoded displacement information, i.e., the wavelet coefficients. For the information decoded by the video, the corresponding displacement information must first be extracted from the two-dimensional image according to the arrangement method of the encoder.

[0282] 4. Subdivision processing: This subdivision processing operation is the same as the subdivision processing operation at the encoding end, and the subdivision processing method and number of iterations of the base grid are indicated by auxiliary information.

[0283] 5. Reconstruct the deformed mesh. After the base mesh and displacement vectors are decoded and reconstructed, the deformed mesh is reconstructed based on these two parts. Add the corresponding displacement vector to each vertex of the subdivided mesh, as shown in Equation 7:

[0284] deformedmesh[i].v[k]=subdivmesh[i].v[k]+displacement[k] (7);

[0285] Where subdivmesh[i].v[k] represents the geometric coordinates of the k-th vertex after subdivision of the base mesh of the current frame (index i), displacement[k] represents the spatial displacement vector corresponding to the k-th vertex, and deformedmesh[i].v[k] represents the geometric coordinates of the k-th vertex after subdivision and deformation of the current frame.

[0286] 6. Displacement map decoding. The texture map decoder is responsible for decoding the texture map code stream, which is decoded by the video decoder indicated in the auxiliary information. An optional color space conversion is performed on it to obtain the same color space as the input texture at the encoder. Figure 1 The consistent image format is obtained to obtain the final decoded output texture map.

[0287] After completing the above process, the target decoding grid reconstructed by the decoding end and the corresponding attribute map are used as input for corresponding processing.

[0288] The encoding processing method provided in the embodiment of the present application can be executed by an encoding processing device. In the embodiment of the present application, the encoding processing device provided in the embodiment of the present application is described by taking the encoding processing method executed by the encoding processing device as an example.

[0289] Reference Figure 9 , the embodiment of the present application also provides a coding processing device, such as Figure 9 As shown, the encoding processing device 900 includes:

[0290] A first processing module 901 is configured to determine a basic grid code stream based on a grid to be encoded;

[0291] A first encoding module 902 is configured to perform displacement encoding on a first vertex displacement to obtain a displacement code stream, wherein the first vertex displacement is obtained by adjusting a displacement order of a reconstructed basic mesh obtained by reconstructing the basic mesh code stream according to the displacement information, wherein the displacement information is obtained by subdividing and deforming a first mesh, and the first mesh is obtained by mesh simplification and mesh parameterization of the mesh to be encoded;

[0292] The generation module 903 is used to generate a target code stream based on the texture map to be encoded corresponding to the grid to be encoded, the basic grid code stream and the displacement code stream, wherein the target code stream includes target identification information, and the target identification information is used to indicate whether the encoding method of the displacement code stream is a video encoding method or an entropy encoding method.

[0293] Optionally, when the encoding mode is an entropy encoding mode, the first encoding module 902 is specifically configured to perform any one of the following:

[0294] Performing displacement processing on the first vertex displacement to obtain a second vertex displacement, and inputting the second vertex displacement into an entropy encoder for displacement encoding to obtain the displacement code stream;

[0295] The first vertex displacement is input into an entropy encoder for displacement coding to obtain the displacement code stream.

[0296] Optionally, the displacement processing includes wavelet transform and coefficient quantization.

[0297] Optionally, the generating module 903 includes:

[0298] An encoding unit, configured to encode auxiliary information to obtain an auxiliary information code stream, wherein the auxiliary information is used to assist a decoding end in decoding;

[0299] a conversion unit, configured to perform texture conversion based on the texture map to be encoded and the grid to be encoded to obtain a target texture map, and to perform compression encoding on the target texture map to obtain a texture map code stream;

[0300] The mixing unit is used to mix the basic grid code stream, the displacement code stream, the auxiliary information code stream and the texture map code stream to obtain the target code stream.

[0301] Optionally, the auxiliary information includes at least one of the following: first coding information corresponding to the basic mesh code stream, second coding information corresponding to the texture map code stream, coding mode of the displacement code stream, and processing parameters for displacement processing of the first vertex displacement.

[0302] Optionally, the target code stream includes an auxiliary information code stream determined based on auxiliary information, the target identification information is part of the auxiliary information code stream, and the auxiliary information is used to assist a decoding end in decoding.

[0303] Optionally, the target identification information is information for indicating functions and algorithms that the decoding end needs to support.

[0304] Optionally, the target code stream includes an auxiliary information code stream, and the auxiliary information code stream includes at least one of the following:

[0305] First field information, where the first field information is used to indicate the number of first vertices;

[0306] Second field information, where the second field information is used to indicate whether the absolute value of the kth component of the first displacement is equal to 0;

[0307] third field information, the third field information being used to indicate whether the absolute value of the kth component of the first displacement is greater than 0;

[0308] Fourth field information, the fourth field information is used to indicate whether the absolute value of the kth component of the first displacement is greater than 1;

[0309] Fifth field information, where the fifth field information is used to represent a value of the kth component of the first displacement minus a first preset value;

[0310] The first vertex represents a vertex corresponding to a first mesh in a coding unit, the first displacement represents the vertex displacement of the vth vertex in the first vertices or the displacement of the vth vertex in the first vertices after shifting, and k and v are both positive integers.

[0311] Optionally, the target code stream includes an auxiliary information code stream, and the auxiliary information code stream includes at least one of the following:

[0312] First field information, where the first field information is used to indicate the number of first vertices;

[0313] Sixth field information, the sixth field information is used to indicate the number of iterations of subdivision processing of a first grid in a coding unit;

[0314] seventh field information, where the seventh field information is used to indicate the number of second vertices;

[0315] Eighth field information, the eighth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 0;

[0316] Ninth field information, the ninth field information is used to indicate whether the kth component of the second displacement is greater than 0;

[0317] The tenth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 1;

[0318] 11th field information, the 11th field information is used for the value of the kth component of the second shift minus the second preset value;

[0319] Among them, the first vertex represents a vertex corresponding to a first mesh in a coding unit, the second vertex represents a vertex generated by the j-th subdivision processing of a first mesh in a coding unit, the second displacement represents the vertex displacement of the v-th vertex in the second vertex or the displacement of the vertex displacement of the v-th vertex in the second vertex after shifting, and j, k and v are all positive integers.

[0320] The decoding processing method provided in the embodiment of the present application can be executed by a decoding processing device. In the embodiment of the present application, the decoding processing device provided in the embodiment of the present application is described by taking the decoding processing device executing the encoding processing method as an example.

[0321] Reference Figure 10 , the embodiment of the present application also provides a decoding processing device, such as Figure 10 As shown, the decoding processing device 1000 includes:

[0322] The receiving module 1001 is configured to receive a target code stream, wherein the target code stream includes a displacement code stream and target identification information, wherein the target identification information is used to indicate whether the coding mode of the displacement code stream is a video coding mode or an entropy coding mode;

[0323] A determination module 1002 is configured to determine a target decoding mode according to the target identification information;

[0324] The first decoding module 1003 is configured to decode the displacement code stream according to the target decoding mode to obtain a third vertex displacement.

[0325] Optionally, when the target decoding mode is an entropy decoding mode, the first decoding module 1003 is specifically configured to perform any one of the following:

[0326] Inputting the displacement code stream into an entropy decoder for decoding to obtain a fourth vertex displacement, and performing displacement reconstruction processing on the fourth vertex displacement to obtain the third vertex displacement;

[0327] The displacement code stream is input into an entropy decoder for decoding to obtain the third vertex displacement.

[0328] Optionally, the displacement reconstruction process includes inverse coefficient quantization and inverse wavelet transform.

[0329] Optionally, the target code stream further includes a basic grid code stream and an auxiliary information code stream, and the decoding processing device 1000 further includes: a grid reconstruction module

[0330] The first decoding module 1003 is further configured to decode the auxiliary information code stream to obtain auxiliary information;

[0331] A reconstruction module is configured to decode the basic grid code stream based on the auxiliary information to obtain a reconstructed basic grid; subdivide the reconstructed basic grid based on the auxiliary information to obtain a reconstructed subdivided grid; and deform the reconstructed subdivided grid based on the third vertex displacement to obtain a target decoded grid.

[0332] Optionally, the auxiliary information includes at least one of the following: first coding information corresponding to the basic mesh code stream, a coding mode of the displacement code stream, and processing parameters for performing displacement processing on the first vertex displacement;

[0333] The first vertex displacement is used to determine the displacement code stream.

[0334] Optionally, the first decoding module 1003 is specifically used to: when the auxiliary information includes processing parameters for performing displacement processing on the first vertex displacement, input the displacement code stream into the entropy decoder for decoding to obtain a fourth vertex displacement; perform displacement reconstruction processing on the fourth vertex displacement according to the processing parameters to obtain the third vertex displacement; wherein the first vertex displacement is used to determine the displacement code stream.

[0335] Optionally, the first decoding module 1003 is specifically used to: when the auxiliary information does not include processing parameters for performing displacement processing on the first vertex displacement, input the displacement code stream into the entropy decoder for decoding to obtain the third vertex displacement; wherein the first vertex displacement is used to determine the displacement code stream.

[0336] Optionally, the displacement processing includes wavelet transform and coefficient quantization.

[0337] Optionally, the target code stream includes an auxiliary information code stream determined based on auxiliary information, the target identification information is part of the auxiliary information code stream, and the auxiliary information is used to assist a decoding end in decoding.

[0338] Optionally, the target identification information is information for indicating functions and algorithms that the decoding end needs to support.

[0339] Optionally, the target code stream further includes an auxiliary information code stream, and the auxiliary information code stream includes at least one of the following:

[0340] First field information, where the first field information is used to indicate the number of first vertices;

[0341] Second field information, where the second field information is used to indicate whether the absolute value of the kth component of the first displacement is equal to 0;

[0342] third field information, the third field information being used to indicate whether the absolute value of the kth component of the first displacement is greater than 0;

[0343] Fourth field information, the fourth field information is used to indicate whether the absolute value of the kth component of the first displacement is greater than 1;

[0344] Fifth field information, where the fifth field information is used to represent a value of the kth component of the first displacement minus a first preset value;

[0345] The first vertex represents a vertex corresponding to a first mesh in a coding unit, the first displacement represents the vertex displacement of the vth vertex in the first vertices or the displacement of the vth vertex in the first vertices after shifting, and k and v are both positive integers.

[0346] Optionally, the target code stream further includes an auxiliary information code stream, and the auxiliary information code stream includes at least one of the following:

[0347] First field information, where the first field information is used to indicate the number of first vertices;

[0348] Sixth field information, the sixth field information is used to indicate the number of iterations of subdivision processing of a first grid in a coding unit;

[0349] seventh field information, where the seventh field information is used to indicate the number of second vertices;

[0350] Eighth field information, the eighth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 0;

[0351] Ninth field information, the ninth field information is used to indicate whether the kth component of the second displacement is greater than 0;

[0352] The tenth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 1;

[0353] 11th field information, the 11th field information is used for the value of the kth component of the second shift minus the second preset value;

[0354] Among them, the first vertex represents a vertex corresponding to a first mesh in a coding unit, the second vertex represents a vertex generated by the j-th subdivision processing of a first mesh in a coding unit, the second displacement represents the vertex displacement of the v-th vertex in the second vertex or the displacement of the vertex displacement of the v-th vertex in the second vertex after shifting, and j, k and v are all positive integers.

[0355] The encoding processing device and decoding processing device in the embodiments of the present application can be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or chip. The electronic device can be a terminal or other device other than a terminal. For example, the terminal can include but is not limited to the types of terminal 11 listed above, and other devices can be servers, network attached storage (NAS), etc., which are not specifically limited in the embodiments of the present application.

[0356] The encoding processing device and decoding processing device provided in the embodiments of the present application can achieve Figures 3 to 8 The various processes implemented by the method embodiment achieve the same technical effect and are not described here again to avoid repetition.

[0357] Optionally, an embodiment of the present application further provides a coding processing device, the coding processing device including:

[0358] A second processing module is used to determine a basic grid code stream based on the grid to be encoded;

[0359] The second encoding module is used to perform displacement encoding on the first vertex displacement to obtain a displacement code stream, where the displacement encoding method is an entropy encoding method.

[0360] Optionally, the second encoding module is further configured to encode the auxiliary information to obtain an auxiliary information code stream.

[0361] Optionally, the auxiliary information code stream includes at least one of the following:

[0362] First field information, where the first field information is used to indicate the number of first vertices;

[0363] Second field information, where the second field information is used to indicate whether the absolute value of the kth component of the first displacement is equal to 0;

[0364] third field information, the third field information being used to indicate whether the absolute value of the kth component of the first displacement is greater than 0;

[0365] Fourth field information, the fourth field information is used to indicate whether the absolute value of the kth component of the first displacement is greater than 1;

[0366] Fifth field information, where the fifth field information is used to represent a value of the kth component of the first displacement minus a first preset value;

[0367] The first vertex represents a vertex corresponding to a first mesh in a coding unit, the first displacement represents the vertex displacement of the vth vertex in the first vertices or the displacement of the vth vertex in the first vertices after shifting, and k and v are both positive integers.

[0368] Optionally, the auxiliary information code stream includes at least one of the following:

[0369] First field information, where the first field information is used to indicate the number of first vertices;

[0370] Sixth field information, the sixth field information is used to indicate the number of iterations of subdivision processing of a first grid in a coding unit;

[0371] seventh field information, where the seventh field information is used to indicate the number of second vertices;

[0372] Eighth field information, the eighth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 0;

[0373] Ninth field information, the ninth field information is used to indicate whether the kth component of the second displacement is greater than 0;

[0374] The tenth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 1;

[0375] 11th field information, the 11th field information is used for the value of the kth component of the second shift minus the second preset value;

[0376] Among them, the first vertex represents a vertex corresponding to a first mesh in a coding unit, the second vertex represents a vertex generated by the j-th subdivision processing of a first mesh in a coding unit, the second displacement represents the vertex displacement of the v-th vertex in the second vertex or the displacement of the vertex displacement of the v-th vertex in the second vertex after shifting, and j, k and v are all positive integers.

[0377] Optionally, an embodiment of the present application further provides a decoding processing device, the decoding processing device comprising:

[0378] Acquisition module, used to obtain the target code stream;

[0379] The second decoding module is configured to perform entropy decoding on the target code stream to obtain vertex displacements.

[0380] Optionally, the target code stream further includes an auxiliary information code stream, and the auxiliary information code stream includes at least one of the following:

[0381] First field information, where the first field information is used to indicate the number of first vertices;

[0382] Second field information, where the second field information is used to indicate whether the absolute value of the kth component of the first displacement is equal to 0;

[0383] third field information, the third field information being used to indicate whether the absolute value of the kth component of the first displacement is greater than 0;

[0384] Fourth field information, the fourth field information is used to indicate whether the absolute value of the kth component of the first displacement is greater than 1;

[0385] Fifth field information, where the fifth field information is used to represent a value of the kth component of the first displacement minus a first preset value;

[0386] The first vertex represents a vertex corresponding to a first mesh in a coding unit, the first displacement represents the vertex displacement of the vth vertex in the first vertices or the displacement of the vth vertex in the first vertices after shifting, and k and v are both positive integers.

[0387] Optionally, the target code stream further includes an auxiliary information code stream, and the auxiliary information code stream includes:

[0388] First field information, where the first field information is used to indicate the number of first vertices;

[0389] Sixth field information, the sixth field information is used to indicate the number of iterations of subdivision processing of a first grid in a coding unit;

[0390] seventh field information, where the seventh field information is used to indicate the number of second vertices;

[0391] Eighth field information, the eighth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 0;

[0392] Ninth field information, the ninth field information is used to indicate whether the kth component of the second displacement is greater than 0;

[0393] The tenth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 1;

[0394] 11th field information, the 11th field information is used for the value of the kth component of the second shift minus the second preset value;

[0395] Among them, the first vertex represents a vertex corresponding to a first mesh in a coding unit, the second vertex represents a vertex generated by the j-th subdivision processing of a first mesh in a coding unit, the second displacement represents the vertex displacement of the v-th vertex in the second vertex or the displacement of the vertex displacement of the v-th vertex in the second vertex after shifting, and j, k and v are all positive integers.

[0396] Optional, such as Figure 11 As shown, an embodiment of the present application further provides a communication device 1100, including a processor 1101 and a memory 1102, wherein the memory 1102 stores a program or instruction that can be run on the processor 1101. For example, when the communication device 1100 is an encoding end device, the program or instruction is executed by the processor 1101 to implement the various steps of the above-mentioned encoding processing method embodiment and can achieve the same technical effect. When the communication device 1100 is a decoding end device, the program or instruction is executed by the processor 1101 to implement the various steps of the above-mentioned decoding processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0397] An embodiment of the present application further provides an electronic device, including a processor and a communication interface, wherein:

[0398] When the electronic device is an encoding end, the processor is configured to determine a basic mesh bitstream based on a mesh to be encoded; perform displacement encoding on a first vertex displacement to obtain a displacement bitstream, wherein the first vertex displacement is obtained by adjusting a displacement order of a reconstructed basic mesh obtained based on the basic mesh bitstream according to displacement information, wherein the displacement information is obtained based on subdivision and deformation processing of a first mesh, wherein the first mesh is obtained based on mesh simplification and mesh parameterization of the mesh to be encoded; and generate a target bitstream based on a texture map to be encoded corresponding to the mesh to be encoded, the basic mesh bitstream, and the displacement bitstream, wherein the target bitstream includes target identification information, wherein the target identification information is used to indicate whether an encoding mode of the displacement bitstream is a video encoding mode or an entropy encoding mode.

[0399] When the electronic device is a decoding end, the communication interface is used to receive a target code stream, the target code stream including a displacement code stream and target identification information, the target identification information being used to indicate whether the encoding mode of the displacement code stream is a video encoding mode or an entropy encoding mode; the processor is used to determine a target decoding mode based on the target identification information; and decode the displacement code stream according to the target decoding mode to obtain a third vertex displacement;

[0400] or,

[0401] When the electronic device is an encoding end, the processor is configured to determine a basic grid code stream based on the grid to be encoded; perform displacement encoding on the first vertex displacement to obtain a displacement code stream, wherein the displacement encoding method is an entropy encoding method;

[0402] When the electronic device is a decoding end, the processor is configured to obtain a target code stream; and perform entropy decoding on the target code stream to obtain vertex displacements.

[0403] This electronic device embodiment corresponds to the above-mentioned method embodiment on the codec end device side. The various implementation processes and implementation methods of the above-mentioned method embodiment are applicable to this electronic device embodiment and can achieve the same technical effects. Specifically, Figure 12 A schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0404] The electronic device 1200 includes but is not limited to: a radio frequency unit 1201, a network module 1202, an audio output unit 1203, an input unit 1204, a sensor 1205, a display unit 1206, a user input unit 1207, an interface unit 1208, a memory 1209 and at least some of the components of the processor 1210.

[0405] Those skilled in the art will understand that the electronic device 1200 may also include a power source (such as a battery) to power each component, and the power source may be logically connected to the processor 1210 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system. Figure 12 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be repeated here.

[0406] It should be understood that in an embodiment of the present application, the input unit 1204 may include a graphics processing unit (GPU) 12041 and a microphone 12042, and the graphics processor 12041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 1206 may include a display panel 12061, and the display panel 12061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1207 includes a touch panel 12071 and at least one of other input devices 12072. The touch panel 12071 is also called a touch screen. The touch panel 12071 may include two parts: a touch detection device and a touch controller. Other input devices 12072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0407] In the embodiment of the present application, after receiving downlink data from a network-side device, the RF unit 1201 may transmit the data to the processor 1210 for processing. Furthermore, the RF unit 1201 may send uplink data to the network-side device. Typically, the RF unit 1201 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and the like.

[0408] The memory 1209 can be used to store software programs or instructions and various data. The memory 1209 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1209 may include a volatile memory or a non-volatile memory, or the memory 1209 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 1209 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0409] Processor 1210 may include one or more processing units. Optionally, processor 1210 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 1210.

[0410] When the electronic device is an encoding end, the processor 1210 is configured to determine a basic mesh bitstream based on a mesh to be encoded; perform displacement encoding on a first vertex displacement to obtain a displacement bitstream, wherein the first vertex displacement is obtained by adjusting a displacement order of a reconstructed basic mesh obtained based on the basic mesh bitstream according to displacement information, wherein the displacement information is obtained based on subdivision and deformation processing of a first mesh, wherein the first mesh is obtained based on mesh simplification and mesh parameterization of the mesh to be encoded; and generate a target bitstream based on a texture map to be encoded corresponding to the mesh to be encoded, the basic mesh bitstream, and the displacement bitstream, wherein the target bitstream includes target identification information, wherein the target identification information is used to indicate whether the encoding mode of the displacement bitstream is a video encoding mode or an entropy encoding mode.

[0411] When the electronic device is a decoding end, the radio frequency unit 1201 is configured to receive a target code stream, where the target code stream includes a displacement code stream and target identification information, where the target identification information is used to indicate whether the encoding mode of the displacement code stream is a video encoding mode or an entropy encoding mode; the processor 1210 is configured to determine a target decoding mode based on the target identification information; and decode the displacement code stream according to the target decoding mode to obtain a third vertex displacement.

[0412] or,

[0413] When the electronic device is an encoding end, the processor 1210 is configured to determine a basic grid code stream based on the grid to be encoded; perform displacement coding on the first vertex displacement to obtain a displacement code stream, wherein the displacement coding method is an entropy coding method;

[0414] When the electronic device is a decoding end, the processor 1210 is configured to obtain a target bitstream; and perform entropy decoding on the target bitstream to obtain vertex displacements.

[0415] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned encoding processing method or decoding processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0416] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0417] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned encoding processing method or decoding processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0418] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0419] An embodiment of the present application further provides a computer program / program product, which is stored in a storage medium. The computer program / program product is executed by at least one processor to implement the various processes of the above-mentioned encoding processing method or decoding processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0420] The embodiment of the present application also provides a video encoding and decoding system, including: an encoding end device and a decoding end device, wherein the encoding end device is used to perform the following Figure 3 And each process of each method embodiment of the above encoding end device, the decoding end device is used to perform the following Figure 7 And each process of each method embodiment of the above-mentioned decoding end device can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0421] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0422] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0423] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A coding processing method, applied to a coding end, characterized in that: include: Determine a basic grid code stream based on the grid to be coded; performing displacement encoding on a first vertex displacement to obtain a displacement code stream, wherein the first vertex displacement is obtained by adjusting a displacement order of a reconstructed basic mesh obtained by reconstructing the basic mesh code stream according to displacement information, the displacement information being obtained based on subdividing and deforming a first mesh, and the first mesh being obtained based on mesh simplification and mesh parameterization of the mesh to be encoded; A target code stream is generated based on the texture map to be encoded corresponding to the grid to be encoded, the basic grid code stream and the displacement code stream, wherein the target code stream includes target identification information, and the target identification information is used to indicate that the encoding method of the displacement code stream is a video encoding method or an entropy encoding method.

2. The method according to claim 1, characterized in that When the encoding mode is an entropy encoding mode, performing displacement encoding on the first vertex displacement to obtain a displacement code stream includes any one of the following: Performing displacement processing on the first vertex displacement to obtain a second vertex displacement, and inputting the second vertex displacement into an entropy encoder for displacement encoding to obtain the displacement code stream; The first vertex displacement is input into an entropy encoder for displacement coding to obtain the displacement code stream.

3. The method according to claim 2, characterized in that The displacement processing includes wavelet transform and coefficient quantization.

4. The method according to any one of claims 1 to 3, characterized in that The target code stream includes an auxiliary information code stream determined based on auxiliary information. The target identification information is part of the information in the auxiliary information code stream. The auxiliary information is used to assist a decoding end in decoding.

5. The method according to any one of claims 1 to 3, characterized in that The target identification information is information used to indicate the functions and algorithms that the decoding end needs to support.

6. The method according to any one of claims 1 to 5, characterized in that The target code stream includes an auxiliary information code stream, and the auxiliary information code stream includes at least one of the following: First field information, where the first field information is used to indicate the number of first vertices; Second field information, where the second field information is used to indicate whether the absolute value of the kth component of the first displacement is equal to 0; third field information, the third field information being used to indicate whether the absolute value of the kth component of the first displacement is greater than 0; Fourth field information, the fourth field information is used to indicate whether the absolute value of the kth component of the first displacement is greater than 1; Fifth field information, where the fifth field information is used to represent a value of the kth component of the first displacement minus a first preset value; The first vertex represents a vertex corresponding to the first mesh in a coding unit, the first displacement represents the vertex displacement of the vth vertex in the first vertices or the displacement of the vth vertex in the first vertices after shifting, and k and v are both positive integers.

7. The method according to any one of claims 1 to 5, characterized in that The target code stream includes an auxiliary information code stream, and the auxiliary information code stream includes at least one of the following: First field information, where the first field information is used to indicate the number of first vertices; Sixth field information, the sixth field information is used to indicate the number of iterations of subdivision processing of a first grid in a coding unit; seventh field information, where the seventh field information is used to indicate the number of second vertices; Eighth field information, the eighth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 0; Ninth field information, the ninth field information is used to indicate whether the kth component of the second displacement is greater than 0; The tenth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 1; 11th field information, the 11th field information is used for the value of the kth component of the second shift minus the second preset value; In which, the first vertex represents a vertex corresponding to the first mesh in a coding unit, the second vertex represents a vertex generated by the j-th subdivision processing of the first mesh in a coding unit, the second displacement represents the vertex displacement of the v-th vertex in the second vertex or the displacement of the vertex displacement of the v-th vertex in the second vertex after shifting, and j, k and v are all positive integers.

8. A decoding processing method, applied to a decoding end, characterized in that: include: receiving a target code stream, the target code stream including a displacement code stream and target identification information, the target identification information being used to indicate whether a coding mode of the displacement code stream is a video coding mode or an entropy coding mode; determining a target decoding mode according to the target identification information; The displacement code stream is decoded according to the target decoding mode to obtain a third vertex displacement.

9. The method according to claim 8, characterized in that When the target decoding mode is an entropy decoding mode, decoding the displacement code stream according to the target decoding mode to obtain the third vertex displacement includes any one of the following: Inputting the displacement code stream into an entropy decoder for decoding to obtain a fourth vertex displacement, and performing displacement reconstruction processing on the fourth vertex displacement to obtain the third vertex displacement; The displacement code stream is input into an entropy decoder for decoding to obtain the third vertex displacement.

10. The method according to claim 9, characterized in that The displacement reconstruction process includes inverse coefficient quantization and inverse wavelet transform.

11. The method according to claim 9, characterized in that Inputting the displacement code stream into an entropy decoder for decoding to obtain a fourth vertex displacement, and performing displacement reconstruction processing on the fourth vertex displacement to obtain the third vertex displacement includes: In a case where the auxiliary information includes processing parameters for performing displacement processing on the first vertex displacement, inputting the displacement code stream into an entropy decoder for decoding to obtain a fourth vertex displacement; Performing displacement reconstruction processing on the fourth vertex displacement according to the processing parameters to obtain the third vertex displacement; The auxiliary information is used to assist a decoding end in decoding, and the first vertex displacement is used to determine the displacement code stream.

12. The method according to claim 11, characterized in that Inputting the displacement code stream into an entropy decoder for decoding to obtain the third vertex displacement includes: In a case where the auxiliary information does not include processing parameters for performing displacement processing on the first vertex displacement, inputting the displacement code stream into an entropy decoder for decoding to obtain the third vertex displacement; The first vertex displacement is used to determine the displacement code stream.

13. The method according to claim 11 or 12, characterized in that The displacement processing includes wavelet transform and coefficient quantization.

14. The method according to any one of claims 8 to 13, characterized in that The target code stream includes an auxiliary information code stream determined based on auxiliary information. The target identification information is part of the information in the auxiliary information code stream. The auxiliary information is used to assist a decoding end in decoding.

15. The method according to any one of claims 8 to 13, characterized in that The target identification information is information used to indicate the functions and algorithms that the decoding end needs to support.

16. The method according to any one of claims 8 to 10, characterized in that The target code stream further includes an auxiliary information code stream, and the auxiliary information code stream includes at least one of the following: First field information, where the first field information is used to indicate the number of first vertices; Second field information, where the second field information is used to indicate whether the absolute value of the kth component of the first displacement is equal to 0; third field information, the third field information being used to indicate whether the absolute value of the kth component of the first displacement is greater than 0; Fourth field information, the fourth field information is used to indicate whether the absolute value of the kth component of the first displacement is greater than 1; Fifth field information, where the fifth field information is used to represent a value of the kth component of the first displacement minus a first preset value; The first vertex represents a vertex corresponding to a first mesh in a coding unit, the first displacement represents the vertex displacement of the vth vertex in the first vertices or the displacement of the vth vertex in the first vertices after shifting, and k and v are both positive integers.

17. The method according to any one of claims 8 to 10, characterized in that The target code stream also includes an auxiliary information code stream, and the auxiliary information code stream includes: First field information, where the first field information is used to indicate the number of first vertices; Sixth field information, the sixth field information is used to indicate the number of iterations of subdivision processing of a first grid in a coding unit; seventh field information, where the seventh field information is used to indicate the number of second vertices; Eighth field information, the eighth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 0; Ninth field information, the ninth field information is used to indicate whether the kth component of the second displacement is greater than 0; The tenth field information is used to indicate whether the absolute value of the kth component of the second displacement is greater than 1; 11th field information, the 11th field information is used for the value of the kth component of the second shift minus the second preset value; Among them, the first vertex represents a vertex corresponding to a first mesh in a coding unit, the second vertex represents a vertex generated by the j-th subdivision processing of a first mesh in a coding unit, the second displacement represents the vertex displacement of the v-th vertex in the second vertex or the displacement of the vertex displacement of the v-th vertex in the second vertex after shifting, and j, k and v are all positive integers.

18. A coding processing device, applied to a coding end, characterized in that: include: A first processing module, configured to determine a basic grid code stream based on the grid to be coded; a first encoding module configured to perform displacement encoding on a first vertex displacement to obtain a displacement code stream, wherein the first vertex displacement is obtained by adjusting a displacement order of a reconstructed basic mesh obtained by reconstructing the basic mesh code stream according to the displacement information, the displacement information being obtained by subdividing and deforming a first mesh, and the first mesh being obtained by mesh simplification and mesh parameterization of the mesh to be encoded; A generation module is used to generate a target code stream based on the texture map to be encoded corresponding to the grid to be encoded, the basic grid code stream and the displacement code stream, wherein the target code stream includes target identification information, and the target identification information is used to indicate that the encoding method of the displacement code stream is a video encoding method or an entropy encoding method.

19. A decoding processing device, applied to a decoding end, characterized in that: include: A receiving module, configured to receive a target code stream, wherein the target code stream includes a displacement code stream and target identification information, wherein the target identification information is used to indicate whether the encoding mode of the displacement code stream is a video encoding mode or an entropy encoding mode; A determination module, configured to determine a target decoding mode according to the target identification information; The first decoding module is configured to decode the displacement code stream according to the target decoding mode to obtain a third vertex displacement.

20. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the encoding processing method according to any one of claims 1 to 7 are implemented, or when the program or instruction is executed by the processor, the steps of the decoding processing method according to any one of claims 8 to 17 are implemented.

21. A readable storage medium, characterized in that The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the encoding processing method according to any one of claims 1 to 7 are implemented, or when the program or instruction is executed by the processor, the steps of the decoding processing method according to any one of claims 8 to 17 are implemented.

Images