Three-dimensional grid inter-frame prediction encoding method, decoding method, device and electronic equipment

Abstract

The present application discloses a three-dimensional grid inter-frame prediction coding method, decoding method, device and electronic device, belonging to the technical field of three-dimensional dynamic grid coding and decoding. The coding method of the embodiment of the present application includes: the coding end performs a first processing on the three-dimensional grid to be coded to obtain a basic grid; the coding end performs sub-grid division on the basic grid to obtain P sub-grids; the coding end determines the target coding mode and the first target motion vector prediction MVP value of the grid vertices to be coded in the P sub-grid, the first target MVP value is one of N MVP values ​​included in a first candidate list, each of the MVP values ​​in the first candidate list corresponds to an index, and N is an integer greater than 1; the coding end encodes the target coding mode and the index corresponding to the first target MVP value to obtain a code stream including first information, and the first information is used to represent the target coding mode and the index corresponding to the first target MVP value.

Description

Technical Field

[0001] The present application belongs to the field of three-dimensional dynamic grid encoding and decoding technology, and specifically relates to a three-dimensional grid inter-frame prediction encoding method, decoding method, device and electronic equipment. Background Art

[0002] In recent years, with the rapid development of multimedia technology, three-dimensional models have become a new generation of digital media, following audio, images, and video. Three-dimensional meshes are a commonly used representation of three-dimensional models. After mesh simplification, mesh parameterization, and subdivision deformation, a base mesh is obtained. When encoding the base mesh, it is divided into three categories: I sub-meshes, P sub-meshes, and Skip sub-meshes. For P sub-meshes, inter-frame coding is used, requiring encoding of sub-mesh reference information and the motion vector (MV) of each vertex. Currently, a fixed method is typically used for motion vector prediction, which is not applicable to all vertices, resulting in inaccurate MV prediction for some vertices. Summary of the Invention

[0003] The embodiments of the present application provide a three-dimensional grid inter-frame prediction encoding method, decoding method, device and electronic device, which can solve the problem of low accuracy of three-dimensional grid inter-frame prediction in related technologies.

[0004] In a first aspect, a three-dimensional grid inter-frame prediction coding method is provided, which is performed by an encoding end and includes:

[0005] The encoding end performs a first processing on the three-dimensional grid to be encoded to obtain a basic grid;

[0006] The encoding end divides the basic grid into subgrids to obtain P subgrids;

[0007] The encoder determines a target coding mode and a first target motion vector prediction MVP value for a mesh vertex to be encoded in the P sub-mesh, where the first target MVP value is one of N MVP values ​​included in a first candidate list, each MVP value in the first candidate list corresponds to an index, and N is an integer greater than 1;

[0008] The encoding end encodes the target coding mode and the index corresponding to the first target MVP value to obtain a code stream including first information, where the first information is used to represent the target coding mode and the index corresponding to the first target MVP value.

[0009] In a second aspect, a three-dimensional grid inter-frame prediction decoding method is provided, which is executed by a decoding end and includes:

[0010] The decoding end obtains the code stream sent by the encoding end, wherein the code stream includes the basic grid code stream;

[0011] The decoding end performs sub-grid type decoding on the basic grid code stream to obtain a P sub-grid code stream including first information, where the first information is used to represent the target coding mode and the index corresponding to the first target MVP value;

[0012] The decoding end determines a target coding mode according to the first information, and determines a first target MVP value from a first candidate list according to an index corresponding to the first target MVP value, the first candidate list including N MVP values ​​and an index corresponding to each MVP value, the first target MVP value being one of the N MVP values, where N is an integer greater than 1;

[0013] The decoding end decodes the P sub-grid code stream according to the target coding mode and the first target MVP value to obtain the MV value of the grid vertex to be decoded in the P sub-grid.

[0014] According to a third aspect, a three-dimensional grid inter-frame prediction encoding apparatus is provided, comprising:

[0015] a processing module, configured to perform a first processing on the three-dimensional grid to be encoded to obtain a basic grid;

[0016] A division module, configured to divide the base grid into subgrids to obtain P subgrids;

[0017] A first determination module is configured to determine a target coding mode and a first target motion vector prediction MVP value for a mesh vertex to be encoded in the P sub-mesh, where the first target MVP value is one of N MVP values ​​included in a first candidate list, each MVP value in the first candidate list corresponds to an index, and N is an integer greater than 1;

[0018] The encoding module is used to encode the target coding mode and the index corresponding to the first target MVP value to obtain a code stream including first information, where the first information is used to represent the target coding mode and the index corresponding to the first target MVP value.

[0019] In a fourth aspect, a three-dimensional grid inter-frame prediction decoding device is provided, comprising:

[0020] An acquisition module, configured to acquire a code stream sent by an encoding end, wherein the code stream includes a basic grid code stream;

[0021] A first decoding module is configured to perform sub-grid type decoding on the basic grid code stream to obtain a P sub-grid code stream including first information, where the first information is used to represent a target coding mode and an index corresponding to a first target MVP value;

[0022] a second determination module, configured to determine a target coding mode according to the first information, and determine a first target MVP value from a first candidate list according to an index corresponding to the first target MVP value, the first candidate list including N MVP values ​​and an index corresponding to each MVP value, the first target MVP value being one of the N MVP values, where N is an integer greater than 1;

[0023] The second decoding module is used to decode the P sub-grid code stream according to the target coding mode and the first target MVP value to obtain the MV value of the grid vertex to be decoded in the P sub-grid.

[0024] In a fifth aspect, an electronic device is provided, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0025] In a sixth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the second aspect are implemented.

[0026] In the seventh aspect, a chip is provided, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the method described in the first aspect, or to implement the method described in the second aspect.

[0027] In an eighth aspect, a computer program / program product is provided, wherein the computer program / program product is stored in a storage medium, and the program / program product is executed by at least one processor to implement the method as described in the first aspect, or to implement the method as described in the second aspect.

[0028] In an embodiment of the present application, the encoding end is no longer limited to using a fixed motion vector prediction method for encoding, but can flexibly select the first target MVP value from the first candidate list for motion vector prediction, so that the encoding method of the encoding end for the P sub-grid motion vector is more flexible, thereby helping to improve the accuracy of the encoding end for the P sub-grid inter-frame prediction, and also helping to improve the encoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1a It is a flowchart of encoding in VDMC mode at the encoding end;

[0030] Figure 1b This is a flowchart of decoding by the decoding end in VDMC mode;

[0031] Figure 2 This is a flowchart of a three-dimensional grid inter-frame prediction coding method provided by an embodiment of the present application;

[0032] Figure 3 This is a flowchart of a three-dimensional grid inter-frame prediction decoding method provided by an embodiment of the present application;

[0033] Figure 4a This is a schematic diagram of a grid simplification in a three-dimensional grid inter-frame prediction coding method provided by an embodiment of the present application;

[0034] Figure 4b This is a flowchart of an existing MV coding scheme in a three-dimensional grid inter-frame prediction coding method provided in an embodiment of the present application;

[0035] Figure 4c It is a schematic diagram of MV prediction in an existing MV coding scheme;

[0036] Figure 4d This is a flowchart of an MV coding scheme based on an MVP candidate list in a three-dimensional grid inter-frame prediction coding method provided by an embodiment of the present application;

[0037] Figure 4e is a flow chart of reordering the MVP candidate list in the MV coding scheme based on the MVP candidate list;

[0038] Figure 4f This is a schematic diagram of subdivision deformation in a three-dimensional grid inter-frame prediction coding method provided by an embodiment of the present application;

[0039] Figure 5a This is a flowchart of an existing MV decoding scheme in a three-dimensional grid inter-frame prediction decoding method provided in an embodiment of the present application;

[0040] Figure 5b This is a flowchart of an MV decoding scheme based on an MVP candidate list in a three-dimensional grid inter-frame prediction decoding method provided by an embodiment of the present application;

[0041] Figure 6 This is a structural diagram of a three-dimensional grid inter-frame prediction encoding device provided by an embodiment of the present application;

[0042] Figure 7 This is a structural diagram of a three-dimensional grid inter-frame prediction decoding device provided by an embodiment of the present application;

[0043] Figure 8 This is one of the structural diagrams of an electronic device provided in an embodiment of the present application;

[0044] Figure 9 This is the second structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the accompanying drawings in the embodiments of this application to clearly describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0046] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and do not limit the number of objects, for example, the first object can be one or more. In addition, "or" in this application represents at least one of the connected objects. For example, "A or B" covers three options, namely, Option 1: including A but not including B; Option 2: including B but not including A; Option 3: including both A and B. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0047] For better understanding, the relevant concepts and principles that may be involved in the embodiments of this application are explained below.

[0048] With the rapid development of multimedia technology, related research results have been rapidly industrialized and have become an indispensable part of people's lives. Three-dimensional models have become the next generation of digital media, following audio, images, and video. Three-dimensional meshes are a commonly used representation method for 3D models. Compared to traditional multimedia such as images and videos, 3D mesh models offer greater interactivity and realism, leading to their increasing application in various fields, including commerce, manufacturing, construction, education, medicine, entertainment, art, and the military.

[0049] While there are many ways to represent 3D meshes, 3D meshes remain the most common. A 3D mesh can be considered to be composed of three basic elements: vertices, edges, and faces. Vertices are the most basic elements in a mesh, defining a position in 3D space. Edges are line segments connecting two vertices in the mesh. Faces can be considered polygons formed by closed paths of edges. For a triangular mesh, each face is a triangle.

[0050] The information contained in the mesh is usually divided into three categories: geometric information, connection information, and attribute information. Geometric information refers to the position of each vertex of the mesh in three-dimensional space. Connection information describes the association between the elements in the mesh, that is, the connection relationship between vertices. Attribute information is optional, and it can associate attributes with corresponding mesh elements (such as vertex color, normal vector, etc. can be associated with mesh vertices). Mesh parameterization can also be used to map the mesh from three-dimensional space to a two-dimensional plane area. This mapping relationship is usually described by a set of parameter coordinates, called UV coordinates or texture coordinates, which are associated with mesh vertices. This two-dimensional mapping can be used to represent high-resolution attribute information, such as textures, normal vectors, etc.

[0051] In nearly all application fields using 3D meshes (such as computational simulation, entertainment, medical imaging, digitized artifacts, computer design, and e-commerce), the demand for visually appealing 3D mesh models is increasing, leading to increasingly complex models and higher precision. Consequently, the amount of data required to represent the 3D mesh is also increasing. These issues have led to increasing complexity in the processing, visualization, transmission, and storage of 3D meshes. 3D mesh compression can be considered a solution to these problems. It reduces the size of model data and facilitates the processing, storage, and transmission of 3D meshes. Therefore, it is necessary to propose an efficient and universal 3D mesh compression algorithm.

[0052] Recently, the Moving Pictures Experts Group (MPEG), an international organization for standardization specializing in audio and video coding and compression, has begun developing a compression standard for 3D meshes called Video-based dynamic mesh coding (VDMC). This standard is based on the existing Visual Volumetric Video-based Coding (V3C) standard, which provides a general method for compressing 3D models, which can be represented by point clouds, meshes, or panoramic videos. Making 3D mesh compression methods compatible with this standard will facilitate their widespread adoption and applicability. Therefore, optimizing the 3D mesh encoding and decoding methods in VDMC and integrating these optimizations with the V3C standard is of great significance.

[0053] V3C standard:

[0054] The V3C standard provides a method for encoding and decoding various 3D media using video or image coding techniques. Specifically, prior to encoding, 3D media content is converted from a 3D representation into multiple 2D representations (called V3C components) through methods such as projection. These 2D representations are then encoded using existing video or image coding techniques. V3C components primarily include occupancy, geometry, and attribute components. The occupancy component indicates which areas of the 2D representation are associated with the 3D data; the geometry component indicates information related to the spatial position of the 3D data; and the attribute component provides vertex attribute information, such as material and texture. Furthermore, the components contain information on how to reconstruct a 3D model from these components, known as atlas information. Atlas information is used to link all components, and additional information for 3D reconstruction from 2D is also included in the atlas components. An atlas consists of multiple basic units, called patches. Each patch represents an area within the available 2D components and contains the information needed to project that area back into 3D space.

[0055] VDMC:

[0056] VDMC is a standard developed by MPEG for compressing three-dimensional meshes. Its main idea is to compress three-dimensional meshes by leveraging the existing V3C standard. Since three-dimensional meshes contain connection information that needs to be encoded, its specific encoding process is slightly different from V3C. The syntax, semantics, and decoding operations of the V3C standard decoding end need to be expanded to support the decoding and reconstruction of three-dimensional meshes.

[0057] The overall framework of the encoding end is as follows Figure 1aAs shown in the figure, the input mesh is first simplified by the simplification module. Mesh parameterization is then used to generate new texture coordinates for the mesh. The parameterized mesh is then subdivided and deformed. This involves inserting new vertices according to a specific subdivision method and calculating the distances between the vertices of the subdivided mesh and their nearest neighbors on the input mesh, which is referred to as displacement information. The parameterized mesh, representing the vertex positions of the pre-deformed mesh, is then adjusted based on the displacement information. This adjusted mesh, called the base mesh, is then sent to the base mesh encoding module for compression. During encoding, the base mesh is divided into three sub-meshes for encoding and decoding. For I sub-meshes, intra-frame encoding is used to encode their geometry, connectivity, and attribute information. For P sub-meshes, inter-frame encoding is used, requiring the encoding of sub-mesh reference information and the motion vector (MV) of each vertex. For skip sub-meshes, skip-frame encoding is used to encode only the reference information. After encoding, the base mesh is reconstructed, and the displacement order is adjusted based on the vertex order of the reconstructed base mesh. Subsequently, the reordered vertex displacement information is first subjected to a wavelet transform, and the transformed coefficients are quantized. These quantized coefficients are then arranged into a two-dimensional image according to a specific scan order, and the two-dimensional image is encoded using a video encoder. The reconstructed displacement information is then applied to the subdivided base mesh to produce a reconstructed, subdivided, deformed mesh. This mesh, along with the original input mesh and its corresponding texture map, is input into the corresponding texture map conversion module to produce the texture map corresponding to the reconstructed mesh. This texture map is also encoded using the video encoder. Parameters used in the encoding process, such as the video encoder type, mesh encoder type, transform parameters, and quantization parameters, are passed to the decoder via auxiliary information.

[0058] The overall framework of the decoding end is as follows Figure 1b As shown, for the received bitstream, the decoder first demultiplexes the various bitstream components to obtain a base mesh bitstream, a displacement video bitstream, a texture map video bitstream, and an auxiliary information bitstream. The base mesh bitstream is decoded using a mesh decoder indicated by the auxiliary information to obtain the base mesh. Specifically, based on the decoded sub-mesh type identifier, the corresponding bitstream segments are fed into decoders of different modes for decoding. The decoded sub-meshes are then merged and concatenated, ultimately outputting a reconstructed base mesh. The displacement video bitstream and texture map video bitstream are decoded by the video decoder. For the displacement component, after video decoding, the displacement is extracted from the image using a displacement decoding module, and dequantized and inversely transformed. This displacement is then applied to the subdivided base mesh to obtain the deformed mesh reconstructed by the decoder. After decoding, the texture map corresponds to the reconstructed deformed mesh. Subsequent application or rendering modules process the reconstructed deformed mesh and the decoded texture map as input.

[0059] Currently, inter-frame prediction methods for dynamic 3D grids typically use fixed motion vector prediction methods, which results in inaccurate motion vector prediction values ​​and low inter-frame coding efficiency. To address the above issues, embodiments of the present application propose a 3D grid inter-frame prediction method.

[0060] The following, in conjunction with the accompanying drawings, describes in detail the three-dimensional grid inter-frame prediction encoding method, decoding method and related devices provided by the embodiments of the present application through some embodiments and their application scenarios.

[0061] Please refer to Figure 2 , Figure 2 This is a flowchart of a three-dimensional grid inter-frame prediction coding method provided by an embodiment of the present application, and the method is applied to the coding end. Figure 2 As shown, the method includes the following steps:

[0062] Step 201: The encoding end performs a first process on the three-dimensional mesh to be encoded to obtain a basic mesh.

[0063] It should be noted that after receiving the input 3D mesh to be encoded, the encoder may first perform mesh simplification on the 3D mesh to be encoded, thereby simplifying the 3D mesh to be encoded into a mesh with a relatively small number of points and faces while preserving the shape of the original mesh as much as possible. Furthermore, the simplified 3D mesh to be encoded may be subjected to mesh parameterization to obtain a base mesh.

[0064] In the embodiment of the present application, the first processing may include a processing process such as mesh simplification and mesh parameterization, that is, the first processing is used to obtain a basic mesh.

[0065] Step 202: The encoding end divides the basic grid into sub-grids to obtain P sub-grids.

[0066] In the embodiment of the present application, after the basic grid is obtained through the first processing, the encoding end divides the basic grid into sub-grids, and divides it into three types of sub-grids: I sub-grid, P sub-grid, and skip sub-grid.

[0067] It should be noted that these three types of sub-grids correspond to different encoding methods, that is, they need to be encoded using different encoding methods. Specifically, the I sub-grid, P sub-grid, and Skip sub-grid are intra-frame encoded, inter-frame encoded, and Skip encoded, respectively.

[0068] Step 203: The encoder determines a target coding mode and a first target MVP value for the grid vertices to be encoded in the P sub-grid.

[0069] The first target MVP value is one of N MVP values ​​included in the first candidate list, each MVP value in the first candidate list corresponds to an index, and N is an integer greater than 1.

[0070] It should be noted that when encoding the P subgrid, the reference information and the motion vector need to be encoded in sequence, and when encoding the motion vector (MV) within the P subgrid, the MV encoding scheme based on the first candidate list may be adopted.

[0071] In an embodiment of the present application, the encoding end may be a pre-constructed first candidate list, which includes N motion vector prediction (MVP) values ​​and an index corresponding to each MVP value. Exemplarily, the encoding end may determine the MVP value in the first candidate list based on the MV value of the encoded mesh vertex adjacent to the mesh vertex to be encoded. For example, the MV value of an encoded mesh vertex corresponds to an MVP value, and based on the MV values ​​of N encoded mesh vertices, M MVP values ​​(M>N) can be obtained accordingly, and the first candidate list is constructed based on these MVP values. Each of the MVP values ​​in the first candidate list corresponds to an index, that is, the first candidate list includes N MVP values ​​and an index corresponding to each MVP value.

[0072] Optionally, the first candidate list can also be obtained based on other methods, for example, the N MVP values ​​in the first candidate list can be determined based on historical experience values. Of course, the MVP values ​​in the first candidate list can also be obtained by other methods, which are not listed here.

[0073] Furthermore, the encoder may select an MVP value from the first candidate list as the first target MVP value for the to-be-encoded mesh vertex in the P sub-mesh. For example, the encoder may select a MVP value randomly, select a value closer to a preset value, or select a value by calculating the rate-distortion cost corresponding to each of these MVP values. Once the first target MVP value is determined, the index corresponding to the first target MVP value may be obtained.

[0074] In addition, in an embodiment of the present application, the target coding mode may refer to a specific coding mode, such as a mode for encoding directly based on the MVP value or a mode for encoding based on the MVP value and motion vector difference (MVD), etc.

[0075] Optionally, the encoder may determine the target coding mode on its own, or may determine the target coding mode based on an agreement with the decoder, or may select one from multiple candidate coding modes as the target coding mode, etc. It should be noted that the target coding mode refers to the encoding method for the motion vector within the P subgrid.

[0076] Step 204: The encoding end encodes the target coding mode and the index corresponding to the first target MVP value to obtain a code stream including first information, where the first information is used to represent the target coding mode and the index corresponding to the target MVP value.

[0077] In an embodiment of the present application, after determining the target coding mode of the mesh vertex to be encoded in the P sub-grid and the index corresponding to the first target MVP value, the encoder encodes the target coding mode and the index corresponding to the first target MVP value into the bitstream to obtain a bitstream including first information, where the first information is used to characterize the target coding mode of the mesh vertex to be encoded in the P sub-grid and the index corresponding to the first target MVP value. Then, after receiving the bitstream from the encoder, the decoder can determine the encoding method and the MVP value adopted by the encoder for the motion vector in the P sub-grid based on the first information in the bitstream. Then, the decoder can use the encoding method and MVP value corresponding to the encoder to decode the P sub-grid bitstream to obtain the same P sub-grid as the encoder.

[0078] It should be noted that the code stream including the first information obtained by the encoding end may refer to the P sub-grid code stream, or it may refer to the mixed code stream finally obtained by the encoding end including the basic grid code stream, the displacement code stream, the texture map code stream, etc. The embodiment of the present application does not make specific limitations on this.

[0079] In an embodiment of the present application, after the encoding end divides the basic grid into subgrids to obtain P subgrids, it determines the target coding mode of the grid vertex to be encoded in the P subgrid and the index corresponding to the first target MVP value, and encodes the target coding mode and the index corresponding to the first target MVP value into the code stream, thereby realizing the encoding of the motion vector of the grid vertex to be encoded in the P subgrid. Among them, the first target MVP value is one selected from the N MVP values ​​included in the first candidate list, so that the encoding end is no longer limited to using a fixed motion vector prediction method for encoding, but can flexibly select the first target MVP value from the first candidate list for motion vector prediction, so that the encoding method of the encoding end for the P subgrid motion vector is more flexible, thereby helping to improve the accuracy of the encoding end for the inter-frame prediction of the P subgrid, and also helping to improve the encoding efficiency.

[0080] Optionally, before the encoder determines the target coding mode and the first target MVP value of the mesh vertices to be encoded in the P sub-grid, the method further includes:

[0081] The encoding end constructs the first candidate list based on encoded mesh vertices adjacent to the mesh vertex to be encoded in the P sub-grid.

[0082] Optionally, the encoder may pre-specify the length of the first candidate list, that is, pre-determine the number of MVP values ​​included in the first candidate list, that is, the value of N. For example, assuming that N is equal to 10, the encoder may determine 10 MVP values ​​based on 10 adjacent coded mesh vertices of the to-be-coded mesh vertex in the P sub-mesh, thereby constructing the first candidate list. Each MVP value in the first candidate list includes a corresponding index.

[0083] It should be noted that the encoded mesh vertices adjacent to the mesh vertex to be encoded can be the N encoded mesh vertices directly connected to the mesh vertex to be encoded, or the N encoded mesh vertices closest to the mesh vertex to be encoded, or the adjacent encoded mesh vertices in a certain direction of the mesh vertex to be encoded, which is not specifically limited here.

[0084] In an embodiment of the present application, the encoding end constructs a first candidate list including N MVP values, and then when the encoding end encodes the motion vector in the P sub-grid, it can select an MVP value from the first candidate list as the first target MVP value, and encode the index corresponding to the first target MVP value. In this way, the encoding method of the encoding end for the motion vector in the P sub-grid is more flexible.

[0085] Optionally, the encoder constructs the first candidate list based on coded mesh vertices adjacent to the to-be-coded mesh vertex in the P sub-grid, including:

[0086] The encoder obtains L adjacent already-encoded mesh vertices and M adjacent already-encoded mesh vertices of the mesh vertex to be encoded in the P sub-grid, where L and M are integers greater than 1;

[0087] The encoder obtains a motion vector MV value of each of the L adjacent encoded mesh vertices, and determines L first MVP values ​​of the to-be-encoded mesh based on the MV values ​​of the L encoded mesh vertices;

[0088] The encoding end determines a second MVP value based on the MV values ​​corresponding to the M encoded mesh vertices;

[0089] The encoding end constructs the first candidate list based on L first MVP values ​​and the second MVP value, where L+1≤N.

[0090] It should be noted that the values ​​of L and M can be the same or different. For example, L = 10, M = 5, that is, the encoder obtains 5 encoded mesh vertices adjacent to the mesh vertex to be encoded in the P sub-mesh, and 10 adjacent encoded mesh vertices; wherein, these 10 adjacent encoded mesh vertices may include some or all of the 5 adjacent encoded mesh vertices, or not include any of them at all. Further, the encoder obtains the MV value of each encoded mesh vertex in the 10 encoded mesh vertices, and uses these MV values ​​as the first MVP value, that is, obtaining 10 first MVP values, and at the same time determines a second MVP value based on the MV value corresponding to each of the 5 adjacent encoded mesh vertices. For example, the second MVP value can be the average of the MV values ​​corresponding to the 5 encoded mesh vertices; a first candidate list is constructed based on these 10 first MVP values ​​and one second MVP value. For example, the first candidate list can include 10 first MVP values ​​and one second MVP value, that is, a total of 11 MVP values, that is, N = 11.

[0091] Optionally, if the first MVP value and the second MVP value have the same value, only one of the same MVP values ​​may be retained and filled into the first candidate list. For example, the encoding end may obtain the MV values ​​of the encoded mesh vertices adjacent to the mesh vertex to be encoded in a certain order, and fill the MV values ​​of the adjacent encoded mesh vertices as MVP values ​​into the first candidate list one by one according to the number of MVP values ​​that can be filled in the first candidate list. If the current MVP value to be filled is equal to the MVP value already in the first candidate list, the filling of the current MVP value to be filled is skipped, and the next MVP value to be filled is detected to see if it is the same as the existing MVP value, and the first candidate list is filled in this way.

[0092] In an embodiment of the present application, the MVP value can be determined based on the MV value of the coded mesh vertex adjacent to the mesh vertex to be coded, so as to construct a first candidate list, thereby effectively improving the flexibility of the encoder for encoding the motion vector in the P sub-grid. It can be understood that the MV value of the coded mesh vertex can be known, and the MV value of the coded mesh vertex is used as the MVP value of the mesh vertex to be coded. The subsequent encoder can select an MVP value (that is, the first target MVP value) from the first candidate list as the MV prediction value of the mesh vertex to be coded, so as to realize multiple inter-frame prediction methods for the P sub-grid, which helps to improve the accuracy of inter-frame prediction and coding efficiency.

[0093] Optionally, the encoder determines a second MVP value based on the MV values ​​corresponding to the M encoded mesh vertices, including:

[0094] The encoding end obtains the MV values ​​corresponding to the M encoded mesh vertices, and determines the average of the M MV values ​​as the second MVP value; or

[0095] The encoding end obtains the MV values ​​corresponding to the M encoded mesh vertices, performs weighted average calculation on the M MV values, and determines the calculated result as the second MVP value.

[0096] For example, if M=5, that is, the MV values ​​corresponding to the five coded mesh vertices adjacent to the mesh vertex to be coded are obtained, and the average of these five MV values ​​is used as the second MVP value in the first candidate list; or, a weighted average calculation can be performed on these five MV values, and the result of the weighted average calculation is used as the second MVP value. In the case of weighted average calculation, the weight corresponding to each MV value can be related to the distance between the coded mesh vertex and the mesh vertex to be coded. For example, the closer the distance between the coded mesh vertex and the mesh vertex to be coded, the greater the weight corresponding to the MV value of the coded mesh vertex.

[0097] It should be noted that the encoding end may randomly select a method to determine the second MVP value, which makes the encoding end more flexible in calculating the second MVP value.

[0098] Optionally, an arrangement order of the L first MVP values ​​in the first candidate list is related to a first distance, where the first distance is a distance between a coded mesh vertex corresponding to the first MVP value and the mesh vertex to be coded. For example, the closer the distance between the coded mesh vertex and the mesh vertex to be coded, the higher the position of the first MVP value corresponding to the coded mesh vertex in the first candidate list.

[0099] Optionally, the L first MVP values ​​are arranged in ascending order according to the first distance. That is, the L first MVP values ​​are arranged in the first candidate list in ascending order according to the distances between their corresponding L coded mesh vertices and the mesh vertex to be coded. For example, if L = 10, among the 10 coded mesh vertices, the first MVP value corresponding to the coded mesh vertex with the smallest distance to the mesh vertex to be coded is arranged first, and the first MVP value corresponding to the coded mesh vertex with the largest distance to the mesh vertex to be coded is arranged last.

[0100] It can be understood that based on this method, the encoding end is more likely to select the MVP value of the encoded grid vertex that is closer to it as the first target MVP value, that is, it can select the MVP value of the encoded grid vertex that is closer to it for inter-frame prediction, which can better ensure the accuracy of inter-frame prediction.

[0101] It should be noted that, in the first candidate list, the second MVP value may be arranged in front of all first MVP values, that is, the second MVP value is ranked first in the first candidate list, followed by the L first MVP values ​​arranged according to the above sorting method.

[0102] In the embodiment of the present application, when L+1<N, the encoding end constructs the first candidate list based on L first MVP values ​​and the second MVP value, including:

[0103] The encoding end constructs a sub-candidate list based on the L first MVP values ​​and the second MVP value;

[0104] The encoding end performs a zero-padding operation on the sub-candidate list to obtain the first candidate list including N MVP values.

[0105] For example, the number of MVP values ​​that can be filled in the first candidate list is 15, that is, N=15. If L+1<15, that is, L first MVP values ​​and one second MVP value cannot completely fill the first candidate list, then the encoder can fill the vacancies in the first candidate list with zeros, for example, filling these vacancies with MVP values ​​of 0, until a first candidate list including 15 MVP values ​​is obtained. In this way, a complete first candidate list can be obtained, ensuring that the encoder can select an MVP value from the first candidate list as the first target MVP value to implement inter-frame prediction within the P subgrid.

[0106] It should be noted that, for the MVP values ​​obtained by the zero-padding operation, the encoding end may arrange these MVP values ​​at the end of the first candidate list, that is, after the second MVP value and the L first MVP values.

[0107] In the embodiment of the present application, after the encoder constructs the first candidate list based on the encoded mesh vertices adjacent to the mesh vertex to be encoded in the P sub-mesh, the method further includes:

[0108] The encoder reorders the N MVP values ​​in the first candidate list to obtain a reordered first candidate list;

[0109] In this case, determining the first target MVP value from the first candidate list includes:

[0110] The encoder determines a first target MVP value from the reordered first candidate list.

[0111] Exemplarily, after the encoding end constructs the first candidate list based on N MVP values, it may reorder the N MVP values ​​in the first candidate list. For example, the MVP values ​​with a high probability of being selected may be arranged at the front of the first candidate list. When the encoding end selects the first target MVP value based on the reordered first candidate list, it is more likely to select these MVP values ​​arranged at the front as the first target MVP value.

[0112] It should be noted that the encoder can adaptively reorder the MVP values ​​in the first candidate list. The indexes corresponding to the MVP values ​​are generally encoded using variable-length codes. The purpose of adaptive reordering is to place MVP values ​​with a high probability of selection at the front of the first candidate list. This reduces the bit consumption when encoding the index information corresponding to the first target MVP value, further helping to improve the encoding efficiency of the encoder.

[0113] Optionally, the encoder reorders the N MVP values ​​in the first candidate list, including:

[0114] The encoder obtains, the sum of errors between a second target MVP value and a motion vector MV of an encoded mesh vertex adjacent to the mesh vertex to be encoded, where the second target MVP value is one of the N MVP values;

[0115] The encoding end reorders the N MVP values ​​based on the error corresponding to each MVP value in the first candidate list.

[0116] Exemplarily, the encoding end may first sort the MVP values ​​in the first candidate list according to the default order, and then traverse the first candidate list to calculate the sum of errors between each MVP value in the first candidate list and the adjacent encoded mesh vertices of the mesh vertex to be encoded, thereby obtaining the sum of errors corresponding to each MVP value, and reordering the MVP values ​​in the first candidate list based on these error sums.

[0117] Optionally, the N MVP values ​​in the reordered first candidate list are arranged in ascending order based on their corresponding error sums. That is, based on the magnitude of the error sums corresponding to the N MVP values ​​in the first candidate list, the N MVP values ​​are reordered in ascending order based on the error sums. That is, the MVP value with the smaller error sum is ranked closer to the front of the first candidate list. This makes it more likely that the encoder will select the MVP value with the smaller error sum as the first target MVP value, thereby improving the accuracy of inter-frame prediction at the encoder.

[0118] In the embodiment of the present application, the encoder determines the first target MVP value, which may specifically include:

[0119] The encoder obtains, by the encoder end, a first rate-distortion cost of each MVP value in the first candidate list in the first coding mode and a second rate-distortion cost in the second coding mode, to obtain N first rate-distortion costs and N second rate-distortion costs;

[0120] The encoding end determines a first target MVP value from the N MVP values ​​according to the N first rate-distortion costs and the N second rate-distortion costs.

[0121] Optionally, the first encoding mode is a mode for encoding directly based on the MVP value, and the second encoding mode is a mode for encoding based on the MVP value and the MVD value.

[0122] When the encoder selects the first target MVP value from the first candidate list, it may calculate the rate-distortion cost of each MVP value in the first candidate list under the first coding mode and the rate-distortion cost under the second coding mode, that is, each MVP value corresponds to two rate-distortion costs, and then compare all the rate-distortion costs of all the MVP values ​​in the first candidate list, and select one MVP value as the first target MVP value based on these rate-distortion costs. For example, the MVP value with the smallest rate-distortion cost may be selected as the first target MVP value, or the MVP value with the smallest rate-distortion cost calculated under a certain coding mode may be selected as the first target MVP value, etc. The embodiments of the present application do not specifically limit this.

[0123] Optionally, the first rate-distortion cost or the second rate-distortion cost corresponding to the first target MVP value is the smallest of the N first rate-distortion costs and the N second rate-distortion costs. That is to say, if the first rate-distortion cost calculated for a certain MVP value in the first coding mode is the smallest of all rate-distortion costs, then the MVP value is used as the first target MVP value; or, if the second rate-distortion cost calculated for a certain MVP value in the second coding mode is the smallest of all rate-distortion costs, then the MVP value is used as the first target MVP value. In this way, it is possible to ensure that the rate-distortion cost of the first target MVP value is the smallest, thereby more effectively guaranteeing the accuracy of the P sub-grid inter-frame prediction.

[0124] Optionally, when the first rate-distortion cost corresponding to the first target MVP value is the smallest of N first rate-distortion costs and N second rate-distortion costs, the target coding mode is the first coding mode; or, when the second rate-distortion cost corresponding to the first target MVP value is the smallest of N first rate-distortion costs and N second rate-distortion costs, the target coding mode is the second coding mode.

[0125] Furthermore, when the target coding mode is the second coding mode, the method further includes:

[0126] The encoding end encodes the MVD of the grid to be encoded in the P sub-grid.

[0127] In an embodiment of the present application, the second coding mode is a mode for encoding based on the MVP value and the MVD value. In this coding mode, after the encoding end encodes the index corresponding to the first target MVP value and the target coding mode, it also needs to encode the MVD of the grid to be encoded in the P sub-grid.

[0128] It should be noted that if the target coding mode is the first coding mode, that is, a mode for encoding directly based on the MVP value, the encoder does not need to encode the MVD of the coding grid.

[0129] Optionally, when calculating the rate-distortion cost, the rate-distortion cost may be calculated for the MVP values ​​in the reordered first candidate list.

[0130] It should be noted that the rate-distortion cost corresponding to the MVP value is calculated as follows:

[0131] J=D+λR

[0132] Where J is the rate-distortion cost, D is the distortion introduced by encoding, λ is the Lagrange coefficient, and R is the number of bits spent encoding the MV information in the corresponding coding mode. When the coding mode is the second coding mode, that is, when the MVD needs to be encoded, D in the corresponding rate-distortion cost is 0; when the coding mode does not encode the MVD, that is, when the coding mode is the first coding mode, D is not 0.

[0133] Optionally, the target rate-distortion cost is related to a first length difference and a first angle, the first length difference is the difference between the modulus of the third target MVP value and the modulus of the MV value of the mesh vertex to be encoded, and the first angle is the angle between the third target MVP value and the MV value of the mesh vertex to be encoded; wherein, the target rate-distortion cost is the first rate-distortion cost or the second rate-distortion cost, and the third target MVP value is one of the N MVP values ​​in the first candidate list.

[0134] For example, a possible calculation method of the distortion D is as follows:

[0135]

[0136] Among them, MV is the MV value of the mesh vertex to be encoded, MVP is the MVP value in the first candidate list, (MV-MVP) is the length difference between MV and MVP, (MV, MVP) is the size of the angle between MV and MVP, and α and β are weight coefficients. Among them, the length difference between MV and MVP refers to the difference between the modulus length of the MV value of the mesh vertex to be encoded and the modulus length of the MVP value; the size of the angle between MV and MVP refers to the size of the angle between the MV value and the MVP value of the mesh vertex to be encoded. Furthermore, the rate-distortion cost corresponding to the MVP value of the encoded mesh vertex is also related to the modulus length difference and angle between the MVP value of the encoded mesh vertex and the MV value of the mesh vertex to be encoded, which helps the encoder select the first target MVP value from the first candidate list according to the rate-distortion cost, thereby ensuring the accuracy of the inter-frame prediction of the P sub-grid.

[0137] In an embodiment of the present application, the encoding end is no longer limited to using a fixed motion vector prediction method for encoding, but can flexibly select the first target MVP value from the first candidate list for motion vector prediction, so that the encoding method of the encoding end for the P sub-grid motion vector is more flexible, thereby helping to improve the accuracy of the encoding end for the P sub-grid inter-frame prediction.

[0138] Please refer to Figure 3 , Figure 3 This is a flowchart of another three-dimensional grid inter-frame prediction decoding method provided by an embodiment of the present application, which is applied to the decoding end. Figure 3 As shown, the method includes the following steps:

[0139] Step 301: The decoding end obtains a code stream sent by the encoding end, where the code stream includes a basic grid code stream.

[0140] Optionally, the code stream may be the basic grid code stream, or may be a mixed code stream including the basic grid code stream, the displacement video code stream, the texture map code stream, etc.

[0141] Step 302: The decoding end performs sub-grid type decoding on the basic grid code stream to obtain a P sub-grid code stream including first information, where the first information is used to represent the target coding mode and the index corresponding to the first target MVP value.

[0142] It can be understood that at the encoding end, the basic grid will be divided into I subgrids, P subgrids and Skip subgrids, and different subgrid types will be encoded using different encoding methods, thereby obtaining different subgrid type code streams.

[0143] In the embodiment of the present application, after obtaining the basic grid code stream, the decoding end performs sub-grid type decoding on the basic grid code stream, thereby obtaining an I sub-grid code stream, a P sub-grid code stream, and a Skip sub-grid code stream. The P sub-grid code stream includes first information for representing the target coding mode and the index corresponding to the first target MVP value. For example, it can be a code stream including the first information. The decoding end can obtain the first information by decoding the first information code stream, or the encoding end can directly encode the first information into the P sub-grid code stream.

[0144] Step 303: The decoding end determines a target coding mode according to the first information, and determines a first target MVP value from a first candidate list according to an index corresponding to the first target MVP value.

[0145] The first candidate list includes N MVP values ​​and an index corresponding to each MVP value, the first target MVP value is one of the N MVP values, and N is an integer greater than 1.

[0146] It should be noted that the decoding end may pre-build the first candidate list in the same manner as the encoding end, or may obtain the first candidate list from the encoding end. The construction method of the first candidate list may refer to the description in the above-mentioned encoding end embodiment, which will not be repeated here.

[0147] It can be understood that the decoding end can determine the target coding mode adopted by the encoding end for the motion vector in the P sub-grid based on the first information, and can find the corresponding first target MVP value from the first candidate list based on the index corresponding to the first target MVP value.

[0148] Step 304: The decoding end decodes the P sub-grid code stream according to the target coding mode and the first target MVP value to obtain the MV value of the grid vertex to be decoded in the P sub-grid.

[0149] It should be noted that the decoding of the basic grid code stream can be divided into intra-frame mode decoding for the I sub-grid code stream, inter-frame mode decoding for the P sub-grid, and Skip mode decoding for the Skip sub-grid code stream. The basic grid decoding module will send different code stream segments to the corresponding mode decoder for decoding according to the type identifier of the sub-grid in the code stream. After the sub-grid is decoded, the sub-grids are merged and spliced ​​to generate a basic grid for decoding and reconstruction. Among them, the code stream segment corresponding to the I sub-grid will be sent to the intra-frame mode decoder for decoding to obtain the I sub-grid; the code stream segment corresponding to the P sub-grid will be sent to the inter-frame mode decoder for decoding to obtain the P sub-grid; the code stream segment corresponding to the Skip sub-grid will be sent to the Skip mode decoder for decoding to obtain the Skip sub-grid.

[0150] In an embodiment of the present application, after obtaining the P subgrid code stream and the first information, the decoding end determines the MV value of the grid to be decoded in the P subgrid according to the target coding mode and the first target MVP value. For example, if the target coding mode is a mode for encoding directly based on the MVP value, the first target MVP value can be directly determined as the MV value of the grid to be decoded in the P subgrid; if the target coding mode is a mode for encoding based on the MVP value and the MVD value, the decoding end also needs to decode the MVD, and add the decoded MVD to the first target MVP value as the MV value of the grid to be decoded in the P subgrid.

[0151] In the solution provided in the embodiment of the present application, the decoding end determines the index corresponding to the first target MVP value based on the first information, selects the corresponding first target MVP value from the first candidate list, and decodes the P sub-grid code stream in combination with the first target MVP value and the target encoding method determined from the code stream, thereby obtaining the MV value of the grid vertex to be decoded in the P sub-grid.

[0152] Optionally, before determining the first target MVP value from the first candidate list according to the index corresponding to the target MVP value, the method further includes:

[0153] The decoding end constructs the first candidate list based on decoded mesh vertices adjacent to the mesh vertex to be decoded in the P sub-mesh.

[0154] Optionally, the decoding end constructs the first candidate list based on decoded mesh vertices adjacent to the mesh vertex to be decoded in the P sub-mesh, including:

[0155] The decoding end obtains L decoded mesh vertices and M decoded mesh vertices adjacent to the mesh vertex to be decoded in the P sub-grid, where L and M are integers greater than 1;

[0156] The decoding end obtains the MV value of each decoded mesh vertex among the adjacent L decoded mesh vertices, and determines L first MVP values ​​of the mesh vertex to be decoded based on the MV values ​​of the L decoded mesh vertices;

[0157] The decoding end determines a second MVP value based on the MV values ​​corresponding to the M decoded mesh vertices;

[0158] The decoding end constructs the first candidate list based on L first MVP values ​​and the second MVP value, where L+1≤N.

[0159] Optionally, the decoding end determines a second MVP value based on the MV values ​​corresponding to the M decoded mesh vertices, including:

[0160] The decoding end obtains the MV values ​​corresponding to the M decoded mesh vertices, and determines the average of the M MV values ​​as the second MVP value; or

[0161] The decoding end obtains the MV values ​​corresponding to the M decoded mesh vertices, performs weighted average calculation on the M MV values, and determines the calculated result as the second MVP value.

[0162] Optionally, an arrangement order of the L first MVP values ​​in the first candidate list is related to a first distance, where the first distance is a distance between a decoded mesh vertex corresponding to the first MVP value and the mesh vertex to be decoded.

[0163] Optionally, the L first MVP values ​​are arranged in ascending order of the first distance.

[0164] Optionally, when L+1<N, the decoding end constructs the first candidate list based on L first MVP values ​​and the second MVP value, including:

[0165] The decoding end constructs a sub-candidate list based on the L first MVP values ​​and the second MVP value;

[0166] The decoding end performs a zero-padding operation on the sub-candidate list to obtain the first candidate list including N MVP values.

[0167] It should be noted that if the first MVP value and the second MVP value have the same value, only one of the same MVP values ​​can be retained and filled in the first candidate list, that is, when constructing the first candidate list, the MVP values ​​in the first candidate list can be deduplicated.

[0168] Optionally, after the decoding end constructs the first candidate list based on decoded mesh vertices adjacent to the mesh vertex to be decoded in the P sub-mesh, the method further includes:

[0169] The decoding end reorders the N MVP values ​​in the first candidate list to obtain a reordered first candidate list;

[0170] The determining the first target MVP value from the first candidate list according to the index corresponding to the first target MVP value includes:

[0171] The decoding end determines a first target MVP value from the reordered first candidate list according to an index corresponding to the first target MVP value.

[0172] Optionally, the decoding end reorders the N MVP values ​​in the first candidate list, including:

[0173] The decoding end obtains a sum of errors between a second target MVP value and MV values ​​of decoded mesh vertices adjacent to the to-be-decoded mesh vertex, wherein the second target MVP value is one of the N MVP values;

[0174] The decoding end reorders the N MVP values ​​based on the error corresponding to each MVP value in the first candidate list.

[0175] Optionally, the N MVP values ​​in the reordered first candidate list are arranged in ascending order according to corresponding errors.

[0176] It should be noted that the decoding end may perform the processes of constructing and reordering the first candidate list in the same manner as the encoding end. The specific implementation method may refer to the description in the above-mentioned encoding end method embodiment, which will not be repeated here.

[0177] Optionally, the target coding mode is a first coding mode or a second coding mode, the first coding mode is a mode for encoding directly based on the MVP value, and the second coding mode is a mode for encoding based on the MVP value and the MVD value.

[0178] Optionally, when the target coding mode is the second coding mode, the P sub-grid code stream further includes an MVD value, and the method further includes:

[0179] The decoding end performs MVD decoding on the P sub-grid code stream to obtain MVD values ​​corresponding to the grid vertices to be decoded in the P sub-grid;

[0180] The step of obtaining the MV value of the to-be-decoded mesh vertex in the P sub-mesh includes:

[0181] The MV value corresponding to the mesh vertex to be decoded in the P sub-grid is determined according to the MVD value corresponding to the mesh vertex to be decoded in the P sub-grid and the first target MVP value.

[0182] In the embodiment of the present application, when the target encoding mode is a mode for encoding based on the MVP value and the MVD value, that is, the encoder also encodes the MVD value of the mesh vertex to be encoded in the P sub-mesh, the decoder also needs to perform MVD decoding to obtain the MVD value corresponding to the mesh vertex to be decoded, and then determine the MV value corresponding to the mesh vertex to be decoded based on the MVD value corresponding to the mesh vertex to be decoded and the first target MVP value, for example, the MV value is the sum of the MVD value and the first target MVP value. In this way, it can be ensured that the decoder can decode and obtain the motion vector of the P sub-mesh consistent with that of the encoder.

[0183] For better understanding, the methods involved in the embodiments of the present application are explained below with respect to the encoding method of the encoding end and the decoding method of the decoding end respectively.

[0184] The three-dimensional grid encoding method of the encoding end specifically includes the following process (see Figure 1a Flowchart shown):

[0185] (1) Mesh simplification

[0186] Mesh simplification is to simplify the current input mesh into a base mesh with relatively few points and faces, and to keep the shape of the original mesh as much as possible. The key points of mesh simplification are the simplification operation and the corresponding error metric. A feasible mesh simplification operation is Figure 4a As shown in the figure, the vertices at both ends of the edge are merged into a single vertex and the connection between the two vertices is deleted. This process is repeated throughout the mesh according to a certain rule to reduce the number of faces and vertices of the mesh to the target value.

[0187] During the simplification process, a specific error metric can be selected to optimize the simplified result. For example, the error metric for a vertex can be the sum of the coefficients of the equations of all adjacent faces. The error metric for an edge can be the sum of the error metrics of the two vertices on the edge. In other words, the error resulting from merging an edge is the sum of the distances from the merged vertex to all adjacent faces of the original two vertices on the edge.

[0188] After determining the simplification operation and the corresponding error metric, the mesh simplification process begins iteratively. First, the vertex errors of the initial mesh are calculated to obtain the error for each edge. Edges are then sorted from smallest to largest error, and the edge with the smallest error is merged each time. Simultaneously, the positions of the merged vertices are calculated, and the errors of all edges associated with the merged vertices are updated. This means that the order of edge arrangement is updated to ensure that each iteration is based on a global error metric. Through iteration, the mesh faces are simplified to the number required for lossy encoding.

[0189] (2) Mesh parameterization

[0190] This step requires regenerating texture coordinates for the simplified mesh. Currently, there are many algorithms for parameterizing meshes, such as the Isocharts algorithm, which uses spectral analysis to achieve stretch-driven 3D mesh parameterization, UV-unwrapping, slicing, and packing the 3D mesh into a 2D texture domain.

[0191] (3) Basic mesh compression

[0192] When encoding the base grid, it is divided into three sub-grids: I sub-grid, P sub-grid, and Skip sub-grid. These three sub-grids are then subjected to intra-frame coding, inter-frame coding, and Skip coding, respectively. The encoder can determine the coding mode for each sub-grid by comparing the rate-distortion cost. When encoding the I sub-grid, the existing static grid encoder is used to encode its geometric information, connectivity information, and attribute information. When encoding the P sub-grid, the reference information and motion vectors need to be encoded sequentially. When encoding the Skip sub-grid, only the reference information needs to be encoded.

[0193] There are two feasible solutions for encoding the MVs in the P subgrid: an existing MV encoding solution and an MV encoding solution based on the MVP candidate list (ie, the first candidate list mentioned above).

[0194] ①VDMC already has MV coding scheme

[0195] There are MV coding schemes such as Figure 4b As shown. When encoding MV, the vertices are grouped in ascending order of vertex index, with 16 vertices forming a group, and then the MV is encoded group by group in the traversal order. There are two encoding modes for each group of MVs: one is to directly encode the MV; the other is to predict the MV first and then encode the MVD. The encoder estimates the bit overhead of using these two encoding modes for each group of MVs, and selects the MV encoding mode that consumes fewer bits. Therefore, the encoding mode of each group needs to be identified by a flag bit. When encoding, the mode identifier is encoded first, and then the MV or MVD is encoded.

[0196] The MV prediction technology used in existing solutions is a relatively simple mean prediction technology, which uses the mean of the adjacent encoded MVs (up to 3) of the current vertex as the motion vector prediction value of the current MV. Figure 4c As shown in the figure, assuming that point D is the current point and points A, B, and C are adjacent points whose MVs have been encoded, the calculation formula for the MV prediction value of point D is as follows:

[0197] MVP D =ave(MV A ,MV b ,MV c )

[0198] Among them, MVP D is the MV prediction value of point D, MV A is the MV value of point A, MV b is the MV value of point B, MV c is the MV value of point C.

[0199] The calculation formula of the MV prediction residual value MVD of point D is as follows:

[0200] MVD D =MV D -MVP D

[0201] Among them, MVD D is the MV prediction residual value of point D, MV D is the MV value of D, MVP D is the predicted MV value of point D.

[0202] ②MV coding scheme based on MVP candidate list

[0203] like Figure 4d As shown, for the motion vector coding of the P subgrid, this application designs a MV coding scheme based on the MV prediction candidate list. When the encoding end encodes each MV, it first constructs the MV prediction candidate list according to fixed rules, then adaptively reorders the list, and then traverses the list to determine the coding mode and the best MVP according to the rate-distortion criterion. Finally, the MVP index and mode identifier are encoded in turn. Whether to encode MVD depends on the coding mode. The following is a detailed introduction to the three parts of MVP candidate list construction, adaptive reordering, coding mode and determination of the best MVP.

[0204] a) Build an MVP candidate list

[0205] Table 1. Default MVP candidate list

[0206]

[0207] The default construction of the MVP candidate list is shown in Table 1 above. The maximum length of the list is specified as MaxCand, and then N-point MVPs (N>1) and single-point MVPs are sequentially populated. There are two possible methods for calculating the N-point MVP: one is to calculate the average of N adjacent encoded MVs as the N-point MVP value; the other is to take a weighted average of the adjacent MVs based on the distance from the vertex corresponding to the adjacent encoded MVs to the current vertex, and use the weighted average as the N-point MVP value. Single-point MVP directly stores adjacent encoded MVs as MVPs in the list. When the number of adjacent encoded MVs is greater than one, single-point MVPs are sorted in ascending order of distance from the current vertex. It is important to note that when populating the MV prediction candidate list, duplicate MVP checks are performed. That is, if the current MVP value to be stored is equal to an existing MVP value in the list, the current MVP value is skipped. If, after the N-point MVP and single-point MVP values ​​are populated, the maximum value of the list, MaxCand, is not reached, the list is padded with zero vectors.

[0208] b) Adaptive reordering of MVP candidate list

[0209] MVP indexes are generally encoded using variable-length codes. The purpose of adaptive reordering is to place the MVP with a high probability of being selected at the head of the list, so that fewer bits can be used when encoding index information. Therefore, it is necessary to use some kind of adaptive reordering algorithm to reorder the candidate list. A possible MVP candidate list reordering algorithm is as follows: Figure 4e shown.

[0210] Specifically, the MVP candidate list of the current vertex is first constructed in the default order. Then, the MVP candidate list is traversed, and the error sum between each MVP and the adjacent encoded MV is calculated. Finally, the MVP list is reordered in ascending order of error sum.

[0211] c) Traverse the list to determine the encoding mode and the best MVP

[0212] To determine the coding mode and optimal MVP, the reordered MVP candidate list is traversed, and the rate-distortion cost J = D + λR for each MVP encoding mode is calculated, where D represents the distortion introduced by the encoding, λ is the Lagrange coefficient, and R is the number of bits spent encoding the MV information for the corresponding coding mode. When the coding mode encodes the MVD, the D component of the corresponding rate-distortion cost is 0; when the coding mode does not encode the MVD, D is non-zero.

[0213] One possible calculation of the distortion D is as follows:

[0214]

[0215] Among them, (MV-MVP) is used to measure the length difference between MV and MVP, (MV,MVP) is used to measure the angle between MV and MVP, and α and β are weight coefficients.

[0216] After the above three steps, the encoder has determined the encoding content of the MV information. The encoder will encode the best MVP index and mode flag in sequence, and then decide whether to encode the MVD based on the mode flag.

[0217] (4) Subdivision deformation

[0218] The subdivision and deformation module is applied to the input 3D mesh to generate displacement vector information. For example, the input 2D curve (represented by a 2D polyline), called the "original" curve, is first downsampled to generate a basic curve / polyline, called the "simplified" curve. The subdivision scheme is then applied to the simplified polyline to generate a "subdivided" curve. The subdivided polyline is then deformed to obtain a better approximation of the original curve. That is, a geometric displacement vector is calculated for each vertex of the subdivided mesh so that the shape of the subdivided curve is as close as possible to the shape of the original curve. These geometric displacement vectors are the geometric displacement vector information output by the module. The same deformation process is also applied to the attribute information corresponding to the vertex to obtain the corresponding attribute displacement vector.

[0219] The subdivision deformation module takes the parameterized sub-mesh as input. This step first subdivides the input mesh. The subdivision scheme can be chosen arbitrarily. One possible scheme is the midpoint subdivision scheme, which subdivides each triangle into four sub-triangles in each subdivision iteration, such as Figure 4f As shown in Figure 2, a new vertex is introduced in the middle of each edge, and the subdivision of geometric information and attribute information is performed independently because the connection relationship between geometric information and attribute information is usually different.

[0220] The scheme calculates the midpoint v of the newly introduced edge (v1, v2) 12 Position Pos(v 12 ) is shown in formula (1):

[0221]

[0222] Among them, Pos(v1) and Pos(v2) are the geometric coordinates of vertices v1 and v2 respectively.

[0223] For each point in the subdivided mesh, we find the nearest neighbor of each point on the original input mesh (including points on the original mesh surface). This can be accelerated using data structures such as kdTree. We calculate the distance between each vertex on the subdivided mesh and the geometric coordinates of its nearest neighbor on the original input mesh to obtain the displacement vector of each vertex's geometric coordinates. This module passes the generated displacement vector to subsequent modules for encoding.

[0224] At the same time, for the generated displacement vector, which is in the same global coordinate system as the input mesh, a possible optimization method is to transform it into a local coordinate system, where the local coordinate system of each vertex is defined by the normal vector of the vertex on the subdivided mesh. The advantage of this method is that the normal component of the geometric displacement vector has a more significant impact on the quality of the reconstructed mesh than the two tangential components. Therefore, a larger quantization parameter can be set for the tangential component.

[0225] (5) Wavelet transform

[0226] A transformation can be applied to the displacement vector to reduce the correlation between its data. An optional transformation is linear wavelet transform, and its prediction process is defined as shown in formula (2):

[0227]

[0228] Where v is the newly inserted midpoint on the edge (v1, v2), Signal(v), Signal(v1), and Signal(v2) are the displacement vectors corresponding to vertices v, v1, and v2, respectively. The displacement vector of vertex v is predicted and then updated. The update process is defined as shown in Equation (3):

[0229]

[0230] Where v* is the set of all vertices adjacent to vertex v. The transformed displacement vector is called the wavelet coefficient.

[0231] (6) Coefficient quantization

[0232] The transformed displacement vector, i.e., the wavelet coefficient, can be quantized. There are many ways to quantize it. One method is shown in equations (4) and (5):

[0233] disp[v].d[k]=floor(disp[v].d[k]*scale[k])#(4)

[0234]

[0235] Where disp[v] represents the transformed value of the displacement vector at the vth vertex, d[k] represents the kth value of the displacement vector, and floor indicates rounding down. bitDepthPosition represents the bit depth of the current mesh vertex's geometric position, and qp[k] represents the quantization parameter for the kth coefficient. As mentioned earlier, after transforming the coordinate system of the displacement vector, its normal component has a more significant impact on quality than its tangential component, so a larger quantization parameter can be used for the tangential component.

[0236] At the same time, according to the characteristics of wavelet transform, different quantization parameters can be used for the newly generated vertices and the original vertices. That is, for the subdivided vertices, the quantization parameter update is shown in formula (6):

[0237] scale[k]=scale[k]*lodScale[k]#(6)

[0238] Among them, lodScale[k] represents the coefficient of the quantization parameter of the current subdivision level.

[0239] (7) Displacement encoding

[0240] The displacement coding part uses the quantized wavelet coefficients as input for video encoding. The quantized wavelet coefficients need to be arranged into a two-dimensional image. One arrangement method is as follows:

[0241] Traverse the wavelet coefficients in order from low frequency to high frequency;

[0242] For each coefficient, determine the index of the NxM pixel block (e.g., N=M=16) where it should be stored in raster scan order of the block;

[0243] The position of the corresponding NxM pixel block on the image is calculated according to the Morton order.

[0244] The arrangement is not limited here, and other arrangement schemes may also be used, such as zigzag order, raster order, etc. The encoder may explicitly specify the corresponding arrangement scheme in the bitstream.

[0245] After arranging the wavelet coefficients on a two-dimensional image, they can be directly encoded using a video encoder. The proposed scheme can use any existing video encoder, and the type of video encoder needs to be encoded in the auxiliary information.

[0246] (8) Deformed mesh reconstruction

[0247] The displacement encoding module obtains the reconstructed displacement, i.e., obtains the displacement vector consistent with the decoding end through inverse quantization and inverse transformation. After obtaining the reconstructed geometric displacement vector, the reconstructed base mesh is subdivided and the reconstructed subdivided and deformed mesh is obtained based on the corresponding displacement vector, which is then passed to the texture map conversion module.

[0248] (9) Texture conversion

[0249] The texture map conversion module performs texture map conversion based on the input original mesh, the input original texture map and the reconstructed deformed mesh.

[0250] The steps for texture conversion are as follows:

[0251] Calculate the texture coordinates of each pixel on the texture map to be generated, for example, the texture coordinates corresponding to pixel A(i,j) are P(u,v);

[0252] Determine whether the texture coordinate is within a certain triangle face after the parameterization of the subdivided deformed mesh;

[0253] If the texture coordinate does not belong to any triangle, the pixel is marked as an empty pixel and can be filled with a filling algorithm later;

[0254] If the texture coordinate belongs to a triangle, then:

[0255] Mark the pixel as filled;

[0256] Calculate the center of gravity coordinates of the texture in the current triangle according to the texture coordinates;

[0257] According to the barycentric coordinates and the corresponding triangular face, the two-dimensional texture coordinates are mapped to three-dimensional geometric coordinates, that is, mapped to the points on the subdivided deformed grid corresponding to the texture coordinates;

[0258] Find the point closest to the three-dimensional coordinate on the input original grid;

[0259] Calculate the barycentric coordinates of the three-dimensional coordinates according to the triangle face on which they are located and map them to two dimensions to calculate their texture coordinates;

[0260] The texture coordinates are used to sample the input original texture map to obtain the value A′(i′, j′) of the corresponding pixel position;

[0261] Assign this value to the corresponding pixel A(i,j) on the texture map to be generated.

[0262] (10) Texture image compression

[0263] After obtaining the converted texture map, existing filling algorithms (such as the Push-Pull algorithm) can be used to fill empty pixels. This can then be encoded using existing video encoders such as H.264 / AVC, H.265 / HEVC, and H.266 / VVC to produce the output texture map bitstream. Furthermore, color space conversion and chroma subsampling can be selectively applied to achieve better rate-distortion performance in video encoding, such as converting from RGB444 to YUV420.

[0264] (11) Auxiliary information

[0265] During the encoding process, various modules can employ alternative schemes, such as the type of grid encoder, the type of video encoder, the grid subdivision scheme, the displacement vector transformation scheme, and the displacement encoding method. The proposed framework allows for the use of different schemes. Therefore, the selected scheme needs to be communicated to the decoder to guide correct decoding.

[0266] After all modules are encoded, the basic grid part stream, texture coordinate part stream, displacement vector video stream, attribute map video stream and auxiliary information stream are mixed to obtain the final encoded stream output by the encoding end.

[0267] The three-dimensional grid decoding method of the decoding end specifically includes the following process (see Figure 1b Flowchart shown):

[0268] (1) Auxiliary information decoding

[0269] The decoding end first determines the decoding scheme based on the auxiliary information, which mainly includes the displacement coding method, indicating whether the displacement is encoded by the video encoder or the entropy encoder; the static grid encoder type, which guides the decoding end to use the corresponding static grid decoder; the video encoder type, which guides the decoding end to use the corresponding video decoder; the subdivision scheme, that is, the scheme for reconstructing the basic grid subdivision in the deformed grid, and the subdivision scheme of the encoder and decoder should be consistent; there are also optional spatial domain displacement transformation schemes, coefficient arrangement schemes, etc.

[0270] (2) Basic grid decoding

[0271] Basic grid decoding can be categorized into intra-mode decoding, inter-mode decoding, and skip-mode decoding. The basic grid decoding module feeds different bitstream segments into the corresponding decoder for decoding based on the subgrid type identifier in the bitstream. After subgrid decoding, the subgrids are merged and concatenated to generate a basic grid for decoding and reconstruction. The bitstream segment corresponding to the I subgrid is fed into the intra-mode decoder for decoding; the bitstream segment corresponding to the P subgrid is fed into the inter-mode decoder for decoding; and the bitstream segment corresponding to the skip subgrid is fed into the skip-mode decoder for decoding.

[0272] The decoding process of the P subgrid in the above process is roughly as follows:

[0273] Decode the reference information and obtain the reference subgrid information of the corresponding reference frame from the decoding buffer;

[0274] Decode the motion vectors of each vertex in the sub-mesh and use them to correct the vertex positions of the reference sub-mesh.

[0275] For the second step of the P subgrid decoding process, there are currently two feasible MV decoding schemes: the existing MV decoding scheme and the MV decoding scheme based on the MVP candidate list. These two MV decoding schemes are now described in detail.

[0276] ① Existing MV decoding solution

[0277] The existing process for decoding P subgrid MV is as follows Figure 5a As shown. First, the coding mode of each group is decoded. After decoding, the coding mode of all vertex MVs in the group can be obtained. If the decoded mode identifier indicates that the encoder directly encodes the MV, the code stream is directly decoded and output to obtain the MV of each vertex in the group; if the decoded identifier indicates that the encoder encodes the MVD, it is necessary to use the decoded MVs of adjacent vertices to predict the MV of the current vertex to generate the MVP, and then add the decoded MVD and MVP and output them as the decoded MV.

[0278] ②MV decoding scheme based on MVP candidate list

[0279] This application designs a MV decoding scheme based on the MVP candidate list, such as Figure 5b As shown, the MV decoding process is as follows:

[0280] Utilize adjacent decoded and reconstructed MVs to construct a MVP candidate list and adaptively reorder it;

[0281] Decode the best MVP index and find the corresponding MVP in the MVP candidate list;

[0282] Decoding MV coding mode: If the coding mode is encoding MVD, the MVD is further decoded and then added to the MVP as the current decoded MV output; if the coding mode is not encoding MVD, the MVP is directly output as the decoded MV.

[0283] It should be noted that, in the process of constructing the MVP candidate list and adaptive reordering at the decoding end, except for using adjacent decoded MVs, the rest of the process is exactly the same as that at the encoding end, and will not be described in detail here.

[0284] (3) Displacement decoding module

[0285] In the displacement decoding module, it is necessary to determine the decoding method of the displacement based on the auxiliary information identifier. If the auxiliary information indicates that the displacement information is encoded through the video encoder, the decoding end calls the corresponding video decoder to decode the displacement code stream; if the auxiliary information indicates that the displacement information is encoded through the entropy encoder, the entropy decoder is directly used for decoding.

[0286] The decoded displacement information must be reconstructed using a displacement reconstruction module to obtain the displacement vectors corresponding to the subdivided mesh vertices. This module primarily performs inverse quantization and inverse transformation on the decoded displacement information, i.e., the wavelet coefficients, using the quantization and transform parameters indicated by the auxiliary information. Furthermore, for the decoded video information, the corresponding displacement information must first be extracted from the 2D image according to the encoding arrangement.

[0287] (4) Segmentation

[0288] The subdivision module operates in the same way as the encoding end, with auxiliary information indicating the subdivision method of the base grid and the number of iterations.

[0289] (5) Reconstruct the deformed mesh

[0290] After the base mesh and the displacement vector are decoded and reconstructed, the deformed mesh is reconstructed based on the two parts. The corresponding displacement vector is added to each vertex of the subdivided mesh, as shown in formula (7):

[0291] deformedmesh[i].v[k]=subdivmesh[i].v[k]+displacement[k]#(7)

[0292] Among them, subdivmesh[i].v[k] is the geometric coordinates of the k-th vertex after the base mesh of the current frame (index is i) is subdivided, displacement[k] is the spatial displacement vector corresponding to the k-th vertex, and deformedmesh[i].v[k] is the geometric coordinates of the k-th vertex after subdivision and deformation in the current frame.

[0293] (6) Texture image decoding

[0294] The texture decoder is responsible for decoding the texture stream using the video decoder indicated in the auxiliary information. It then performs an optional color space conversion to obtain an image format consistent with the input texture image at the encoder, resulting in the final decoded texture output.

[0295] After each module is processed, the decoder finally obtains the reconstructed deformed mesh and the corresponding attribute map. Subsequent applications use the reconstructed deformed mesh and attribute map as input for processing.

[0296] The 3D grid inter-frame prediction method provided in the embodiment of the present application can be executed by a 3D grid inter-frame prediction device. In the embodiment of the present application, the 3D grid inter-frame prediction method is executed by the 3D grid inter-frame prediction device as an example to illustrate the 3D grid inter-frame prediction device provided in the embodiment of the present application.

[0297] Please refer to Figure 6 , Figure 6 This is a structural diagram of a three-dimensional grid inter-frame prediction coding device provided by an embodiment of the present application. Figure 6 As shown, the three-dimensional grid inter-frame prediction encoding device 600 includes:

[0298] A processing module 601 is configured to perform a first processing on the three-dimensional grid to be encoded to obtain a basic grid;

[0299] A division module 602 is configured to divide the base grid into subgrids to obtain P subgrids;

[0300] A first determination module 603 is configured to determine a target coding mode and a first target motion vector prediction MVP value for a mesh vertex to be encoded in the P sub-mesh, where the first target MVP value is one of N MVP values ​​included in a first candidate list, where each MVP value in the first candidate list corresponds to an index, and N is an integer greater than 1;

[0301] The encoding module 604 is configured to encode the target coding mode and the index corresponding to the first target MVP value to obtain a code stream including first information, where the first information is used to represent the target coding mode and the index corresponding to the first target MVP value.

[0302] Optionally, the device further comprises:

[0303] A construction module is used to construct the first candidate list based on the encoded mesh vertices adjacent to the mesh vertex to be encoded in the P sub-grid.

[0304] Optionally, the building block is further configured to:

[0305] Obtain L adjacent already-encoded mesh vertices and M adjacent already-encoded mesh vertices of the mesh vertex to be encoded in the P sub-grid, where L and M are integers greater than 1;

[0306] Obtaining a motion vector MV value of each of the L adjacent encoded mesh vertices, and determining L first MVP values ​​of the to-be-encoded mesh based on the MV values ​​of the L adjacent encoded mesh vertices;

[0307] Determine a second MVP value based on the MV values ​​corresponding to each of the M encoded mesh vertices;

[0308] The first candidate list is constructed based on L first MVP values ​​and L second MVP values, where L+1≤N.

[0309] Optionally, the building block is further configured to:

[0310] Obtaining MV values ​​corresponding to the M encoded mesh vertices, and determining an average of the M MV values ​​as the second MVP value; or,

[0311] Obtain the MV values ​​corresponding to the M encoded mesh vertices, perform weighted average calculation on the M MV values, and determine the calculated result as the second MVP value.

[0312] Optionally, an arrangement order of the L first MVP values ​​in the first candidate list is related to a first distance, where the first distance is a distance between a coded mesh vertex corresponding to the first MVP value and the mesh vertex to be coded.

[0313] Optionally, the L first MVP values ​​are arranged in ascending order of the first distance.

[0314] Optionally, in the case where L+1<N, the building block is further configured to:

[0315] Constructing a sub-candidate list based on the L first MVP values ​​and the second MVP values;

[0316] Perform a zero-padding operation on the sub-candidate list to obtain the first candidate list including N MVP values.

[0317] Optionally, the device further comprises:

[0318] a reordering module, configured to reorder the N MVP values ​​in the first candidate list to obtain a reordered first candidate list;

[0319] The first determining module is further configured to:

[0320] The encoder determines a first target MVP value from the reordered first candidate list.

[0321] Optionally, the reordering module is further configured to:

[0322] Obtaining a sum of errors between a second target MVP value and a motion vector MV of an encoded mesh vertex adjacent to the mesh vertex to be encoded, wherein the second target MVP value is one of the N MVP values;

[0323] The N MVP values ​​are reordered based on the sum of errors corresponding to each MVP value in the first candidate list.

[0324] Optionally, the N MVP values ​​in the reordered first candidate list are arranged in ascending order according to corresponding errors.

[0325] Optionally, the first determining module is further configured to:

[0326] Obtaining a first rate-distortion cost in a first coding mode and a second rate-distortion cost in a second coding mode for each MVP value in the first candidate list, to obtain N first rate-distortion costs and N second rate-distortion costs;

[0327] A first target MVP value is determined from the N MVP values ​​according to the N first rate-distortion costs and the N second rate-distortion costs.

[0328] Optionally, the first rate-distortion cost or the second rate-distortion cost corresponding to the first target MVP value is the smallest one of N first rate-distortion costs and N second rate-distortion costs.

[0329] Optionally, when a first rate-distortion cost corresponding to the first target MVP value is the smallest one of N first rate-distortion costs and N second rate-distortion costs, the target coding mode is the first coding mode; or,

[0330] In a case where the second rate-distortion cost corresponding to the first target MVP value is the smallest one of N first rate-distortion costs and N second rate-distortion costs, the target coding mode is the second coding mode.

[0331] Optionally, the first encoding mode is a mode for encoding directly based on the MVP value, and the second encoding mode is a mode for encoding based on the MVP value and the motion vector residual (MVD) value;

[0332] Wherein, when the target coding mode is the second coding mode, the coding module is further configured to:

[0333] The MVD of the to-be-encoded grid in the P subgrid is encoded.

[0334] Optionally, the target rate-distortion cost is related to a first length difference and a first angle, wherein the first length difference is a difference between a modulus length of a third target MVP value and a modulus length of an MV value of the to-be-encoded mesh vertex, and the first angle is an angle between the third target MVP value and the MV value of the to-be-encoded mesh vertex;

[0335] The target rate-distortion cost is the first rate-distortion cost or the second rate-distortion cost, and the third target MVP value is one of the N MVP values ​​in the first candidate list.

[0336] In an embodiment of the present application, the encoding end is no longer limited to using a fixed motion vector prediction method for encoding, but can flexibly select the first target MVP value from the first candidate list for motion vector prediction, so that the encoding method of the encoding end for the P sub-grid motion vector is more flexible, thereby helping to improve the accuracy of the encoding end for the P sub-grid inter-frame prediction, and also helping to improve the encoding efficiency.

[0337] The three-dimensional grid inter-frame prediction coding device in the embodiments of the present application can be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or chip. The electronic device can be a terminal or other device other than a terminal. For example, the terminal can include but is not limited to the types of terminals listed above, and other devices can be servers, network attached storage (NAS), etc., which are not specifically limited in the embodiments of the present application.

[0338] The three-dimensional grid inter-frame prediction coding device provided by the embodiment of the present application can achieve Figure 2 The various processes implemented in the method embodiment achieve the same technical effects, and to avoid repetition, they will not be described here.

[0339] Please refer to Figure 7 , Figure 7 This is a structural diagram of a three-dimensional grid inter-frame prediction decoding device provided by an embodiment of the present application. Figure 7 As shown, the three-dimensional grid inter-frame prediction decoding device 700 includes:

[0340] An acquisition module 701 is configured to acquire a code stream sent by an encoding end, wherein the code stream includes a basic grid code stream;

[0341] A first decoding module 702 is configured to perform sub-grid type decoding on the basic grid code stream to obtain a P sub-grid code stream including first information, where the first information is used to represent a target coding mode and an index corresponding to a first target MVP value;

[0342] a second determining module 703, configured to determine a target coding mode according to the first information, and determine a first target MVP value from a first candidate list according to an index corresponding to the first target MVP value, the first candidate list including N MVP values ​​and an index corresponding to each MVP value, the first target MVP value being one of the N MVP values, where N is an integer greater than 1;

[0343] The second decoding module 704 is configured to decode the P sub-grid code stream according to the target coding mode and the first target MVP value to obtain the MV value of the grid vertex to be decoded in the P sub-grid.

[0344] Optionally, the device further comprises:

[0345] A construction module is configured to construct the first candidate list based on decoded mesh vertices adjacent to the mesh vertex to be decoded in the P sub-grid.

[0346] Optionally, the building block is further configured to:

[0347] Obtain L decoded mesh vertices and M decoded mesh vertices adjacent to the mesh vertex to be decoded in the P sub-grid, where L and M are integers greater than 1;

[0348] Obtaining the MV value of each decoded mesh vertex among the adjacent L decoded mesh vertices, and determining L first MVP values ​​of the mesh vertex to be decoded based on the MV values ​​of the L decoded mesh vertices;

[0349] Determine a second MVP value based on the MV values ​​corresponding to each of the M decoded mesh vertices;

[0350] The first candidate list is constructed based on L first MVP values ​​and L second MVP values, where L+1≤N.

[0351] Optionally, the building block is further configured to:

[0352] Obtaining MV values ​​corresponding to the M decoded mesh vertices, and determining an average of the M MV values ​​as the second MVP value; or,

[0353] Obtain the MV values ​​corresponding to the M decoded mesh vertices, perform weighted average calculation on the M MV values, and determine the calculated result as the second MVP value.

[0354] Optionally, an arrangement order of the L first MVP values ​​in the first candidate list is related to a first distance, where the first distance is a distance between a decoded mesh vertex corresponding to the first MVP value and the mesh vertex to be decoded.

[0355] Optionally, the L first MVP values ​​are arranged in ascending order of the first distance.

[0356] Optionally, in the case where L+1<N, the building block is further configured to:

[0357] Constructing a sub-candidate list based on the L first MVP values ​​and the second MVP values;

[0358] Perform a zero-padding operation on the sub-candidate list to obtain the first candidate list including N MVP values.

[0359] Optionally, the device further comprises:

[0360] a reordering module, configured to reorder the N MVP values ​​in the first candidate list to obtain a reordered first candidate list;

[0361] The second determining module is further configured to:

[0362] The decoding end determines a first target MVP value from the reordered first candidate list according to an index corresponding to the first target MVP value.

[0363] Optionally, the reordering module is further configured to:

[0364] Obtaining a sum of errors between a second target MVP value and MV values ​​of decoded mesh vertices adjacent to the to-be-decoded mesh vertex, where the second target MVP value is one of the N MVP values;

[0365] The N MVP values ​​are reordered based on the sum of errors corresponding to each MVP value in the first candidate list.

[0366] Optionally, the N MVP values ​​in the reordered first candidate list are arranged in ascending order according to corresponding errors.

[0367] Optionally, the target coding mode is a first coding mode or a second coding mode, the first coding mode is a mode for encoding directly based on the MVP value, and the second coding mode is a mode for encoding based on the MVP value and the MVD value.

[0368] Optionally, when the target coding mode is the second coding mode, the P sub-grid code stream also includes an MVD value, and the second decoding module is further configured to:

[0369] Performing MVD decoding on the P sub-grid code stream to obtain MVD values ​​corresponding to the grid vertices to be decoded in the P sub-grid;

[0370] The MV value corresponding to the to-be-decoded mesh vertex in the P sub-mesh is determined according to the MVD value corresponding to the to-be-decoded mesh vertex and the first target MVP value.

[0371] In an embodiment of the present application, the decoding end is no longer limited to decoding the motion vector of the P sub-grid through a fixed MV decoding method, but can adopt the same method as the encoding end to construct a first candidate list and determine the first target MVP value from the first candidate list.

[0372] The three-dimensional grid inter-frame prediction decoding device provided by the embodiment of the present application can achieve Figure 3 The various processes implemented in the method embodiment achieve the same technical effects, and to avoid repetition, they will not be described here.

[0373] like Figure 8 As shown, the embodiment of the present application further provides an electronic device 800, including a processor 801 and a memory 802, wherein the memory 802 stores a program or instruction that can be run on the processor 801. For example, when the electronic device 800 is an encoding end, the program or instruction is executed by the processor 801 to implement the above Figure 2 When the electronic device 800 is a decoding end, the program or instruction is executed by the processor 801 to achieve the above Figure 3 The various steps of the method embodiment can achieve the same technical effect, and to avoid repetition, they will not be described here.

[0374] The embodiment of the present application also provides an electronic device, including a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the following Figure 2 or Figure 3 The steps in the method embodiment shown, the various implementation processes and implementation methods of the above method embodiment are all applicable to the electronic device embodiment and can achieve the same technical effects. Specifically, Figure 9 A schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0375] The electronic device 900 includes but is not limited to: a radio frequency unit 901, a network module 902, an audio output unit 903, an input unit 904, a sensor 905, a display unit 906, a user input unit 907, an interface unit 908, a memory 909 and at least some of the components of the processor 910.

[0376] Those skilled in the art will understand that the electronic device 900 may also include a power source (such as a battery) to power each component, and the power source may be logically connected to the processor 910 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system. Figure 9 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be repeated here.

[0377] It should be understood that in an embodiment of the present application, the input unit 904 may include a graphics processing unit (GPU) 9041 and a microphone 9042, and the graphics processor 9041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 906 may include a display panel 9061, and the display panel 9061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 907 includes a touch panel 9071 and at least one of other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 may include two parts: a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and an operating stick, which will not be repeated here.

[0378] In the embodiment of the present application, after receiving downlink data from a network-side device, the RF unit 901 may transmit the data to the processor 910 for processing. Furthermore, the RF unit 901 may send uplink data to the network-side device. Typically, the RF unit 901 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and the like.

[0379] The memory 909 can be used to store software programs or instructions and various data. The memory 909 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 909 may include a volatile memory or a non-volatile memory 90. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct memory bus random access memory (DRRAM). The memory 909 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0380] Processor 910 may include one or more processing units. Optionally, processor 910 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 910.

[0381] When the electronic device 900 is an encoding end, the processor 910 is configured to:

[0382] Performing a first processing on the three-dimensional grid to be encoded to obtain a basic grid;

[0383] Performing subgrid division on the base grid to obtain P subgrids;

[0384] Determine a target coding mode and a first target motion vector prediction MVP value for a mesh vertex to be encoded in the P sub-grid, where the first target MVP value is one of N MVP values ​​included in a first candidate list, each MVP value in the first candidate list corresponds to an index, and N is an integer greater than 1;

[0385] The target coding mode and the index corresponding to the first target MVP value are encoded to obtain a code stream including first information, where the first information is used to represent the target coding mode and the index corresponding to the first target MVP value.

[0386] Optionally, the electronic device 900 can implement all the technical processes in the above-mentioned encoding end method embodiment and achieve the same technical effect. To avoid repetition, they are not described here.

[0387] When the electronic device 900 is a decoding end, the processor 910 is configured to:

[0388] Obtaining a code stream sent by an encoding end, wherein the code stream includes a basic grid code stream;

[0389] Performing sub-grid type decoding on the basic grid code stream to obtain a P sub-grid code stream including first information, where the first information is used to represent a target coding mode and an index corresponding to a first target MVP value;

[0390] Determining a target coding mode according to the first information, and determining a first target MVP value from a first candidate list according to an index corresponding to the first target MVP value, the first candidate list including N MVP values ​​and an index corresponding to each MVP value, the first target MVP value being one of the N MVP values, where N is an integer greater than 1;

[0391] The P sub-grid code stream is decoded according to the target coding mode and the first target MVP value to obtain the MV value of the grid vertex to be decoded in the P sub-grid.

[0392] Optionally, the electronic device 900 can implement all the technical processes in the above-mentioned decoding end method embodiment and achieve the same technical effect. To avoid repetition, they are not described here.

[0393] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned encoding end or decoding end method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0394] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. In some examples, the readable storage medium may be a non-transitory readable storage medium.

[0395] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned encoding or decoding method embodiments, and can achieve the same technical effects. To avoid repetition, they will not be repeated here.

[0396] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0397] An embodiment of the present application further provides a computer program / program product, which is stored in a storage medium. The computer program / program product is executed by at least one processor to implement the various processes of the above-mentioned encoding or decoding method embodiments, and can achieve the same technical effects. To avoid repetition, they are not described here.

[0398] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0399] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of a computer software product plus a necessary general hardware platform, or of course, by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes a number of instructions for enabling an electronic device or network-side device to execute the methods described in each embodiment of the present application.

[0400] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms of implementation methods without departing from the purpose of this application and the scope of protection of the claims. These implementation methods are all within the protection of this application.

Claims

1. A three-dimensional grid inter-frame prediction coding method, characterized in that: include: The encoding end performs a first processing on the three-dimensional grid to be encoded to obtain a basic grid; The encoding end divides the basic grid into subgrids to obtain P subgrids; The encoder determines a target coding mode and a first target motion vector prediction MVP value for a mesh vertex to be encoded in the P sub-mesh, where the first target MVP value is one of N MVP values ​​included in a first candidate list, each MVP value in the first candidate list corresponds to an index, and N is an integer greater than 1; The encoding end encodes the target coding mode and the index corresponding to the first target MVP value to obtain a code stream including first information, where the first information is used to represent the target coding mode and the index corresponding to the first target MVP value.

2. The method according to claim 1, characterized in that Before the encoder determines the target coding mode and the first target MVP value of the mesh vertices to be encoded in the P sub-grid, the method further includes: The encoding end constructs the first candidate list based on encoded mesh vertices adjacent to the mesh vertex to be encoded in the P sub-grid.

3. The method according to claim 2, characterized in that The encoder constructs the first candidate list based on coded mesh vertices adjacent to the mesh vertex to be coded in the P sub-grid, including: The encoder obtains L adjacent already-encoded mesh vertices and M adjacent already-encoded mesh vertices of the mesh vertex to be encoded in the P sub-grid, where L and M are integers greater than 1; The encoder obtains a motion vector MV value of each of the L adjacent encoded mesh vertices, and determines L first MVP values ​​of the to-be-encoded mesh based on the MV values ​​of the L encoded mesh vertices; The encoding end determines a second MVP value based on the MV values ​​corresponding to the M encoded mesh vertices; The encoding end constructs the first candidate list based on L first MVP values ​​and the second MVP value, where L+1≤N.

4. The method according to claim 3, characterized in that The encoder determines a second MVP value based on the MV values ​​corresponding to the M encoded mesh vertices, including: The encoding end obtains the MV values ​​corresponding to the M encoded mesh vertices, and determines the average of the M MV values ​​as the second MVP value; or The encoding end obtains the MV values ​​corresponding to the M encoded mesh vertices, performs weighted average calculation on the M MV values, and determines the calculated result as the second MVP value.

5. The method according to claim 3 or 4, characterized in that An arrangement order of the L first MVP values ​​in the first candidate list is related to a first distance, where the first distance is a distance between a coded mesh vertex corresponding to the first MVP value and the mesh vertex to be coded.

6. The method according to claim 5, characterized in that The L first MVP values ​​are arranged in ascending order according to the first distance.

7. The method according to claim 3 or 4, characterized in that When L+1<N, the encoding end constructs the first candidate list based on L first MVP values ​​and the second MVP value, including: The encoding end constructs a sub-candidate list based on the L first MVP values ​​and the second MVP value; The encoding end performs a zero-padding operation on the sub-candidate list to obtain the first candidate list including N MVP values.

8. The method according to any one of claims 2 to 4, characterized in that After the encoder constructs the first candidate list based on coded mesh vertices adjacent to the mesh vertex to be coded in the P sub-grid, the method further includes: The encoder reorders the N MVP values ​​in the first candidate list to obtain a reordered first candidate list; Determining the first target MVP value includes: The encoder determines a first target MVP value from the reordered first candidate list.

9. The method according to claim 8, characterized in that The encoder reorders the N MVP values ​​in the first candidate list, including: The encoder obtains, the sum of errors between a second target MVP value and a motion vector MV of an encoded mesh vertex adjacent to the mesh vertex to be encoded, where the second target MVP value is one of the N MVP values; The encoding end reorders the N MVP values ​​based on the error corresponding to each MVP value in the first candidate list.

10. The method according to claim 9, characterized in that The N MVP values ​​in the reordered first candidate list are arranged in ascending order according to the corresponding errors.

11. The method according to any one of claims 1 to 4, characterized in that Determining the first target MVP value includes: The encoder obtains, by the encoder end, a first rate-distortion cost of each MVP value in the first candidate list in the first coding mode and a second rate-distortion cost in the second coding mode, to obtain N first rate-distortion costs and N second rate-distortion costs; The encoder determines a first target MVP value from the N MVP values ​​according to the N first rate-distortion costs and the N second rate-distortion costs.

12. The method according to claim 11, characterized in that The first rate-distortion cost or the second rate-distortion cost corresponding to the first target MVP value is the smallest one of the N first rate-distortion costs and the N second rate-distortion costs.

13. The method according to claim 11, characterized in that When a first rate-distortion cost corresponding to the first target MVP value is the smallest one of N first rate-distortion costs and N second rate-distortion costs, the target coding mode is the first coding mode; or, In a case where the second rate-distortion cost corresponding to the first target MVP value is the smallest one of N first rate-distortion costs and N second rate-distortion costs, the target coding mode is the second coding mode.

14. The method according to claim 13, characterized in that The first coding mode is a mode for encoding directly based on the MVP value, and the second coding mode is a mode for encoding based on the MVP value and the motion vector residual (MVD) value; Wherein, when the target coding mode is the second coding mode, the method further includes: The encoding end encodes the MVD of the grid to be encoded in the P sub-grid.

15. The method according to claim 11, characterized in that The target rate-distortion cost is related to a first length difference and a first angle, wherein the first length difference is a difference between a modulus length of a third target MVP value and a modulus length of an MV value of the to-be-encoded mesh vertex, and the first angle is an angle between the third target MVP value and the MV value of the to-be-encoded mesh vertex; The target rate-distortion cost is the first rate-distortion cost or the second rate-distortion cost, and the third target MVP value is one of the N MVP values ​​in the first candidate list.

16. A three-dimensional grid inter-frame prediction decoding method, characterized in that: include: The decoding end obtains the code stream sent by the encoding end, wherein the code stream includes the basic grid code stream; The decoding end performs sub-grid type decoding on the basic grid code stream to obtain a P sub-grid code stream including first information, where the first information is used to represent the target coding mode and the index corresponding to the first target MVP value; The decoding end determines a target coding mode according to the first information, and determines a first target MVP value from a first candidate list according to an index corresponding to the first target MVP value, the first candidate list including N MVP values ​​and an index corresponding to each MVP value, the first target MVP value being one of the N MVP values, where N is an integer greater than 1; The decoding end decodes the P sub-grid code stream according to the target coding mode and the first target MVP value to obtain the MV value of the grid vertex to be decoded in the P sub-grid.

17. The method according to claim 16, characterized in that Before determining the first target MVP value from the first candidate list according to the index corresponding to the target MVP value, the method further includes: The decoding end constructs the first candidate list based on decoded mesh vertices adjacent to the mesh vertex to be decoded in the P sub-mesh.

18. The method according to claim 17, characterized in that The decoding end constructs the first candidate list based on decoded mesh vertices adjacent to the mesh vertex to be decoded in the P sub-mesh, including: The decoding end obtains L decoded mesh vertices and M decoded mesh vertices adjacent to the mesh vertex to be decoded in the P sub-grid, where L and M are integers greater than 1; The decoding end obtains the MV value of each decoded mesh vertex among the adjacent L decoded mesh vertices, and determines L first MVP values ​​of the mesh vertex to be decoded based on the MV values ​​of the L decoded mesh vertices; The decoding end determines a second MVP value based on the MV values ​​corresponding to the M decoded mesh vertices; The decoding end constructs the first candidate list based on L first MVP values ​​and the second MVP value, where L+1≤N.

19. The method according to claim 18, characterized in that The decoding end determines a second MVP value based on the MV values ​​corresponding to the M decoded mesh vertices, including: The decoding end obtains the MV values ​​corresponding to the M decoded mesh vertices, and determines the average of the M MV values ​​as the second MVP value; or The decoding end obtains the MV values ​​corresponding to the M decoded mesh vertices, performs weighted average calculation on the M MV values, and determines the calculated result as the second MVP value.

20. The method according to claim 18 or 19, characterized in that An arrangement order of the L first MVP values ​​in the first candidate list is related to a first distance, where the first distance is a distance between a decoded mesh vertex corresponding to the first MVP value and the mesh vertex to be decoded.

21. The method according to claim 20, characterized in that The L first MVP values ​​are arranged in ascending order according to the first distance.

22. The method according to claim 18 or 19, characterized in that In the case where L+1<N, the decoding end constructs the first candidate list based on L first MVP values ​​and the second MVP value, including: The decoding end constructs a sub-candidate list based on the L first MVP values ​​and the second MVP value; The decoding end performs a zero-padding operation on the sub-candidate list to obtain the first candidate list including N MVP values.

23. The method according to any one of claims 17 to 19, characterized in that After the decoding end constructs the first candidate list based on decoded mesh vertices adjacent to the mesh vertex to be decoded in the P sub-mesh, the method further includes: The decoding end reorders the N MVP values ​​in the first candidate list to obtain a reordered first candidate list; The determining the first target MVP value from the first candidate list according to the index corresponding to the first target MVP value includes: The decoding end determines a first target MVP value from the reordered first candidate list according to an index corresponding to the first target MVP value.

24. The method according to claim 23, wherein The decoding end reorders the N MVP values ​​in the first candidate list, including: The decoding end obtains a sum of errors between a second target MVP value and MV values ​​of decoded mesh vertices adjacent to the to-be-decoded mesh vertex, where the second target MVP value is one of the N MVP values; The decoding end reorders the N MVP values ​​based on the error corresponding to each MVP value in the first candidate list.

25. The method according to claim 24, characterized in that The N MVP values ​​in the reordered first candidate list are arranged in ascending order according to the corresponding errors.

26. The method according to any one of claims 16 to 19, characterized in that The target coding mode is a first coding mode or a second coding mode, the first coding mode is a mode for encoding directly based on the MVP value, and the second coding mode is a mode for encoding based on the MVP value and the MVD value.

27. The method according to claim 26, characterized in that When the target coding mode is the second coding mode, the P subgrid code stream further includes an MVD value, and the method further includes: The decoding end performs MVD decoding on the P sub-grid code stream to obtain MVD values ​​corresponding to the grid vertices to be decoded in the P sub-grid; The step of obtaining the MV value of the to-be-decoded mesh vertex in the P sub-mesh includes: The MV value corresponding to the to-be-decoded mesh vertex in the P sub-mesh is determined according to the MVD value corresponding to the to-be-decoded mesh vertex and the first target MVP value.

28. A three-dimensional grid inter-frame prediction encoding device, characterized in that: include: a processing module, configured to perform a first processing on the three-dimensional grid to be encoded to obtain a basic grid; A division module, configured to divide the base grid into subgrids to obtain P subgrids; A first determination module is configured to determine a target coding mode and a first target motion vector prediction MVP value for a mesh vertex to be encoded in the P sub-mesh, where the first target MVP value is one of N MVP values ​​included in a first candidate list, each MVP value in the first candidate list corresponds to an index, and N is an integer greater than 1; The encoding module is used to encode the target coding mode and the index corresponding to the first target MVP value to obtain a code stream including first information, where the first information is used to represent the target coding mode and the index corresponding to the first target MVP value.

29. A three-dimensional grid inter-frame prediction decoding device, characterized in that: include: An acquisition module, configured to acquire a code stream sent by an encoding end, wherein the code stream includes a basic grid code stream; A first decoding module is configured to perform sub-grid type decoding on the basic grid code stream to obtain a P sub-grid code stream including first information, where the first information is used to represent a target coding mode and an index corresponding to a first target MVP value; a second determination module, configured to determine a target coding mode according to the first information, and determine a first target MVP value from a first candidate list according to an index corresponding to the first target MVP value, the first candidate list including N MVP values ​​and an index corresponding to each MVP value, the first target MVP value being one of the N MVP values, where N is an integer greater than 1; The second decoding module is used to decode the P sub-grid code stream according to the target coding mode and the first target MVP value to obtain the MV value of the grid vertex to be decoded in the P sub-grid.

30. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 27 are implemented.

31. A readable storage medium, characterized in that The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 27 are implemented.

Images