Encoding and decoding method, code stream, encoder, decoder and storage medium

Abstract

Provided are an encoding and decoding method, a code stream, an encoder, a decoder and a storage medium, applied to the decoder, the method comprising: determining vertex information of a three-dimensional grid (S701); performing connectivity prediction according to the vertex information of the three-dimensional grid, and determining a predicted weight value of an edge of the three-dimensional grid (S702); analyzing the code stream, and determining a predicted residual value of an edge of the three-dimensional grid (S703); and determining a reconstruction weight value of the edge of the three-dimensional grid according to the predicted weight value and the predicted residual value (S704). According to the encoding and decoding method, the geometric encoding efficiency of the three-dimensional grid can be improved, and the compression performance of the three-dimensional grid is improved.

Description

Coding and decoding method, code stream, encoder, decoder and storage medium Technical Field

[0001] The embodiments of the present application relate to the field of three-dimensional data processing technology, and in particular to a coding and decoding method, a bit stream, an encoder, a decoder, and a storage medium. Background Art

[0002] Advances in three-dimensional (3D) capture, modeling, and rendering have enabled the widespread application of 3D immersive content. A 3D mesh can be composed of multiple polygons. Geometrically, each polygon is defined by its vertices in 3D space and information about how the vertices are connected, known as connectivity.

[0003] In the prior art, geometric compression algorithms for 3D meshes primarily compress two components: the vertices of the 3D mesh and its connectivity. While compression schemes for these components have been proposed, these schemes still suffer from several drawbacks, resulting in poor vertex compression and impacting 3D mesh compression performance.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a coding and decoding method, a code stream, an encoder, a decoder, and a storage medium, which can improve the geometric coding efficiency of three-dimensional meshes and improve the compression performance of three-dimensional meshes.

[0006] The technical solution of the embodiment of the present application can be implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a decoding method, applied to a decoder, the method comprising:

[0008] Determine the vertex information of the three-dimensional mesh;

[0009] Performing connectivity prediction based on vertex information of the three-dimensional mesh and determining predicted weight values ​​of edges of the three-dimensional mesh;

[0010] Parse the code stream to determine the prediction residual value of the edge of the 3D grid;

[0011] The reconstruction weight value of the edge of the three-dimensional grid is determined according to the prediction weight value and the prediction residual value.

[0012] In a second aspect, an embodiment of the present application provides an encoding method, applied to an encoder, the method comprising:

[0013] Determine the vertex information of the three-dimensional mesh;

[0014] Performing connectivity prediction based on vertex information of the three-dimensional mesh and determining predicted weight values ​​of edges of the three-dimensional mesh;

[0015] The true weight value of the edge of the three-dimensional grid is determined, and the prediction residual value of the edge of the three-dimensional grid is determined according to the true weight value and the predicted weight value of the edge of the three-dimensional grid.

[0016] In a third aspect, an embodiment of the present application provides a code stream, which is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least: vertex information of a three-dimensional grid and predicted residual values ​​of edges of the three-dimensional grid.

[0017] In a fourth aspect, an embodiment of the present application provides an encoder, comprising a first determining unit and a first predicting unit; wherein,

[0018] a first determining unit configured to determine vertex information of a three-dimensional mesh;

[0019] a first prediction unit configured to perform connectivity prediction based on vertex information of the three-dimensional mesh and determine a prediction weight value of an edge of the three-dimensional mesh;

[0020] The first determining unit is further configured to determine the true weight value of the edge of the three-dimensional grid, and determine the prediction residual value of the edge of the three-dimensional grid according to the true weight value and the predicted weight value of the edge of the three-dimensional grid.

[0021] In a fifth aspect, an embodiment of the present application provides an encoder, comprising a first memory and a first processor; wherein,

[0022] a first memory for storing a computer program capable of running on the first processor;

[0023] The first processor is configured to execute the method according to the second aspect when running a computer program.

[0024] In a sixth aspect, an embodiment of the present application provides a decoder, comprising a second determining unit, a second predicting unit, and a decoding unit; wherein,

[0025] a second determining unit configured to determine vertex information of the three-dimensional mesh;

[0026] a second prediction unit configured to perform connectivity prediction based on vertex information of the three-dimensional mesh and determine a prediction weight value of an edge of the three-dimensional mesh;

[0027] A decoding unit configured to parse the bitstream and determine prediction residual values ​​of edges of the three-dimensional grid;

[0028] The second determining unit is further configured to determine the reconstruction weight value of the edge of the three-dimensional grid according to the prediction weight value and the prediction residual value.

[0029] In a seventh aspect, an embodiment of the present application provides a decoder, the decoder comprising a second memory and a second processor; wherein,

[0030] a second memory for storing a computer program capable of running on the second processor;

[0031] The second processor is configured to execute the method according to the first aspect when running a computer program.

[0032] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by at least one processor, it implements the method described in the first aspect or the method described in the second aspect.

[0033] The embodiments of the present application provide a coding and decoding method, a bitstream, an encoder, a decoder, and a storage medium. Whether it is an encoding end or a decoding end, vertex information of a three-dimensional mesh is determined; connectivity prediction is performed based on the vertex information of the three-dimensional mesh to determine the predicted weight values ​​of the edges of the three-dimensional mesh. Thus, after determining the predicted weight values ​​of the edges of the three-dimensional mesh, the encoding end determines the predicted residual values ​​of the edges of the three-dimensional mesh based on the actual weight values ​​and the predicted weight values ​​of the edges of the three-dimensional mesh, and transmits the predicted residual values ​​from the encoding end to the decoding end. The decoding end then parses the bitstream to determine the predicted residual values ​​of the edges of the three-dimensional mesh, and determines the reconstruction weight values ​​of the edges of the three-dimensional mesh based on the predicted weight values ​​and the predicted residual values. In this way, by extracting spatial features from the vertices of the three-dimensional mesh, the obtained spatial feature information can be used to guide the predictive coding of connectivity; and what is transmitted from the encoder to the decoder is only the vertex information of the three-dimensional mesh and the predicted residual values ​​of the edges and other code stream information, and there is no need to transmit the vertex mapping code stream reflecting the vertex order, which can reduce the bit rate; in addition, local processing of the three-dimensional mesh based on the k-hop subgraph can also solve the shortcomings of the unstructured and difficult to divide three-dimensional mesh data, so that when the connectivity is predicted based on the vertex information of the three-dimensional mesh, the geometric coding efficiency of the three-dimensional mesh can be significantly improved, and the compression performance of the three-dimensional mesh can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1A is a schematic diagram of vertices of a triangular mesh;

[0035] FIG1B is a schematic diagram of an edge of a triangular mesh;

[0036] FIG1C is a schematic diagram of a triangular mesh;

[0037] FIG2 is a schematic diagram of a three-dimensional structure formed by a triangular grid;

[0038] Figure 3 is a schematic structural diagram of a triangular fan;

[0039] FIG4 is a schematic diagram of the traversal process of an EdgeBreaker algorithm;

[0040] FIG5 is a flowchart of a three-dimensional grid coding algorithm based on V-PCC expansion;

[0041] FIG6 is a flowchart of another three-dimensional grid coding algorithm based on V-PCC expansion;

[0042] FIG7 is a schematic diagram of a flowchart of a decoding method provided in an embodiment of the present application;

[0043] FIG8 is a schematic diagram of the composition structure of a spatial feature extraction network model provided in an embodiment of the present application;

[0044] FIG9 is a schematic diagram of the structure of a downsampling module provided in an embodiment of the present application;

[0045] FIG10 is a schematic diagram of the structure of a downsampling module provided in an embodiment of the present application;

[0046] FIG11 is a schematic diagram of the composition structure of a residual network module provided in an embodiment of the present application;

[0047] FIG12 is a schematic diagram of a flow chart of another decoding method provided in an embodiment of the present application;

[0048] FIG13 is a schematic diagram of the structure of a preset network model provided in an embodiment of the present application;

[0049] FIG14 is a schematic diagram of the composition structure of a vertex feature update module provided in an embodiment of the present application;

[0050] FIG15 is a schematic diagram of a flow chart of an encoding method provided in an embodiment of the present application;

[0051] FIG16 is a flow chart of a coding and decoding method according to an embodiment of the present application;

[0052] FIG17 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0053] FIG18 is a schematic diagram of a specific hardware structure of an encoder provided in an embodiment of the present application;

[0054] FIG19 is a schematic diagram of the structure of a decoder provided in an embodiment of the present application;

[0055] FIG20 is a schematic diagram of a specific hardware structure of a decoder provided in an embodiment of the present application;

[0056] FIG21 is a schematic diagram of the composition structure of a coding and decoding system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0060] In the following description, reference is made to "some embodiments," which describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. It should also be noted that the terms "first, second, and third" in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that "first, second, and third" may be interchanged in a specific order or sequential order where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0061] Before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained first. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0062] Geometry-based Point Cloud Compression (G-PCC or GPCC)

[0063] Video-based Point Cloud Compression (V-PCC or VPCC)

[0064] K Nearest Neighbor (KNN)

[0065] Multiple Layer Perceptron (MLP)

[0066] Three Dimensions mesh (3D mesh)

[0067] Graph Convolutional Network (GCN)

[0068] Luminance component (Luminance, Luma or Y)

[0069] Chroma blue (Cb)

[0070] Chroma red (Cr)

[0071] Point cloud is a three-dimensional representation of the surface of an object. Point cloud (data) of the surface of an object can be collected through acquisition equipment such as photoelectric radar, lidar, laser scanner, and multi-view camera.

[0072] A point cloud refers to a collection of massive three-dimensional points. The points in the point cloud can include both their location information and their attribute information. For example, the location information of a point can be its three-dimensional coordinate information. The location information of a point can also be referred to as its geometric information. For example, the attribute information of a point can include color information and / or reflectivity, etc. For example, color information can be information in any color space. For example, color information can be RGB information, where R represents red (Red, R), G represents green (Green, G), and B represents blue (Blue, B). For another example, color information can be luminance and chrominance (YCbCr, YUV) information, where Y represents brightness, Cb (U) represents blue chrominance, and Cr (V) represents red chrominance.

[0073] For example, a point cloud obtained based on the principle of laser measurement can include the three-dimensional coordinate information of the point and the laser reflection intensity (reflectance) of the point. For another example, a point cloud obtained based on the principle of photogrammetry can include the three-dimensional coordinate information of the point and the color information of the point. For another example, a point cloud obtained by combining the principles of laser measurement and photogrammetry can include the three-dimensional coordinate information of the point, the laser reflection intensity (reflectance) of the point, and the color information of the point.

[0074] Point clouds can be divided into the following categories according to the acquisition method:

[0075] The first type of static point cloud: the object is stationary and the device used to obtain the point cloud is also stationary;

[0076] The second type of dynamic point cloud: the object is moving, but the device that obtains the point cloud is stationary;

[0077] The third type of dynamic point cloud acquisition: the device that acquires the point cloud is moving.

[0078] For example, point clouds can be divided into two categories according to their usage:

[0079] Category 1: Machine perception point cloud, which can be used in scenarios such as autonomous navigation systems, real-time inspection systems, geographic information systems, visual sorting robots, and disaster relief robots;

[0080] Category 2: Human eye perception point cloud, which can be used in point cloud application scenarios such as digital cultural heritage, free viewpoint broadcasting, 3D immersive communication, and 3D immersive interaction.

[0081] Since point clouds are a collection of massive points, storing point clouds not only consumes a lot of memory, but is also not conducive to transmission. There is also not enough bandwidth to support direct transmission of point clouds at the network layer without compression. Therefore, point clouds need to be compressed.

[0082] To date, the point cloud coding framework that can compress point clouds can be the G-PCC codec framework or the V-PCC codec framework provided by the Moving Picture Experts Group (MPEG), or the AVS-PCC codec framework provided by the Audio Video Standard (AVS). Among them, the G-PCC codec framework can be used to compress the first type of static point cloud and the third type of dynamically acquired point cloud, and the V-PCC codec framework can be used to compress the second type of dynamic point cloud. In the embodiments of the present application, the description is mainly based on the G-PCC codec framework.

[0083] In the embodiment of the present application, a three-dimensional point cloud is composed of a large number of points with information such as coordinates and colors, and is a three-dimensional data format. Since point clouds usually have a large number of points, a large amount of data and a large space occupied, in order to better store, transmit and subsequently process them, relevant organizations (for example, the International Organization for Standardization (ISO), the International Electrotechnical Commission (IEC), the Joint Technical Committee for Information Technology (JTC1) or the Seventh Generation Working Group (Work Group 7, WG7), etc.) are currently conducting research on point cloud compression. The G-PCC codec framework is a geometry-based point cloud compression platform proposed and continuously improved by these organizations.

[0084] Understandably, advances in three-dimensional (3D) capture, modeling, and rendering have led to the increasing use of 3D immersive content. At the same time, models are becoming increasingly complex, requiring a large amount of data to create and use. Three-dimensional meshes are widely used to represent such immersive content. A mesh consists of multiple polygons. Geometrically, each polygon is defined by its vertices in three-dimensional space and information about how the vertices are connected, i.e., connectivity information. In addition, polygons have attribute characteristics, such as color and normals, which can be associated directly with mesh vertices or with mesh surfaces using mapping information. Mapping information is typically described by a set of parametric coordinates that are used to associate vertices with two-dimensional attribute maps. Two-dimensional attribute maps are used to store high-resolution attribute information, such as textures, normals, displacements, etc., and can be used for various purposes such as texture mapping and shading.

[0085] Specifically, the elements of a triangular mesh can include vertices, edges, and faces. Figure 1A shows a schematic diagram of a triangular mesh's vertices, Figure 1B shows a schematic diagram of a triangular mesh's edges, and Figure 1C shows a schematic diagram of a triangular mesh's faces. For multiple triangular meshes, the resulting three-dimensional structure is shown in Figure 2.

[0086] In the embodiments of the present application, compressing the geometric information of a 3D mesh generally requires compressing two parts: one part is the vertices (i.e., the 3D coordinates of the vertices in Figure 1A), and the other part is the connectivity (which three vertices connect to form a triangle, i.e., the face in Figure 1C). For example, in a 3D model file (.obj), it is usually represented in the form of "indexv1 indexv2 indexv3", where v1, v2, and v3 represent the three vertices of the face.

[0087] Currently, traditional 3D mesh encoding and decoding algorithms include the Triangle Fan (TFAN) algorithm and the EdgeBreaker algorithm. Furthermore, as an extension of V-PCC, algorithms have been proposed that use V-PCC to encode 3D mesh vertices and TFAN or EdgeBreaker to encode connectivity.

[0088] Currently, there are three algorithms for compressing 3D meshes: TFAN, EdgeBreaker, and an algorithm based on the V-PCC extension. The first two are classic algorithms for 3D mesh compression. The following sections provide a detailed introduction to the relevant technologies of these three algorithms.

[0089] First, the TFAN algorithm is a connectivity-driven three-dimensional mesh coding algorithm, that is, connectivity is encoded first and then geometric information is encoded.

[0090] To address connectivity, the TFAN algorithm decomposes a 3D mesh into numerous triangle fans (as shown in Figure 3), then traverses each triangle fan for encoding. In a 3D model file (.obj), the connectivity of the 3D mesh is represented by point indices. Since each face is represented by the indices of three vertices, and a vertex is shared by an average of six faces, the index value of each vertex is recorded six times on average. TFAN, however, uses a triangle fan-based traversal to reduce the number of point accesses without changing connectivity, thereby achieving compression. See Figure 3 for a schematic diagram of the triangle fan structure. As shown in Figure 3, V1, V2, V3, V4, V5, V6, and V7 correspond to seven vertices. For each vertex, TFAN uses parallelogram-based predictive coding. Specifically, given the known connectivity and the 3D coordinates of some encoded points, the topological relationship between vertices is used to predict the 3D coordinates of the encoded point. The residual between the predicted value and the true value is then encoded.

[0091] Secondly, the EdgeBreaker algorithm is similar to TFAN and is also a connectivity-driven three-dimensional mesh coding algorithm.

[0092] Regarding connectivity, unlike TFAN, EdgeBreaker uses a method similar to region growing. It first randomly selects a triangle as a seed, then uses this triangle as a starting point to spirally traverse each triangle in the 3D mesh, encoding each visited triangle. See Figure 4 for a schematic diagram of the traversal process of the EdgeBreaker algorithm. As shown in Figure 4, a spiral traversal method is used here, and the specific traversal process is based on the principle of region growing: during the traversal of the mesh, a directed boundary composed of edges must be maintained, dividing the mesh into traversed and untraversed parts. For each vertex, similar to TFAN, a parallelogram-based predictive coding is used. That is, given the known connectivity and the 3D coordinates of some encoded points, the topological relationships between vertices are used to predict the 3D coordinates of the encoded point. The residual between the predicted value and the true value is then encoded.

[0093] Again, there are two schemes based on the algorithm extended by V-PCC. Both schemes use V-PCC to compress the vertices of the 3D mesh and use TFAN or EdgeBreaker to compress the connectivity of the 3D mesh. The specific details are different.

[0094] Exemplarily, FIG5 shows a flowchart of a three-dimensional grid coding algorithm based on V-PCC extension. As shown in Figure 5, first, the input mesh frames enter the patch generation and packing module and output patch information. Then, for the input mesh frames and patch information, after being processed by the patch sequence compression module, a patch substream can be generated. After being processed by the occupancy map compression module, an occupancy map substream can be generated. After being processed by the geometry compression module, a geometry substream can be generated. The geometry frame is further reconstructed and passed through the geometry smoothing module to obtain the geometry smoothing parameters. The reconstructed geometry frame and the reconstructed occupancy map frame are combined and processed by the attribute generation module and the attribute compression module to generate an attribute substream. The attribute smoothing parameters can be obtained by the attribute smoothing module. parameters); in addition, for the input mesh frame and the reconstructed geometry frame, the connectivity substream and the vertex map substream can be obtained by updating the vertex index module (Update vertex Indices), the connectivity encoder (Connectivity Encoder) and the vertex map encoder (Vertex Map Encoder); finally, this information is written into the V-PCC bitstream through the multiplexer module (Multiplexer). In other words, the bold box part in Figure 5 is the encoding process of connectivity, and the rest are the encoding process of vertices. The encoding of the vertex part is the same as V-PCC, and the code stream of the connectivity part is divided into two parts, the connectivity code stream and the vertex map code stream. The connectivity code stream is the code stream obtained after encoding the connectivity using TFAN or EdgeBreaker.As for the vertex mapping code stream, because the order of vertices will be changed before and after V-PCC encoding and decoding, and the vertex index value is needed when encoding connectivity, that is, the order of a vertex among all vertices, it is necessary to map the order of vertices before and after encoding and decoding. Therefore, it is necessary to transmit the vertex mapping code stream to indicate the mapping relationship between the original vertex order and the vertex order after decoding.

[0095] For example, Figure 6 shows a flowchart of another three-dimensional mesh coding algorithm based on the V-PCC extension. As shown in Figure 6, the V-PCC encoding method is modified here. Reordering information and connectivity substream are directly obtained through the connectivity encoder. The raw patch generation and packing module then processes the input mesh frame and reordering information to obtain patch information. The geometry substream and attribute substream are then obtained through the geometry compression module, attribute generation module, and attribute compression module. Finally, this information is written into the V-PCC bitstream through the multiplexer module. In other words, Figure 5 requires the transmission of the vertex map stream, but this part of the stream accounts for a large proportion. To avoid transmitting this part of the stream, Figure 6 modifies V-PCC. Modifications are made to the patch generation part so that vertices are projected in the same order as during connectivity coding, thus avoiding the encoding of the vertex map stream.

[0096] Simply put, for the two solutions in Figures 5 and 6 in the related art, the solution shown in Figure 5 requires the transmission of vertex mapping code streams, which increases the total bit rate and deteriorates the effect; while the solution shown in Figure 6 changes the original patch generation method of V-PCC, resulting in worse vertex compression effects. It can be seen that there are some defects in the related technologies that affect the compression performance of three-dimensional meshes.

[0097] Based on this, an embodiment of the present application provides a coding and decoding method, which determines the vertex information of a three-dimensional mesh at both the encoding and decoding ends; performs connectivity prediction based on the vertex information of the three-dimensional mesh, and determines the predicted weight values ​​of the edges of the three-dimensional mesh. In this way, after determining the predicted weight values ​​of the edges of the three-dimensional mesh, the encoding end determines the predicted residual values ​​of the edges of the three-dimensional mesh based on the actual weight values ​​and predicted weight values ​​of the edges of the three-dimensional mesh, and transmits the predicted residual values ​​from the encoding end to the decoding end. The decoding end then parses the bitstream to determine the predicted residual values ​​of the edges of the three-dimensional mesh, and determines the reconstruction weight values ​​of the edges of the three-dimensional mesh based on the predicted weight values ​​and the predicted residual values.

[0098] In this way, by extracting spatial features from the vertices of the three-dimensional mesh, the obtained spatial feature information can be used to guide the predictive coding of connectivity; and by reordering the vertices in a preset manner to ensure that the order at the encoding end and the decoding end is the same, there is no need to transmit the vertex mapping code stream that reflects the vertex order, which can reduce the bit rate; in addition, local processing of the three-dimensional mesh based on the k-hop subgraph can also solve the shortcomings of the unstructured and difficult to divide three-dimensional mesh data, so that when predicting connectivity based on the vertex information of the three-dimensional mesh, the geometric coding efficiency of the three-dimensional mesh can be significantly improved, and the compression performance of the three-dimensional mesh can be improved.

[0099] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0100] In one embodiment of the present application, referring to FIG7 , a schematic flow chart of a decoding method provided by an embodiment of the present application is shown. As shown in FIG7 , the method may include:

[0101] S701: Determine vertex information of a three-dimensional mesh.

[0102] It should be noted that the decoding method in the embodiments of this application specifically refers to a 3D mesh decoding method. More specifically, it is a 3D mesh geometric compression algorithm that primarily predicts the connectivity of the 3D mesh. This method can be applied to a 3D mesh decoder (referred to as the "decoder" in the embodiments of this application).

[0103] It should also be noted that in the embodiments of this application, the elements of a 3D mesh may include vertices, edges, and faces. Compression of geometric information of a 3D mesh typically requires compression of two components: vertices and connectivity. In some embodiments, the vertex information of the 3D mesh can be determined by parsing the bitstream at the decoder end.

[0104] In a specific embodiment, parsing the code stream to determine the vertex information of the three-dimensional mesh may include: parsing the vertex compression code stream to determine the vertex information of the three-dimensional mesh.

[0105] In an embodiment of the present application, the compression of vertex information of a three-dimensional mesh can be achieved by using a point cloud lossless compression algorithm. For example, it can be achieved by V-PCC, G-PCC or other point cloud geometry lossless compression algorithms, and no limitation is made here.

[0106] In addition, in the embodiment of the present application, the vertex information of the three-dimensional mesh can be the three-dimensional coordinate information of the vertex, or it can be the three-dimensional coordinate spliced ​​with the three-dimensional normal vector information, and no limitation is made here.

[0107] S702: Perform connectivity prediction based on vertex information of the three-dimensional mesh, and determine prediction weight values ​​of edges of the three-dimensional mesh.

[0108] It should be noted that in the embodiments of the present application, when predicting connectivity based on vertex information of a three-dimensional mesh, in order to address the shortcomings of unstructured and difficult to segment three-dimensional mesh data, it is necessary to perform local processing on the three-dimensional mesh based on a k-hop subgraph to determine the predicted weight value of at least one edge in the k-hop subgraph. In some embodiments, for S702, performing connectivity prediction based on vertex information of the three-dimensional mesh and determining the predicted weight value of the edge of the three-dimensional mesh may include:

[0109] Constructing a local subgraph based on vertex information of the three-dimensional grid and determining a k-hop subgraph of the current vertex; wherein the k-hop subgraph includes spatial feature information of multiple target vertices and an initial weight value of at least one edge;

[0110] The k-hop subgraph is input into a preset network model, and a predicted weight value of at least one edge in the k-hop subgraph is determined based on the preset network model; wherein k is a positive integer.

[0111] Furthermore, regarding the determination of spatial feature information, in some embodiments, the method may further include:

[0112] Performing spatial feature extraction based on vertex information of the three-dimensional mesh to determine spatial feature information of the vertices of the three-dimensional mesh;

[0113] The spatial feature information of a plurality of target vertices is obtained from the spatial feature information of the vertices of the three-dimensional mesh.

[0114] It should also be noted that in the embodiments of the present application, since the connectivity of a three-dimensional mesh is related to its vertex information, it is necessary to extract the spatial feature information of the vertices before predicting the connectivity. In some embodiments, performing spatial feature extraction based on the vertex information of the three-dimensional mesh to determine the spatial feature information of the vertices of the three-dimensional mesh may include: inputting the vertex information of the three-dimensional mesh into a spatial feature extraction network model, and determining the spatial feature information of the vertices of the three-dimensional mesh based on the spatial feature extraction network model.

[0115] In an embodiment of the present application, spatial feature extraction can be implemented by a spatial feature extraction network model. The spatial feature extraction network model can include at least one of the following: a point-net-based spatial feature extraction network model and a sparse convolution-based spatial feature extraction network model.

[0116] Here, for the spatial feature extraction network model, in addition to the spatial feature extraction network model based on point-net and the spatial feature extraction network model based on sparse convolution, it can also be based on other deep learning network models, such as the spatial feature extraction network model based on point net++, and there is no limitation on this.

[0117] In some embodiments, the network structure of the spatial feature extraction network model may include: at least one downsampling module, at least one upsampling module, and a cropping module; wherein:

[0118] The downsampling module is composed of a convolutional layer, an activation function layer, and a residual network module;

[0119] The upsampling module consists of a deconvolution layer, an activation function layer, and a residual network module.

[0120] In an embodiment of the present application, at least one downsampling module is connected in series, and at least one upsampling module is connected in series, wherein the input end of the first downsampling module is used to receive vertex information of the three-dimensional mesh, the output end of the last downsampling module is connected to the input end of the first upsampling module, the output end of the last upsampling module is connected to the first input end of the cropping module, the second input end of the cropping module is connected to the input end of the first downsampling module, and the output end of the cropping module is used to output spatial feature information of the vertices of the three-dimensional mesh.

[0121] For example, FIG8 shows a schematic diagram of the composition structure of a spatial feature extraction network model provided by an embodiment of the present application. As shown in FIG8 , the spatial feature extraction network model includes three downsampling modules, three upsampling modules, and a cropping module. The vertex information of the input three-dimensional mesh can be three-dimensional coordinate information (N×3), and the spatial feature information of the output three-dimensional mesh vertex can be N×C; where N is the number of vertices, 3 is the dimension of the vertex coordinates, and C is the feature dimension.

[0122] For the downsampling module, Figure 9 shows a schematic diagram of the composition structure of a downsampling module provided in an embodiment of the present application. As shown in Figure 9, the input of the downsampling module is N channels (N channels) and the output is O channels (O channels). The downsampling module may include a first convolutional layer, a second convolutional layer, two activation layers and three residual network (Inception-Residual Net, IRN) modules. Among them, an activation layer is connected after the first convolutional layer, an activation layer is connected after the second convolutional layer, and then three IRN modules are connected in sequence. In Figure 9, the first convolutional layer can be Conv(H,3,1), the second convolutional layer can be Conv(O,2,2), and the activation layer can be a rectified linear unit (Rectified Linear Unit, ReLU) activation function, referred to as ReLU function. The specific connection relationship of the downsampling module is detailed in Figure 9.

[0123] For the upsampling module, Figure 10 shows a schematic diagram of the composition structure of an upsampling module provided by an embodiment of the present application. As shown in Figure 10, the input of the upsampling module is N channels (N channels) and the output is O channels (O channels). The upsampling module may include a first deconvolution layer, a third convolution layer, two activation layers and three IRN modules. Among them, an activation layer is connected after the first deconvolution layer, and an activation layer is also connected after the third convolution layer, and then the three IRN modules are connected in sequence. Here, the first deconvolution layer can be Deconv(H,3,1), the third convolution layer can be the same as the second convolution layer, that is, Conv(O,2,2), and the activation layer can still be a ReLU function. The specific connection relationship of the upsampling module is detailed in Figure 10.

[0124] For the IRN module in the upsampling module and the downsampling module, Figure 11 shows a schematic diagram of the composition structure of an IRN module provided in an embodiment of the present application. As shown in Figure 11, the input of the IRN module is N channels (N channels) and the output is also N channels (N channels). The IRN module may include two fourth convolutional layers, two fifth convolutional layers, two sixth convolutional layers, four activation layers, a concatenation layer (Concatenate) and an adder. Among them, the fourth convolutional layer can be Conv(N / 4,1,1), the fifth convolutional layer can be Conv(N / 4,3,1), the sixth convolutional layer can be Conv(N / 2,3,1), an activation layer is connected after the fourth convolutional layer, and an activation layer is also connected after the fifth convolutional layer, and the activation layer can still be a ReLU function. In addition, the outputs of the two sixth convolutional layers are spliced ​​through the concatenation layer, and the output of the concatenation layer is added to the input of the IRN module through the adder. The output of the adder is the output of the IRN module. The specific connection relationship of the IRN module is detailed in Figure 11.

[0125] In this way, according to the spatial feature extraction network model shown in FIG8 , spatial feature extraction can be performed on the vertex information of the three-dimensional mesh to obtain the spatial feature information of the vertices of the three-dimensional mesh, so as to guide the subsequent prediction and decoding of connectivity.

[0126] It is understood that in the embodiment of the present application, an edge is an edge formed between every two target vertices in a plurality of target vertices. For edge connectivity prediction, in some embodiments, as shown in FIG12 , after S701 , the following steps S1201 to S1204 may be specifically included:

[0127] S1201: Perform spatial feature extraction based on vertex information of the three-dimensional mesh to determine the spatial feature information of the vertices of the three-dimensional mesh.

[0128] S1202: Perform a K-nearest neighbor search based on vertex information of the three-dimensional mesh, and establish a K-nearest neighbor graph of the vertices of the three-dimensional mesh.

[0129] It should be noted that, based on the vertex information of the 3D mesh, spatial features are extracted on the one hand, and a K-nearest neighbor graph is constructed on the other. In a K-nearest neighbor graph, a vertex is connected to its K nearest neighbor points. In other words, a K-nearest-neighbor graph (KNN graph) can be constructed based on the vertex information of the 3D mesh (e.g., 3D coordinate information). Specifically, for each vertex, this vertex can be connected to its K nearest neighbor points.

[0130] S1203: Building a k-hop subgraph of the current vertex according to the K-nearest neighbor graph of the vertices of the three-dimensional grid.

[0131] Here, k and K are both positive integers, and k is not equal to K. The value range of K is K∈(0,N), and N is the number of vertices in the three-dimensional grid. For k, when k is equal to 1, the 1-hop subgraph is composed of the current vertex and its K neighboring points; when k is equal to 2, the 2-hop subgraph is composed of the current vertex, the K neighboring points adjacent to the current vertex, and the K neighboring points adjacent to each neighboring point, and so on to determine other k-hop subgraphs. In the embodiment of the present application, the value of k can be 2, but it is not limited to any value.

[0132] It should also be noted that the embodiment of the present application also needs to determine the order of the vertices of the three-dimensional mesh so that the encoding end and the decoding end are consistent. In some embodiments, the method may also include:

[0133] Determining a starting vertex from the vertices of the three-dimensional mesh, reordering the vertices of the three-dimensional mesh according to a preset traversal order, and determining a preset vertex order;

[0134] According to a preset vertex order, the vertices of the three-dimensional grid are sequentially used as current vertices to establish a k-hop subgraph of the current vertex.

[0135] Here, the preset traversal order can be a breadth-first search (BFS) order. Thus, after determining the starting vertex, the vertices of the three-dimensional mesh can be reordered according to the BFS order to determine the preset vertex order, which is then used as a basis. In the K-nearest neighbor graph, a k-hop subgraph is established for each vertex. Then, according to the preset vertex order after the breadth-first traversal, the connectivity of each k-hop subgraph is predicted using a local connectivity prediction network model.

[0136] Furthermore, for the starting vertex of the sort, one of the vertices can be randomly selected as the starting vertex, or the vertex at a specific position (such as the lower left corner) can be used as the starting vertex. It only needs to be consistent between the encoding and decoding ends. The embodiments of this application do not impose any restrictions.

[0137] In a possible embodiment, determining the starting vertex from the vertices of the three-dimensional mesh may include: selecting a vertex at a preset position from the vertices of the three-dimensional mesh; and determining the vertex at the preset position as the starting vertex.

[0138] In another possible embodiment, determining the starting vertex from the vertices of the three-dimensional mesh may include: parsing a code stream to determine the starting vertex.

[0139] In another possible embodiment, determining a starting vertex from the vertices of a three-dimensional mesh may include: parsing a code stream to determine a starting vertex index value; and determining a starting vertex from the vertices of the three-dimensional mesh based on the starting vertex index value; wherein there is a corresponding relationship between the starting vertex index value and the starting vertex.

[0140] It should be noted that in the embodiments of the present application, the starting vertex for sorting can be any vertex. For example, the starting vertex can be a vertex at a preset position, or can be a starting vertex determined by parsing the code stream, or can be a starting vertex index value determined by parsing the code stream and then determined based on the starting vertex index value. No limitation is imposed on this.

[0141] In a specific embodiment, for a k-hop subgraph, establishing the k-hop subgraph of the current vertex may include:

[0142] If the weight value of the first edge in the k-hop subgraph has been decoded and predicted, the initial weight value of the first edge in the k-hop subgraph is set to the reconstructed weight value of the first edge;

[0143] If the weight value of the first edge in the k-hop subgraph has not been decoded and predicted, the initial weight value of the first edge in the k-hop subgraph is set to a preset value; wherein the first edge represents an edge formed between any two vertices among multiple target vertices of the k-hop subgraph.

[0144] It should be noted that if the weight value of the first edge in the k-hop subgraph has been decoded and predicted, the initial weight value of the first edge can be set to the decoded reconstructed weight value of the first edge. Otherwise, if the weight value of the first edge in the k-hop subgraph has not been decoded and predicted, the initial weight value of the first edge can be set to a preset value. Here, the preset value can be 0.5, but this is not limited to any value.

[0145] S1204: Perform connectivity prediction based on the k-hop subgraph and the preset network model to determine a predicted weight value of at least one edge in the k-hop subgraph.

[0146] In an embodiment of the present application, spatial feature information of multiple target vertices can be obtained from the spatial feature information of the vertices of a three-dimensional mesh. The spatial feature information of the multiple target vertices and the initial weight values ​​of the edges are input into a preset network model. Based on the preset network model, a predicted weight value of at least one edge in a k-hop subgraph is determined. In this way, for each k-hop subgraph, the connectivity of the k-hop subgraph is predicted using the preset network model, and a predicted weight value for each edge in the three-dimensional mesh can be obtained.

[0147] In an embodiment of the present application, the preset network model may also be referred to as a local connectivity prediction network model. The preset network model here may at least include a graph neural network model based on graph convolution, that is, a graph neural network model based on GCN, but there is no limitation on this.

[0148] In some embodiments, the network structure of the preset network model may include: at least one vertex feature updating module and at least one edge feature updating module; wherein:

[0149] The vertex feature update module is composed of a first multi-layer perceptron module, a second multi-layer perceptron module and a graph convolutional network module;

[0150] The edge feature update module is composed of a third multi-layer perceptron module.

[0151] Here, the first multi-layer perceptron module is MLP (E in ,E out ) indicates that the second multi-layer perceptron module is represented by MLP(P in ,P out ) indicates that, where E in Represents the edge feature information of MLP input, E out represents the edge feature information output by MLP; P in Represents the vertex feature information of MLP input, P out Represents the vertex feature information output by MLP. After processing by MLP, E out The edge feature dimension and P out The vertex feature dimensions are the same, and then the vertex features can be updated through the graph convolutional network module (GCN Layer).

[0152] In some embodiments, for the third multilayer perceptron module, the method may further include: inputting the edge to be updated and two vertices connected by the edge to be updated into the third multilayer perceptron module, and determining the output of the third multilayer perceptron module as the updated edge.

[0153] Here, the third multi-layer perceptron module uses f mlp (p1,p2,edge in ) indicates that, where edge in is the edge to be updated, p1 and p2 are edge in The two connected vertices, the output of the third multi-layer perceptron module is the updated edge update , f mlp It represents an MLP network. It should be noted that in the last edge feature update module, f mlp Set the output dimension to 1.

[0154] For example, Figure 13 shows a schematic diagram of the structure of a preset network model provided in an embodiment of the present application. As shown in Figure 13, the input of the preset network model is a k-hop subgraph initialized with connectivity weights, and the output is a k-hop subgraph after local connectivity prediction. The preset network model can include three vertex feature update modules and three edge feature update modules, each of which is connected to an edge feature update module. The specific connection relationship is detailed in Figure 13.

[0155] As for the vertex feature update module, FIG14 shows a schematic diagram of the composition structure of a vertex feature update module provided by an embodiment of the present application. As shown in FIG14, the vertex feature update module may include a first multi-layer perceptron module MLP (E in ,E out ), the second multi-layer perceptron module MLP (P in ,P out ) and graph convolutional network module GCN Layer. Among them, the first multi-layer perceptron module MLP (E in ,E out ) and the second multi-layer perceptron module MLP(P in ,P out ) is the input of the vertex feature update module, the first multi-layer perceptron module MLP(E in ,E out ) and the second multi-layer perceptron module MLP(P in ,P out ) is connected to the input of the graph convolutional network module GCN Layer. The output of the graph convolutional network module GCN Layer is the output of the vertex feature update module. The specific connection relationship is shown in Figure 14.

[0156] For the edge feature update module, it can be expressed by the following formula:

[0157] edge update =f mlp (p1,p2,edge in )

[0158] Among them, edge in is the edge to be updated, p1 and p2 are edge in The two connected vertices, edge update is the updated edge, f mlp Is an MLP network. In the last edge feature update module, f mlp Set the output dimension to 1.

[0159] S703: Parse the code stream to determine the prediction residual value of the edge of the three-dimensional grid.

[0160] S704: Determine the reconstruction weight value of the edge of the three-dimensional mesh according to the prediction weight value and the prediction residual value.

[0161] It should be noted that after connectivity prediction is performed based on the k-hop subgraph and the preset network model to determine the predicted weight values ​​of the edges of the three-dimensional mesh, the predicted residual values ​​of the edges of the three-dimensional mesh can be determined by parsing the bitstream; then, the reconstruction weight values ​​of the edges of the three-dimensional mesh are determined based on the predicted weight values ​​and the predicted residual values. In addition, it should be noted that there is no order in which the predicted weight values ​​and the predicted residual values ​​are determined; S701-S702 can be executed first to determine the predicted weight values, and then S703 can be executed to determine the predicted residual values; alternatively, S703 can be executed first to determine the predicted residual values, and then S701-S702 can be executed to determine the predicted weight values.

[0162] In a specific embodiment, determining the reconstruction weight value of the edge of the three-dimensional grid based on the predicted weight value and the predicted residual value may include: adding the predicted weight value and the predicted residual value to determine the reconstruction weight value of the edge of the three-dimensional grid.

[0163] That is, after the encoder determines the prediction residual values ​​for the edges of the 3D mesh, it encodes these values ​​and writes them into the bitstream, specifically into the connectivity residual compressed bitstream. Thus, the decoder parses the bitstream to determine the prediction residual values ​​for the edges of the 3D mesh. This can be done by parsing the connectivity residual compressed bitstream to determine the prediction residual values ​​for the edges in the k-hop subgraph. In this way, the decoder can determine the reconstruction weight values ​​for the edges in the k-hop subgraph by adding the prediction weight values ​​and the prediction residual values ​​for the edges in the k-hop subgraph.

[0164] Furthermore, after determining the reconstruction weight values ​​of the edges of the three-dimensional mesh, the method may further include:

[0165] If the reconstruction weight value of the edge of the three-dimensional mesh satisfies a first preset range, determining that two target vertices corresponding to the edge of the three-dimensional mesh are connected;

[0166] If the reconstruction weight value of the edge of the three-dimensional mesh satisfies the second preset range, it is determined that the two target vertices corresponding to the edge of the three-dimensional mesh are not connected.

[0167] It should be noted that, in a three-dimensional mesh, taking the first edge as an example, if the reconstruction weight value of the first edge is 1, then it can be determined that the two target vertices corresponding to the first edge are connected; if the reconstruction weight value of the first edge is 0, then it can be determined that the two target vertices corresponding to the first edge are not connected.

[0168] It should also be noted that, considering that the reconstruction weight value is a value between 0 and 1, the closer the reconstruction weight value is to 1, the greater the possibility that the two target vertices corresponding to the first edge are connected; if the reconstruction weight value is closer to 0, the greater the possibility that the two target vertices corresponding to the first edge are not connected. For example, assuming that the first preset range can be between 0.5 and 1, and the second preset range can be between 0 and 0.5; at this time, when the reconstruction weight value of the first edge is between 0.5 and 1, it can be determined that the two target vertices corresponding to the first edge are connected; when the reconstruction weight value of the first edge is between 0 and 0.5, it can be determined that the two target vertices corresponding to the first edge are not connected.

[0169] In short, the embodiment of the present application proposes a geometric compression algorithm for three-dimensional meshes, which is mainly aimed at the connectivity of three-dimensional meshes. Specifically, in the embodiment of the present application, the embodiment of the present application uses G-PCC, V-PCC or other point cloud geometric lossless coding algorithms to encode the vertices of the three-dimensional mesh. Unlike the algorithm in the related art that expands V-PCC to compress the three-dimensional mesh, the algorithm of the embodiment of the present application uses a deep learning-based solution to predict the connectivity between vertices, and then encodes the residual prediction residual value between the predicted value and the true value, so that the decoding end can obtain the prediction residual value by parsing the code stream, and then determine the reconstruction weight value to determine the connectivity between the predicted vertices.

[0170] This embodiment provides a decoding method that determines vertex information of a three-dimensional mesh; performs connectivity prediction based on the vertex information of the three-dimensional mesh to determine predicted weight values ​​for the edges of the three-dimensional mesh; parses the code stream to determine predicted residual values ​​for the edges of the three-dimensional mesh; and determines reconstruction weight values ​​for the edges of the three-dimensional mesh based on the predicted weight values ​​and the predicted residual values. Thus, by extracting spatial features from the vertices of the three-dimensional mesh, the resulting spatial feature information can be used to guide predictive coding of connectivity. Furthermore, by locally processing the three-dimensional mesh based on a k-hop subgraph, the shortcomings of the unstructured and difficult-to-partition three-dimensional mesh data can be addressed. Furthermore, the vertices are reordered in a preset manner to ensure the same order at the encoding and decoding ends, eliminating the need to transmit the vertex mapping code stream. This significantly improves the geometric coding efficiency of the three-dimensional mesh and improves the compression performance of the three-dimensional mesh.

[0171] In another embodiment of the present application, referring to FIG15 , a schematic flow chart of an encoding method provided by an embodiment of the present application is shown. As shown in FIG15 , the method may include:

[0172] S1501: Determine vertex information of a three-dimensional mesh.

[0173] It should be noted that the encoding method in the embodiments of this application specifically refers to a method for encoding a three-dimensional mesh. More specifically, it is a geometric compression algorithm for a three-dimensional mesh that primarily predicts the connectivity of the three-dimensional mesh. This method can be applied to a three-dimensional mesh encoder (referred to as simply "encoder" in the embodiments of this application).

[0174] It should also be noted that in the embodiments of the present application, the elements of a 3D mesh may include vertices, edges, and faces. Compression of geometric information of a 3D mesh typically requires compression of two components: vertices and connectivity. In some embodiments, the method may further include encoding the vertex information of the 3D mesh and writing the resulting encoded bits into a bitstream.

[0175] In this way, the encoder can encode the vertex information of the 3D mesh and write it into a bitstream, specifically into a vertex compressed bitstream. The decoder can then obtain the vertex information of the 3D mesh by parsing the bitstream.

[0176] In an embodiment of the present application, the compression of vertex information of a three-dimensional mesh can be achieved by using a point cloud lossless compression algorithm. For example, it can be achieved by V-PCC, G-PCC or other point cloud geometry lossless compression algorithms, and no limitation is made here.

[0177] In addition, in the embodiment of the present application, the vertex information of the three-dimensional mesh can be the three-dimensional coordinate information of the vertex, or it can be the three-dimensional coordinate spliced ​​with the three-dimensional normal vector information, and no limitation is made here.

[0178] S1502: Perform connectivity prediction based on vertex information of the three-dimensional mesh and determine prediction weight values ​​of edges of the three-dimensional mesh.

[0179] It should be noted that in the embodiments of the present application, when predicting connectivity based on vertex information of a three-dimensional mesh, in order to address the shortcomings of unstructured and difficult to segment three-dimensional mesh data, it is necessary to perform local processing on the three-dimensional mesh based on a k-hop subgraph to determine the predicted weight value of at least one edge in the k-hop subgraph. In some embodiments, for S1502, performing connectivity prediction based on vertex information of the three-dimensional mesh and determining the predicted weight value of the edge of the three-dimensional mesh may include:

[0180] Constructing a local subgraph based on vertex information of the three-dimensional grid and determining a k-hop subgraph of the current vertex; wherein the k-hop subgraph includes spatial feature information of multiple target vertices and an initial weight value of at least one edge;

[0181] The k-hop subgraph is input into a preset network model, and a predicted weight value of at least one edge in the k-hop subgraph is determined based on the preset network model; wherein k is a positive integer.

[0182] Furthermore, regarding the determination of spatial feature information, in some embodiments, the method may further include:

[0183] Performing spatial feature extraction based on vertex information of the three-dimensional mesh to determine spatial feature information of the vertices of the three-dimensional mesh;

[0184] The spatial feature information of a plurality of target vertices is obtained from the spatial feature information of the vertices of the three-dimensional mesh.

[0185] It should be noted that in the embodiments of the present application, since the connectivity of a three-dimensional mesh is related to its vertex information, it is necessary to extract the spatial feature information of the vertices before predicting the connectivity. In some embodiments, performing spatial feature extraction based on the vertex information of the three-dimensional mesh to determine the spatial feature information of the vertices of the three-dimensional mesh may include: inputting the vertex information of the three-dimensional mesh into a spatial feature extraction network model, and determining the spatial feature information of the vertices of the three-dimensional mesh based on the spatial feature extraction network model.

[0186] In an embodiment of the present application, spatial feature extraction can be implemented by a spatial feature extraction network model. The spatial feature extraction network model can include at least one of the following: a point-net-based spatial feature extraction network model and a sparse convolution-based spatial feature extraction network model.

[0187] Here, for the spatial feature extraction network model, in addition to the spatial feature extraction network model based on point-net and the spatial feature extraction network model based on sparse convolution, it can also be based on other deep learning network models, such as the spatial feature extraction network model based on point net++, and there is no limitation on this.

[0188] In some embodiments, the network structure of the spatial feature extraction network model may include: at least one downsampling module, at least one upsampling module, and a cropping module; wherein:

[0189] The downsampling module is composed of a convolutional layer, an activation function layer, and a residual network module;

[0190] The upsampling module consists of a deconvolution layer, an activation function layer, and a residual network module.

[0191] In an embodiment of the present application, at least one downsampling module is connected in series, and at least one upsampling module is connected in series, wherein the input end of the first downsampling module is used to receive vertex information of the three-dimensional mesh, the output end of the last downsampling module is connected to the input end of the first upsampling module, the output end of the last upsampling module is connected to the first input end of the cropping module, the second input end of the cropping module is connected to the input end of the first downsampling module, and the output end of the cropping module is used to output spatial feature information of the vertices of the three-dimensional mesh.

[0192] For example, in the aforementioned embodiment, FIG8 shows a schematic diagram of the composition structure of a spatial feature extraction network model provided by an embodiment of the present application. For the downsampling module and the upsampling module in FIG8 , see the composition structure example of the downsampling module shown in FIG9 and the composition structure example of the upsampling module shown in FIG10 , while FIG11 shows the composition structure example of the IRN module. In this way, according to the spatial feature extraction network model shown in FIG8 , spatial feature extraction of the vertex information of the three-dimensional mesh can be achieved to obtain the spatial feature information of the vertices of the three-dimensional mesh, so as to guide the subsequent predictive coding of connectivity.

[0193] It is understood that in the embodiments of the present application, an edge is an edge formed between every two target vertices among a plurality of target vertices. Regarding edge connectivity prediction, in some embodiments, local subgraph construction is performed based on vertex information of a three-dimensional mesh to determine a k-hop subgraph for the current vertex, which may include:

[0194] Perform K-nearest neighbor search based on vertex information of the three-dimensional grid and establish a K-nearest neighbor graph of the vertices of the three-dimensional grid;

[0195] According to the K nearest neighbor graph of the vertex of the three-dimensional grid, a k-hop subgraph of the current vertex is established.

[0196] It should be noted that in the embodiments of the present application, connectivity prediction is performed based on a k-hop subgraph and a preset network model to determine a predicted weight value for at least one edge in the k-hop subgraph. Thus, for each k-hop subgraph, the connectivity of the k-hop subgraph is predicted using the preset network model, resulting in a predicted weight value for each edge in the three-dimensional grid.

[0197] It's also important to note that in a K-nearest-neighbor graph, a vertex is connected to its K nearest neighboring points. In other words, a K-nearest-neighbor graph (KNN graph) can be constructed based on vertex information (e.g., 3D coordinates) of a 3D mesh. Specifically, each vertex can be connected to its K nearest neighboring points.

[0198] It should also be noted that k and K are both positive integers, and k is not equal to K. K's value range is K∈(0,N), where N is the number of vertices in the three-dimensional mesh. For k, when k is 1, the 1-hop subgraph consists of the current vertex and its K adjacent neighbors. When k is 2, the 2-hop subgraph consists of the current vertex, its K adjacent neighbors, and each of its K adjacent neighbors. This analogy is repeated to determine other k-hop subgraphs. Here, k can be 2, but this is not a limitation.

[0199] It should also be noted that the embodiment of the present application also needs to determine the order of the vertices of the three-dimensional mesh so that the encoding end and the decoding end are consistent. In some embodiments, the method may also include:

[0200] Determining a starting vertex from the vertices of the three-dimensional mesh, reordering the vertices of the three-dimensional mesh according to a preset traversal order, and determining a preset vertex order;

[0201] According to a preset vertex order, the vertices of the three-dimensional grid are sequentially used as current vertices to establish a k-hop subgraph of the current vertex.

[0202] Here, the preset traversal order can be a BFS order. Thus, after determining the starting vertex, the vertices of the three-dimensional mesh can be reordered according to the BFS order to determine the preset vertex order, which is then used as a basis. In the K-nearest neighbor graph, a k-hop subgraph is established for each vertex. Then, according to the preset vertex order after the breadth-first traversal, the connectivity of each k-hop subgraph is predicted using the preset network model.

[0203] Furthermore, for the starting vertex of the sort, one of the vertices can be randomly selected as the starting vertex, or the vertex at a specific position (such as the lower left corner) can be used as the starting vertex. It only needs to be consistent between the encoding and decoding ends. The embodiments of this application do not impose any restrictions.

[0204] In a possible embodiment, determining the starting vertex from the vertices of the three-dimensional mesh may include: selecting a vertex at a preset position from the vertices of the three-dimensional mesh; and determining the vertex at the preset position as the starting vertex.

[0205] In another possible embodiment, determining the starting vertex from the vertices of the three-dimensional mesh may include: encoding the starting vertex and writing the obtained encoding bits into a bitstream.

[0206] In another possible embodiment, determining a starting vertex from vertices of a three-dimensional mesh may include: determining a starting vertex index value corresponding to the starting vertex; encoding the starting vertex index value, and writing the resulting encoded bits into a bitstream, wherein the starting vertex index value corresponds to the starting vertex.

[0207] It should be noted that in the embodiments of the present application, the starting vertex for sorting can be any vertex. For example, the starting vertex can be a vertex at a preset position, or it can be encoded and written into the bitstream so that the decoder can determine the starting vertex by parsing the bitstream. Alternatively, the starting vertex index value can be encoded and written into the bitstream so that the decoder can also determine the starting vertex by parsing the bitstream, thereby ensuring consistency between the encoder and decoder. However, this is not a limitation.

[0208] It should also be noted that, in the embodiment of the present application, one of the vertices is randomly selected as the starting point, and the vertices are reordered in the BFS order, and the adjacency matrix is ​​also reordered based on this order. In some embodiments, the method may also include:

[0209] According to the connectivity of the three-dimensional grid, an adjacency matrix is ​​established;

[0210] Reorder the adjacency matrix according to a preset vertex order to determine the target adjacency matrix;

[0211] Determine the true weight values ​​of the edges of the 3D mesh according to the target adjacency matrix.

[0212] Here, the element at the (i, j)th position in the adjacency matrix indicates whether the i-th vertex is connected to the j-th vertex, and both i and j are positive integers.

[0213] Furthermore, for the true weight value of the edge between the i-th vertex and the j-th vertex, in some embodiments, the method may further include:

[0214] If the i-th vertex is connected to the j-th vertex, the true weight value of the edge between the i-th vertex and the j-th vertex is set to the first value;

[0215] If the i-th vertex is not connected to the j-th vertex, the true weight value of the edge between the i-th vertex and the j-th vertex is set to the second value.

[0216] Here, the first value can be set to 1, and the second value can be set to 0. That is, in the embodiment of the present application, the edge weight value can be used to indicate whether two vertices are connected. If they are truly connected, the actual edge weight value can be set to 1; if they are not connected, the actual edge weight value can be set to 0.

[0217] In a specific embodiment, for a k-hop subgraph, establishing the k-hop subgraph of the current vertex may include:

[0218] If the weight value of the first edge in the k-hop subgraph has been encoded and predicted, the initial weight value of the first edge in the k-hop subgraph is set to the actual weight value of the first edge;

[0219] If the weight value of the first edge in the k-hop subgraph has not been coded and predicted, the initial weight value of the first edge in the k-hop subgraph is set to a preset value; wherein the first edge represents an edge formed between any two vertices among multiple target vertices of the k-hop subgraph.

[0220] It should be noted that if the weight value of the first edge in the k-hop subgraph has been coded and predicted, then the initial weight value of the first edge can be set to the decoded true weight value of the first edge; otherwise, if the weight value of the first edge in the k-hop subgraph has not been coded and predicted, then the initial weight value of the first edge can be set to a preset value. Here, the preset value can be 0.5, but there is no limitation. In other words, when processing a certain k-hop subgraph, since some edges have been predicted and coded and some edges have not been predicted, the weight values ​​of the edges that have been predicted and coded can be initialized to true values ​​(that is, if the two points are connected, they are initialized to 1, and if the two points are not connected, they are initialized to 0), and the weight values ​​of the edges that have not been predicted and coded can be initialized to 0.5.

[0221] It is understood that after establishing a k-hop subgraph, connectivity prediction can be performed based on the k-hop subgraph and a preset network model to determine the predicted weight values ​​of the edges in the k-hop subgraph. Specifically, embodiments of the present application can obtain spatial feature information of multiple target vertices from the spatial feature information of the vertices of the three-dimensional mesh, input the spatial feature information of the multiple target vertices and the initial weight values ​​of the edges into the preset network model, and determine the predicted weight values ​​of the edges in the k-hop subgraph based on the preset network model.

[0222] In an embodiment of the present application, the preset network model may also be referred to as a local connectivity prediction network model. The preset network model here may at least include a graph neural network model based on graph convolution, that is, a graph neural network model based on GCN, but there is no limitation on this.

[0223] In some embodiments, the network structure of the preset network model may include: at least one vertex feature updating module and at least one edge feature updating module; wherein:

[0224] The vertex feature update module is composed of a first multi-layer perceptron module, a second multi-layer perceptron module and a graph convolutional network module;

[0225] The edge feature update module is composed of a third multi-layer perceptron module.

[0226] Here, the first multi-layer perceptron module is MLP (E in ,E out ) indicates that the second multi-layer perceptron module is represented by MLP(P in ,P out ) indicates that, where E in Represents the edge feature information of MLP input, E out represents the edge feature information output by MLP; P in Represents the vertex feature information of MLP input, P out Represents the vertex feature information output by MLP. After processing by MLP, E out The edge feature dimension and P out The vertex feature dimensions are the same, and then the vertex features can be updated through the graph convolutional network module (GCN Layer).

[0227] In some embodiments, for the third multilayer perceptron module, the method may further include: inputting the edge to be updated and two vertices connected by the edge to be updated into the third multilayer perceptron module, and determining the output of the third multilayer perceptron module as the updated edge.

[0228] Here, the third multi-layer perceptron module uses f mlp (p1,p2,edge in ) indicates that, where edge in is the edge to be updated, p1 and p2 are edge in The two connected vertices, the output of the third multi-layer perceptron module is the updated edge update , f mlp It represents an MLP network. It should be noted that in the last edge feature update module, f mlp Set the output dimension to 1.

[0229] For example, in the aforementioned embodiment, FIG13 shows a schematic diagram of the structure of a preset network model provided in an embodiment of the present application. For the vertex feature update module in FIG13 , see FIG14 for an example of the structure of the vertex feature update module; and for the edge feature update module, it can be expressed using the following formula:

[0230] edge update =f mlp (p1,p2,edge in )

[0231] Among them, edge in is the edge to be updated, p1 and p2 are edge in The two connected vertices, edge update is the updated edge, fmlp Is an MLP network. In the last edge feature update module, f mlp Set the output dimension to 1.

[0232] S1503: Determine the true weight value of the edge of the three-dimensional grid, and determine the prediction residual value of the edge of the three-dimensional grid according to the true weight value and the predicted weight value of the edge of the three-dimensional grid.

[0233] It should be noted that after performing connectivity prediction based on the k-hop subgraph and the preset network model and determining the predicted weight values ​​of the edges of the three-dimensional grid, the predicted residual values ​​of the edges of the three-dimensional grid can be further determined.

[0234] In a specific embodiment, determining the predicted residual value of the edge of the three-dimensional grid based on the true weight value and the predicted weight value of the edge of the three-dimensional grid can include: performing a difference calculation on the true weight value and the predicted weight value of the edge of the three-dimensional grid to determine the predicted residual value of the edge of the three-dimensional grid.

[0235] Furthermore, in some embodiments, the method may further include: encoding the prediction residual values ​​of the edges of the three-dimensional grid, and writing the obtained encoding bits into the bitstream.

[0236] That is to say, after the encoder performs local connectivity prediction through the preset network model, it can obtain the predicted weight value of the edge; by subtracting the predicted weight value of the edge from the true weight value, the predicted residual value can be obtained (because the weight value represents the connection relationship, the predicted residual value is the connectivity residual). At this time, the predicted residual value is written into the bitstream, specifically into the connectivity residual compressed bitstream.

[0237] Furthermore, in some embodiments, the embodiments of the present application also provide a code stream, which is generated by bit encoding based on the information to be encoded; wherein the information to be encoded may at least include: vertex information of the three-dimensional grid and predicted residual values ​​of the edges of the three-dimensional grid.

[0238] In this way, after the encoder transmits the vertex information of the three-dimensional mesh and the predicted residual values ​​of the edges of the three-dimensional mesh to the decoder through the code stream, the decoder can obtain the vertex information of the three-dimensional mesh and the predicted residual values ​​of the edges by parsing the code stream, and then guide the predictive coding of the connectivity by extracting the spatial features of the vertex information of the three-dimensional mesh; and then add the predicted weight value obtained by prediction and the decoded prediction residual value to determine the reconstruction weight value of the edge of the three-dimensional mesh, so as to accurately determine the connectivity of each edge.

[0239] In short, the embodiment of the present application proposes a geometric compression algorithm for three-dimensional meshes, which is mainly aimed at the connectivity of three-dimensional meshes. Specifically, in the embodiment of the present application, the embodiment of the present application uses G-PCC, V-PCC or other point cloud geometric lossless coding algorithms to encode the vertices of the three-dimensional mesh. Unlike the algorithm in the related art that expands V-PCC to compress the three-dimensional mesh, the algorithm of the embodiment of the present application uses a deep learning-based solution to predict the connectivity between vertices, and then encodes the residual prediction residual value between the predicted value and the true value, so that the decoding end can obtain the prediction residual value by parsing the code stream, and then determine the reconstruction weight value to determine the connectivity between the predicted vertices.

[0240] This embodiment provides an encoding method, which determines vertex information of a three-dimensional mesh; extracts spatial features based on the vertex information of the three-dimensional mesh to determine the spatial feature information of the vertices of the three-dimensional mesh; predicts connectivity based on the vertex information of the three-dimensional mesh to determine the predicted weight values ​​of the edges of the three-dimensional mesh; determines the true weight values ​​of the edges of the three-dimensional mesh, and determines the predicted residual values ​​of the edges of the three-dimensional mesh based on the true weight values ​​and the predicted weight values ​​of the edges of the three-dimensional mesh. In this way, by extracting spatial features from the vertices of the three-dimensional mesh, the spatial feature information obtained can be used to guide predictive encoding of connectivity; and by locally processing the three-dimensional mesh based on the k-hop subgraph, the shortcomings of the unstructured and difficult-to-divide three-dimensional mesh data can be resolved; in addition, the vertices are reordered in a preset manner to ensure that the order at the encoding end and the decoding end is the same, so that the encoding end does not need to transmit the vertex mapping code stream, thereby significantly improving the geometric encoding efficiency of the three-dimensional mesh and improving the compression performance of the three-dimensional mesh.

[0241] In another embodiment of the present application, based on the method of the aforementioned embodiment, the embodiment of the present application proposes a coding and decoding framework for three-dimensional mesh connectivity prediction based on deep learning, and its overall flow chart is specifically shown in Figure 16. As shown in Figure 16, for a three-dimensional mesh, the encoding end can obtain the vertex information of the three-dimensional mesh, and then use the point cloud lossless compression encoder to encode it and write it into the vertex compression code stream; then decode it through the point cloud lossless compression decoder, so that the decoding end can obtain the vertex information of the three-dimensional mesh. For connectivity, the encoding end also needs to perform real connectivity processing, which may specifically include adjacency matrix and adjacency matrix reordering; for vertex information, it is also necessary to perform steps such as spatial feature extraction and local feature prediction, where local feature prediction may include KNN graph establishment, k-hop subgraph establishment and local connectivity prediction. Then, the predicted weight value and the true weight value are subtracted to obtain the predicted residual value; the predicted residual value is residually encoded and written into the connectivity residual compressed code stream; finally, residual decoding is performed at the decoding end to obtain the predicted residual value; and according to the steps of spatial feature extraction and local feature prediction at the decoding end, the local feature prediction still includes KNN graph establishment, k-hop subgraph establishment and local connectivity prediction to obtain the predicted weight value. According to the predicted residual value and the predicted weight value, the reconstruction weight value can be obtained to determine the connectivity, and then the three-dimensional grid can be restored.

[0242] In the embodiment of the present application, whether it is the encoding end or the decoding end, the encoding and decoding framework here is not limited to processing point cloud data, but can also process other forms of three-dimensional data, and there is no limitation on this.

[0243] Based on Figure 16, this method can be described as follows: the geometry of a 3D mesh is compressed separately: vertices and connectivity. Vertices are encoded using a point cloud lossless compression algorithm (V-PCC, G-PCC, or other), while connectivity is encoded using a deep learning-based predictive coding algorithm. Because the connectivity of a 3D mesh is related to its vertex coordinates, the prediction algorithm first extracts the spatial features of the vertices. Then, these features are used together with the actual connectivity to perform a step-by-step predictive coding of the connectivity.

[0244] In a specific embodiment, the entire algorithm is divided into the following steps:

[0245] Step 1: The vertex information of the three-dimensional mesh is input into the spatial feature extraction network model to obtain the spatial feature information of each vertex.

[0246] Optionally, the network model here can be a point-net-based spatial feature extraction network model or a sparse convolution-based spatial feature extraction network model. The 3D coordinates of the vertices of the 3D mesh (N×3) (or the 3D coordinates concatenated with the 3D normal vectors (N×6)) are input, and the output is the spatial feature information of the vertex (N×C), where C is the dimension of the spatial feature information and N is the number of vertices in the 3D mesh.

[0247] Step 2: Create a K-nearest neighbor graph based on the 3D coordinates of the vertices of the 3D grid. Specifically, for each vertex, connect this point with its K nearest neighbor points to obtain the K-nearest neighbor graph of the vertex.

[0248] Optionally, the value range of K here is K∈(0,N), where N is the number of vertices of the three-dimensional mesh.

[0249] Step 3: Randomly select one of the vertices as the starting vertex, reorder the vertices in BFS order, and reorder the adjacency matrix based on this order.

[0250] Optionally, the starting vertex for sorting can be any vertex, as long as it remains consistent between the encoding and decoding ends.

[0251] Optionally, the sorting method is not limited to BFS, and other sorting algorithms can also be used, as long as the sorting on the encoding end and the decoding end is the same.

[0252] Step 4: For each vertex in the K-nearest neighbor graph, create a k-hop subgraph. Note that k≠K here.

[0253] Step 5: In the order after the breadth-first traversal mentioned above, for each k-hop subgraph, the local connectivity prediction network model is used to predict the connectivity of the k-hop subgraph, thereby performing predictive coding.

[0254] Specifically, the embodiment of the present application uses the weight of the edge in the k-hop subgraph to indicate whether two points are connected. If they are connected, the weight value is set to 1, and if they are not connected, the weight value is set to 0. When processing a certain k-hop subgraph, since some edges have been predictively coded and some have not, the weights of the predictively coded edges can be initialized to real values ​​(i.e., if the two points are connected, they are initialized to 1, and if they are not connected, they are initialized to 0), and the weights of the edges that have not been predictively coded can be initialized to 0.5.

[0255] In step 6, the spatial feature information of the vertices of the k-hop subgraph and the initialized edge weights are input into the local connectivity prediction network model. After passing through the local connectivity prediction network model, the predicted edge weights are obtained. The predicted edge weights are subtracted from the true weights to obtain the predicted residual value (which can be called the weight residual value because the weight value represents the connection relationship, so the weight residual value is the connectivity residual), and then the residual is encoded.

[0256] Optionally, the local connectivity prediction network model can be a GCN-based graph neural network model.

[0257] In another specific embodiment, an embodiment proposed based on the above algorithm is as follows:

[0258] The point cloud compression codec uses G-PCC. The spatial feature extraction network model uses the framework shown in Figure 8. The input is the N×3 3D coordinates of the vertices of a 3D mesh, where N is the number of vertices and 3 is the dimension of the vertex coordinates. The output is the N×C spatial features of the 3D mesh vertices, where C is the feature dimension.

[0259] The components of the downsampling module and the upsampling module shown in FIG8 are shown in FIG9 and FIG10 respectively, and FIG11 specifically shows the IRN module used in both the downsampling module and the upsampling module.

[0260] In addition, the local connectivity prediction network model is shown in Figure 13. The input is the k-hop subgraph after connectivity weight initialization, and the output is the k-hop subgraph after local connectivity prediction. The structure of the vertex feature update module shown in Figure 13 is shown in Figure 14. The edge feature update module in Figure 13 can be expressed by the following formula:

[0261] edge update =f mlp (p1,p2,edge in )

[0262] Among them, edge in is the edge to be updated, p1 and p2 are edge in The two connected vertices, edge update is the updated edge, f mlp Is an MLP network. In the last edge feature update module, f mlp Set the output dimension to 1.

[0263] It can also be understood that the embodiments of the present application can also replace the network model shown in the above embodiments with other deep learning networks, such as a point cloud feature extraction network model based on pointnet++.

[0264] The embodiment of the present application provides a connectivity prediction method. The specific implementation of the aforementioned embodiment is described in detail through the above embodiment. It can be seen that according to the technical solution described in the aforementioned embodiment, on the one hand, spatial feature extraction is performed on the vertex information of the three-dimensional mesh to guide the prediction coding of connectivity; on the other hand, in the connectivity prediction process, the vertices are reordered using a specific algorithm to ensure that the vertex order at the codec end is the same, avoiding the impact caused by inconsistent output of the codec end of the point cloud coding algorithm; based on the k-hop subgraph, the three-dimensional mesh is locally processed to solve the shortcomings of the unstructured and difficult to divide three-dimensional mesh data; in addition, the implementation of the local connectivity prediction network includes the local connectivity prediction network model architecture and the vertex feature update module and the edge feature update module. In this way, by using the connectivity prediction method proposed by this technical solution, combined with the existing point cloud coding algorithm, the geometric coding efficiency of the three-dimensional mesh can be significantly improved, and the compression performance of the three-dimensional mesh can be improved.

[0265] In another embodiment of the present application, based on the same inventive concept as the above embodiment, see FIG17 , which shows a schematic diagram of the structure of an encoder 170 provided by an embodiment of the present application. As shown in FIG17 , the encoder 170 may include: a first determination unit 1701 and a first prediction unit 1702; wherein,

[0266] A first determining unit 1701 is configured to determine vertex information of a three-dimensional mesh;

[0267] A first prediction unit 1702 is configured to perform connectivity prediction based on vertex information of the three-dimensional mesh and determine a prediction weight value of an edge of the three-dimensional mesh;

[0268] The first determining unit 1701 is further configured to determine a true weight value of an edge of the three-dimensional grid, and determine a prediction residual value of the edge of the three-dimensional grid according to the true weight value and the predicted weight value of the edge of the three-dimensional grid.

[0269] In some embodiments, referring to FIG. 17 , the encoder 170 may further include an encoding unit 1703 configured to encode the prediction residual values ​​of the edges of the three-dimensional mesh and write the obtained encoding bits into the bitstream.

[0270] In some embodiments, the encoding unit 1703 is further configured to encode vertex information of the three-dimensional mesh and write the obtained encoding bits into the bitstream.

[0271] In some embodiments, referring to FIG. 17 , the encoder 170 may further include a first construction unit 1704 configured to construct a local subgraph based on vertex information of the three-dimensional mesh, and determine a k-hop subgraph of a current vertex; wherein the k-hop subgraph includes spatial feature information of multiple target vertices and an initial weight value of at least one edge;

[0272] The first prediction unit 1702 is further configured to input the k-hop subgraph into a preset network model, and determine a prediction weight value of at least one edge in the k-hop subgraph based on the preset network model; wherein k is a positive integer.

[0273] In some embodiments, referring to FIG. 17 , the encoder 170 may further include a first extraction unit 1705 configured to perform spatial feature extraction based on vertex information of the three-dimensional mesh, determine the spatial feature information of the vertices of the three-dimensional mesh; and obtain spatial feature information of multiple target vertices from the spatial feature information of the vertices of the three-dimensional mesh.

[0274] In some embodiments, the first extraction unit 1705 is further configured to input the vertex information of the three-dimensional mesh into the spatial feature extraction network model, and determine the spatial feature information of the vertices of the three-dimensional mesh based on the spatial feature extraction network model.

[0275] In some embodiments, the spatial feature extraction network model includes at least one of the following: a spatial feature extraction network model based on a point network (Point-Net) and a spatial feature extraction network model based on a sparse convolution.

[0276] In some embodiments, the network structure of the spatial feature extraction network model includes: at least one downsampling module, at least one upsampling module and a cropping module; wherein: the downsampling module is composed of a convolution layer, an activation function layer and a residual network module; the upsampling module is composed of a deconvolution layer, an activation function layer and a residual network module.

[0277] In some embodiments, the first construction unit 1704 is further configured to perform a K-nearest neighbor search based on the vertex information of the three-dimensional mesh, and establish a K-nearest neighbor graph of the vertices of the three-dimensional mesh; wherein, in the K-nearest neighbor graph, the vertex has a connection relationship with the K nearest neighbor points; and based on the K-nearest neighbor graph of the vertices of the three-dimensional mesh, establish a k-hop subgraph of the current vertex; wherein, k and K are both positive integers, and k is not equal to K.

[0278] In some embodiments, the first determination unit 1701 is further configured to determine a starting vertex from the vertices of the three-dimensional mesh, reorder the vertices of the three-dimensional mesh according to a preset traversal order, and determine a preset vertex order; and use the vertices of the three-dimensional mesh as current vertices in sequence according to the preset vertex order to establish a k-hop subgraph of the current vertex.

[0279] In some embodiments, the first construction unit 1704 is further configured to establish an adjacency matrix based on the connectivity of the three-dimensional grid; wherein the element at the (i, j)th position in the adjacency matrix indicates whether the i-th vertex is connected to the j-th vertex, and i and j are both positive integers;

[0280] The first determining unit 1701 is further configured to reorder the adjacency matrix according to a preset vertex order to determine a target adjacency matrix; and determine the true weight values ​​of the edges of the three-dimensional mesh according to the target adjacency matrix.

[0281] In some embodiments, the first determination unit 1701 is further configured to set the true weight value of the edge between the i-th vertex and the j-th vertex to the first value if the i-th vertex is connected to the j-th vertex; and to set the true weight value of the edge between the i-th vertex and the j-th vertex to the second value if the i-th vertex is not connected to the j-th vertex.

[0282] In some embodiments, the first determining unit 1701 is further configured to select a vertex at a preset position from the vertices of the three-dimensional mesh; and determine the vertex at the preset position as the starting vertex.

[0283] In some embodiments, the encoding unit 1703 is further configured to encode the starting vertex and write the obtained encoding bits into the bitstream.

[0284] In some embodiments, the first determining unit 1701 is further configured to determine a starting vertex index value corresponding to the starting vertex;

[0285] The encoding unit 1703 is further configured to encode the starting vertex index value and write the obtained encoding bits into the bitstream.

[0286] In some embodiments, the first prediction unit 1702 is further configured to set the initial weight value of the first edge in the k-hop subgraph to the actual weight value of the first edge if the weight value of the first edge in the k-hop subgraph has been encoded and predicted; and to set the initial weight value of the first edge in the k-hop subgraph to a preset value if the weight value of the first edge in the k-hop subgraph has not been encoded and predicted; wherein the first edge represents any one of at least one edge in the k-hop subgraph.

[0287] In some embodiments, the preset network model includes at least a graph neural network model based on graph convolution.

[0288] In some embodiments, the network structure of the preset network model includes: at least one vertex feature update module and at least one edge feature update module; wherein: the vertex feature update module is composed of a first multi-layer perceptron module, a second multi-layer perceptron module and a graph convolutional network module; the edge feature update module is composed of a third multi-layer perceptron module.

[0289] In some embodiments, the first determining unit 1701 is further configured to input the edge to be updated and the two vertices connected by the edge to be updated into a third multi-layer perceptron module, and determine that the output of the third multi-layer perceptron module is the updated edge.

[0290] It is understandable that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.

[0291] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0292] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 170. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method described in any one of the aforementioned embodiments.

[0293] Based on the composition of the above-mentioned encoder 170 and the computer-readable storage medium, refer to Figure 18, which shows a specific hardware structure diagram of the encoder 170 provided in an embodiment of the present application. As shown in Figure 18, the encoder 170 may include: a first communication interface 1801, a first memory 1802 and a first processor 1803; each component is coupled together through a first bus system 1804. It can be understood that the first bus system 1804 is used to realize the connection and communication between these components. In addition to the data bus, the first bus system 1804 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 1804 in Figure 18. Among them,

[0294] The first communication interface 1801 is used to receive and send signals when sending and receiving information with other external network elements;

[0295] A first memory 1802 is used to store computer programs that can be run on the first processor 1803;

[0296] The first processor 1803 is configured to, when running the computer program, execute:

[0297] Determine the vertex information of the three-dimensional mesh;

[0298] Performing connectivity prediction based on vertex information of the three-dimensional mesh and determining predicted weight values ​​of edges of the three-dimensional mesh;

[0299] The true weight value of the edge of the three-dimensional grid is determined, and the prediction residual value of the edge of the three-dimensional grid is determined according to the true weight value and the predicted weight value of the edge of the three-dimensional grid.

[0300] It is understood that the first memory 1802 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 1802 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0301] The first processor 1803 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 1803. The above-mentioned first processor 1803 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 1802 , and the first processor 1803 reads the information in the first memory 1802 and completes the steps of the above method in combination with its hardware.

[0302] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0303] Optionally, as another embodiment, the first processor 1803 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0304] This embodiment provides an encoder in which spatial feature information obtained by extracting spatial features from the vertices of a three-dimensional mesh can be used to guide predictive coding of connectivity. Furthermore, local processing of the three-dimensional mesh based on a k-hop subgraph can address the shortcomings of unstructured and difficult-to-divide three-dimensional mesh data. Furthermore, vertices are reordered in a preset manner to ensure the same order at the encoding and decoding ends, eliminating the need to transmit a vertex mapping code stream. This can significantly improve the geometric coding efficiency of the three-dimensional mesh and improve the compression performance of the three-dimensional mesh.

[0305] In another embodiment of the present application, based on the same inventive concept as the above embodiment, refer to FIG19 , which shows a schematic diagram of the structure of a decoder 190 provided by the embodiment of the present application. As shown in FIG19 , the decoder 190 may include: a second determination unit 1901, a second prediction unit 1902 and a decoding unit 1903; wherein,

[0306] The second determining unit 1901 is configured to determine vertex information of a three-dimensional mesh;

[0307] The second prediction unit 1902 is configured to perform connectivity prediction based on vertex information of the three-dimensional mesh and determine prediction weight values ​​of edges of the three-dimensional mesh;

[0308] The decoding unit 1903 is configured to parse the bitstream and determine the prediction residual value of the edge of the three-dimensional grid;

[0309] The second determining unit 1901 is further configured to determine a reconstruction weight value of an edge of the three-dimensional mesh according to the prediction weight value and the prediction residual value.

[0310] In some embodiments, the decoding unit 1903 is further configured to parse the code stream to determine vertex information of the three-dimensional mesh.

[0311] In some embodiments, referring to FIG19 , the decoder 190 may further include a second construction unit 1904 configured to construct a local subgraph based on vertex information of the three-dimensional mesh, and determine a k-hop subgraph of the current vertex; wherein the k-hop subgraph includes spatial feature information of multiple target vertices and an initial weight value of at least one edge;

[0312] The second prediction unit 1902 is further configured to input the k-hop subgraph into a preset network model, and determine a prediction weight value of at least one edge in the k-hop subgraph based on the preset network model; wherein k is a positive integer.

[0313] In some embodiments, referring to FIG. 19 , the decoder 190 may further include a second extraction unit 1905 configured to perform spatial feature extraction based on vertex information of the three-dimensional mesh, determine the spatial feature information of the vertices of the three-dimensional mesh; and obtain spatial feature information of multiple target vertices from the spatial feature information of the vertices of the three-dimensional mesh.

[0314] In some embodiments, the second extraction unit 1905 is configured to input the vertex information of the three-dimensional mesh into the spatial feature extraction network model, and determine the spatial feature information of the vertices of the three-dimensional mesh based on the spatial feature extraction network model.

[0315] In some embodiments, the spatial feature extraction network model includes at least one of the following: a spatial feature extraction network model based on a point network (Point-Net) and a spatial feature extraction network model based on a sparse convolution.

[0316] In some embodiments, the network structure of the spatial feature extraction network model includes: at least one downsampling module, at least one upsampling module and a cropping module; wherein: the downsampling module is composed of a convolution layer, an activation function layer and a residual network module; the upsampling module is composed of a deconvolution layer, an activation function layer and a residual network module.

[0317] In some embodiments, the second construction unit 1904 is configured to perform a K-nearest neighbor search based on the vertex information of the three-dimensional mesh, and establish a K-nearest neighbor graph of the vertices of the three-dimensional mesh; wherein, in the K-nearest neighbor graph, the vertex has a connection relationship with the K nearest neighbor points; and based on the K-nearest neighbor graph of the vertices of the three-dimensional mesh, establish a k-hop subgraph of the current vertex; wherein, k and K are both positive integers, and k is not equal to K.

[0318] In some embodiments, the second determination unit 1901 is further configured to determine a starting vertex from the vertices of the three-dimensional mesh, reorder the vertices of the three-dimensional mesh according to a preset traversal order, and determine a preset vertex order; and use the vertices of the three-dimensional mesh as current vertices in sequence according to the preset vertex order to establish a k-hop subgraph of the current vertex.

[0319] In some embodiments, the second determining unit 1901 is further configured to select a vertex at a preset position from the vertices of the three-dimensional mesh; and determine the vertex at the preset position as the starting vertex.

[0320] In some embodiments, the decoding unit 1903 is further configured to parse the code stream and determine the starting vertex.

[0321] In some embodiments, the decoding unit 1903 is further configured to parse the code stream to determine the starting vertex index value;

[0322] The second determining unit 1901 is further configured to determine a starting vertex from the vertices of the three-dimensional mesh according to the starting vertex index value; wherein there is a corresponding relationship between the starting vertex index value and the starting vertex.

[0323] In some embodiments, the second prediction unit 1902 is further configured to set the initial weight value of the first edge in the k-hop subgraph to the reconstructed weight value of the first edge if the weight value of the first edge in the k-hop subgraph has been decoded and predicted; and to set the initial weight value of the first edge in the k-hop subgraph to a preset value if the weight value of the first edge in the k-hop subgraph has not been decoded and predicted; wherein the first edge represents any one of at least one edge in the k-hop subgraph.

[0324] In some embodiments, the preset network model includes at least a graph neural network model based on graph convolution.

[0325] In some embodiments, the network structure of the preset network model includes: at least one vertex feature update module and at least one edge feature update module; wherein: the vertex feature update module is composed of a first multi-layer perceptron module, a second multi-layer perceptron module and a graph convolutional network module; the edge feature update module is composed of a third multi-layer perceptron module.

[0326] In some embodiments, the second determining unit 1901 is further configured to input the edge to be updated and the two vertices connected by the edge to be updated into a third multi-layer perceptron module, and determine that the output of the third multi-layer perceptron module is the updated edge.

[0327] In some embodiments, the second prediction unit 1902 is further configured to determine that the two target vertices corresponding to the edge of the three-dimensional mesh are connected if the reconstruction weight value of the edge of the three-dimensional mesh satisfies a first preset range; and to determine that the two target vertices corresponding to the edge of the three-dimensional mesh are not connected if the reconstruction weight value of the edge of the three-dimensional mesh satisfies a second preset range.

[0328] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0329] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer-readable storage medium, which is applied to the decoder 190 and stores a computer program. When the computer program is executed by the second processor, it implements any of the methods in the aforementioned embodiments.

[0330] Based on the composition of the above-mentioned decoder 190 and the computer-readable storage medium, refer to Figure 20, which shows a specific hardware structure diagram of the decoder 190 provided in an embodiment of the present application. As shown in Figure 20, the decoder 190 may include: a second communication interface 2001, a second memory 2002 and a second processor 2003; each component is coupled together through a second bus system 2004. It can be understood that the second bus system 2004 is used to achieve connection and communication between these components. In addition to the data bus, the second bus system 2004 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the second bus system 2004 in Figure 20. Among them,

[0331] The second communication interface 2001 is used for sending and receiving signals during the process of sending and receiving information with other external network elements;

[0332] The second memory 2002 is used to store computer programs that can be run on the second processor 2003;

[0333] The second processor 2003 is configured to, when running the computer program, execute:

[0334] Determine the vertex information of the three-dimensional mesh;

[0335] Performing connectivity prediction based on vertex information of the three-dimensional mesh and determining predicted weight values ​​of edges of the three-dimensional mesh;

[0336] Parse the code stream to determine the prediction residual value of the edge of the 3D grid;

[0337] The reconstruction weight value of the edge of the three-dimensional grid is determined according to the prediction weight value and the prediction residual value.

[0338] Optionally, as another embodiment, the second processor 2003 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0339] It can be understood that the hardware functions of the second memory 2002 are similar to those of the first memory 1802, and the hardware functions of the second processor 2003 are similar to those of the first processor 1803; they will not be described in detail here.

[0340] This embodiment provides a decoder in which spatial feature information is extracted from the vertices of a three-dimensional mesh, and can be used to guide predictive coding of connectivity. Furthermore, local processing of the three-dimensional mesh based on a k-hop subgraph can address the shortcomings of unstructured and difficult-to-divide three-dimensional mesh data. Furthermore, vertices are reordered in a preset manner to ensure the same order at the encoding and decoding ends, eliminating the need to transmit a vertex mapping code stream. This can significantly improve the geometric coding efficiency of the three-dimensional mesh and improve the compression performance of the three-dimensional mesh.

[0341] In yet another embodiment of the present application, referring to Figure 21 , a schematic diagram illustrating the structure of a coding and decoding system provided in an embodiment of the present application is shown. As shown in Figure 21 , the coding and decoding system 210 may include an encoder 170 and a decoder 190. The encoder 170 is the encoder described in any of the aforementioned embodiments, and the decoder 190 is the decoder described in any of the aforementioned embodiments.

[0342] In an embodiment of the present application, the encoder 170 encodes information to be encoded (e.g., vertex information of a 3D mesh and prediction residual values ​​of edges of the 3D mesh), and then writes the information into a bitstream for transmission to the decoder 190. Thus, both the encoding and decoding ends can extract spatial features from the vertices of the 3D mesh, and the resulting spatial feature information can be used to guide predictive coding of connectivity. Furthermore, local processing of the 3D mesh based on a k-hop subgraph can address the shortcomings of unstructured 3D mesh data and its difficulty in partitioning. Furthermore, the vertices are reordered in a preset manner to ensure the same order between the encoding and decoding ends, eliminating the need to transmit a vertex mapping bitstream. This significantly improves the geometric coding efficiency of the 3D mesh and improves the compression performance of the 3D mesh.

[0343] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0344] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0345] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0346] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0347] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0348] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0349] In an embodiment of the present application, after determining the vertex information of the 3D mesh, both the encoder and decoder perform connectivity prediction based on the vertex information of the 3D mesh and determine the predicted weight values ​​of the edges of the 3D mesh. In this way, the encoder determines the predicted residual values ​​of the edges of the 3D mesh based on the actual weight values ​​and the predicted weight values ​​of the edges of the 3D mesh, and transmits the predicted residual values ​​from the encoder to the decoder. The decoder then parses the bitstream to determine the predicted residual values ​​of the edges of the 3D mesh, and determines the reconstruction weight values ​​of the edges of the 3D mesh based on the predicted weight values ​​and the predicted residual values. In this way, by extracting spatial features from the vertices of the three-dimensional mesh, the obtained spatial feature information can be used to guide the predictive coding of connectivity; and by reordering the vertices in a preset manner to ensure that the order at the encoding end and the decoding end is the same, there is no need to transmit the vertex mapping code stream that reflects the vertex order, which can reduce the bit rate; in addition, local processing of the three-dimensional mesh based on the k-hop subgraph can also solve the shortcomings of the unstructured and difficult to divide three-dimensional mesh data, so that when predicting connectivity based on the vertex information of the three-dimensional mesh, the geometric coding efficiency of the three-dimensional mesh can be significantly improved, and the compression performance of the three-dimensional mesh can be improved.

Claims

1. A decoding method, applied to a decoder, comprising: Determine the vertex information of the three-dimensional mesh; Performing connectivity prediction based on vertex information of the three-dimensional mesh to determine predicted weight values ​​of edges of the three-dimensional mesh; Parsing a bitstream to determine a prediction residual value of an edge of the three-dimensional grid; The reconstruction weight value of the edge of the three-dimensional grid is determined according to the prediction weight value and the prediction residual value.

2. The method according to claim 1, wherein Determining vertex information of the three-dimensional mesh includes: Parse the code stream to determine vertex information of the three-dimensional mesh.

3. The method according to claim 1, wherein The performing connectivity prediction based on the vertex information of the three-dimensional mesh to determine the predicted weight value of the edge of the three-dimensional mesh includes: Constructing a local subgraph based on the vertex information of the three-dimensional grid to determine a k-hop subgraph of the current vertex; wherein the k-hop subgraph includes spatial feature information of multiple target vertices and an initial weight value of at least one edge; The k-hop subgraph is input into a preset network model, and a predicted weight value of at least one edge in the k-hop subgraph is determined based on the preset network model; wherein k is a positive integer.

4. The method according to claim 3, wherein: The method further comprises: Performing spatial feature extraction based on vertex information of the three-dimensional mesh to determine spatial feature information of the vertices of the three-dimensional mesh; The spatial feature information of the plurality of target vertices is obtained from the spatial feature information of the vertices of the three-dimensional mesh.

5. The method according to claim 4, wherein The extracting spatial features based on the vertex information of the three-dimensional mesh to determine the spatial feature information of the vertices of the three-dimensional mesh includes: The vertex information of the three-dimensional mesh is input into a spatial feature extraction network model, and the spatial feature information of the vertices of the three-dimensional mesh is determined based on the spatial feature extraction network model.

6. The method according to claim 5, wherein: The spatial feature extraction network model includes at least one of the following: a spatial feature extraction network model based on a point network (Point-Net) and a spatial feature extraction network model based on sparse convolution.

7. The method according to claim 5, wherein: The network structure of the spatial feature extraction network model includes: at least one downsampling module, at least one upsampling module and a cropping module; wherein: The downsampling module is composed of a convolutional layer, an activation function layer and a residual network module; The upsampling module is composed of a deconvolution layer, an activation function layer and a residual network module.

8. The method according to claim 3, wherein: The constructing of a local subgraph based on vertex information of the three-dimensional grid to determine a k-hop subgraph of a current vertex includes: Performing a K-nearest neighbor search based on vertex information of the three-dimensional mesh to establish a K-nearest neighbor graph of the vertices of the three-dimensional mesh; wherein, in the K-nearest neighbor graph, the vertex has a connection relationship with the K nearest neighbor points; A k-hop subgraph of the current vertex is established according to a K-nearest neighbor graph of the vertices of the three-dimensional grid; wherein k and K are both positive integers, and k is not equal to K.

9. The method according to claim 8, wherein The method further comprises: Determining a starting vertex from the vertices of the three-dimensional mesh, and reordering the vertices of the three-dimensional mesh according to a preset traversal order to determine a preset vertex order; According to the preset vertex sequence, vertices of the three-dimensional grid are sequentially used as the current vertex to establish a k-hop subgraph of the current vertex.

10. The method according to claim 9, wherein: Determining a starting vertex from vertices of the three-dimensional mesh includes: Selecting a vertex at a preset position from the vertices of the three-dimensional grid; The vertex at the preset position is determined as the starting vertex.

11. The method according to claim 9, wherein Determining a starting vertex from vertices of the three-dimensional mesh includes: Parse the code stream to determine the starting vertex.

12. The method according to claim 9, wherein Determining a starting vertex from vertices of the three-dimensional mesh includes: Parse the code stream and determine the starting vertex index value; The starting vertex is determined from the vertices of the three-dimensional mesh according to the starting vertex index value; wherein there is a corresponding relationship between the starting vertex index value and the starting vertex.

13. The method according to claim 3, wherein: The step of establishing a k-hop subgraph of the current vertex includes: If the weight value of the first edge in the k-hop subgraph has been decoded and predicted, setting the initial weight value of the first edge in the k-hop subgraph as the reconstructed weight value of the first edge; If the weight value of the first edge in the k-hop subgraph has not been decoded and predicted, the initial weight value of the first edge in the k-hop subgraph is set to a preset value; wherein the first edge represents any one of at least one edge in the k-hop subgraph.

14. The method according to claim 3, wherein: The preset network model includes at least a graph neural network model based on graph convolution.

15. The method according to claim 3, wherein The network structure of the preset network model includes: at least one vertex feature updating module and at least one edge feature updating module; wherein: The vertex feature update module is composed of a first multi-layer perceptron module, a second multi-layer perceptron module and a graph convolutional network module; The edge feature updating module is composed of a third multi-layer perceptron module.

16. The method according to claim 15, wherein The method further comprises: The edge to be updated and two vertices connected by the edge to be updated are input into the third multi-layer perceptron module, and the output of the third multi-layer perceptron module is determined to be the updated edge.

17. The method according to any one of claims 1 to 16, wherein: The method further comprises: If the reconstruction weight value of the edge of the three-dimensional mesh satisfies a first preset range, determining that two target vertices corresponding to the edge of the three-dimensional mesh are connected; If the reconstruction weight value of the edge of the three-dimensional mesh satisfies a second preset range, it is determined that two target vertices corresponding to the edge of the three-dimensional mesh are not connected.

18. A coding method, applied to an encoder, comprising: Determine the vertex information of the three-dimensional mesh; Performing connectivity prediction based on vertex information of the three-dimensional mesh to determine predicted weight values ​​of edges of the three-dimensional mesh; The true weight value of the edge of the three-dimensional grid is determined, and the prediction residual value of the edge of the three-dimensional grid is determined according to the true weight value of the edge of the three-dimensional grid and the predicted weight value.

19. The method according to claim 18, wherein The determining, based on the true weight value of the edge of the three-dimensional grid and the predicted weight value, a predicted residual value of the edge of the three-dimensional grid comprises: A difference calculation is performed between the actual weight value of the edge of the three-dimensional grid and the predicted weight value to determine the predicted residual value of the edge of the three-dimensional grid.

20. The method according to claim 19, wherein The method further comprises: The prediction residual values ​​of the edges of the three-dimensional grid are encoded, and the obtained encoding bits are written into a bitstream.

21. The method according to claim 18, wherein The method further comprises: The vertex information of the three-dimensional mesh is encoded, and the obtained encoding bits are written into a bit stream.

22. The method according to claim 18, wherein Performing connectivity prediction based on vertex information of the three-dimensional mesh to determine predicted weight values ​​of edges of the three-dimensional mesh includes: Constructing a local subgraph based on the vertex information of the three-dimensional grid to determine a k-hop subgraph of the current vertex; wherein the k-hop subgraph includes spatial feature information of multiple target vertices and an initial weight value of at least one edge; The k-hop subgraph is input into a preset network model, and a predicted weight value of at least one edge in the k-hop subgraph is determined based on the preset network model; wherein k is a positive integer.

23. The method according to claim 22, wherein The method further comprises: Performing spatial feature extraction based on vertex information of the three-dimensional mesh to determine spatial feature information of the vertices of the three-dimensional mesh; The spatial feature information of the plurality of target vertices is obtained from the spatial feature information of the vertices of the three-dimensional mesh.

24. The method according to claim 23, wherein The extracting spatial features based on the vertex information of the three-dimensional mesh to determine the spatial feature information of the vertices of the three-dimensional mesh includes: The vertex information of the three-dimensional mesh is input into a spatial feature extraction network model, and the spatial feature information of the vertices of the three-dimensional mesh is determined based on the spatial feature extraction network model.

25. The method according to claim 24, wherein The spatial feature extraction network model includes at least one of the following: a spatial feature extraction network model based on a point network (Point-Net) and a spatial feature extraction network model based on sparse convolution.

26. The method according to claim 24, wherein The network structure of the spatial feature extraction network model includes: at least one downsampling module, at least one upsampling module and a cropping module; wherein: The downsampling module is composed of a convolutional layer, an activation function layer and a residual network module; The upsampling module is composed of a deconvolution layer, an activation function layer and a residual network module.

27. The method according to claim 22, wherein The constructing of a local subgraph based on vertex information of the three-dimensional grid to determine a k-hop subgraph of a current vertex includes: Performing a K-nearest neighbor search based on vertex information of the three-dimensional mesh to establish a K-nearest neighbor graph of the vertices of the three-dimensional mesh; wherein, in the K-nearest neighbor graph, the vertex has a connection relationship with the K nearest neighbor points; A k-hop subgraph of the current vertex is established according to a K-nearest neighbor graph of the vertices of the three-dimensional grid; wherein k and K are both positive integers, and k is not equal to K.

28. The method according to claim 27, wherein The method further comprises: Determining a starting vertex from the vertices of the three-dimensional mesh, and reordering the vertices of the three-dimensional mesh according to a preset traversal order to determine a preset vertex order; According to the preset vertex sequence, vertices of the three-dimensional grid are sequentially used as the current vertex to establish a k-hop subgraph of the current vertex.

29. The method according to claim 28, wherein The method further comprises: An adjacency matrix is ​​established based on the connectivity of the three-dimensional grid; wherein the element at the (i, j)th position in the adjacency matrix indicates whether the i-th vertex is connected to the j-th vertex, and i and j are both positive integers; Reordering the adjacency matrix according to the preset vertex order to determine a target adjacency matrix; According to the target adjacency matrix, the actual weight values ​​of the edges of the three-dimensional grid are determined.

30. The method according to claim 29, wherein The method further comprises: If the i-th vertex is connected to the j-th vertex, setting the true weight value of the edge between the i-th vertex and the j-th vertex to the first value; If the i-th vertex is not connected to the j-th vertex, the true weight value of the edge between the i-th vertex and the j-th vertex is set to a second value.

31. The method of claim 28, wherein Determining a starting vertex from vertices of the three-dimensional mesh includes: Selecting a vertex at a preset position from the vertices of the three-dimensional grid; The vertex at the preset position is determined as the starting vertex.

32. The method of claim 28, wherein: Determining a starting vertex from the vertices of the three-dimensional mesh further includes: The starting vertex is encoded, and the obtained encoding bits are written into a bitstream.

33. The method of claim 28, wherein: Determining a starting vertex from the vertices of the three-dimensional mesh further includes: Determine the starting vertex index value corresponding to the starting vertex; The starting vertex index value is encoded, and the obtained encoding bits are written into a bitstream.

34. The method of claim 22, wherein: The step of establishing a k-hop subgraph of the current vertex includes: If the weight value of the first edge in the k-hop subgraph has been coded and predicted, setting the initial weight value of the first edge in the k-hop subgraph to the actual weight value of the first edge; If the weight value of the first edge in the k-hop subgraph has not been coded and predicted, the initial weight value of the first edge in the k-hop subgraph is set to a preset value; wherein the first edge represents any one of at least one edge in the k-hop subgraph.

35. The method according to any one of claims 18 to 34, wherein The preset network model includes at least a graph neural network model based on graph convolution.

36. The method according to any one of claims 18 to 34, wherein The network structure of the preset network model includes: at least one vertex feature updating module and at least one edge feature updating module; wherein: The vertex feature update module is composed of a first multi-layer perceptron module, a second multi-layer perceptron module and a graph convolutional network module; The edge feature updating module is composed of a third multi-layer perceptron module.

37. The method according to claim 36, wherein The method further comprises: The edge to be updated and two vertices connected by the edge to be updated are input into the third multi-layer perceptron module, and the output of the third multi-layer perceptron module is determined to be the updated edge.

38. A code stream, the code stream being generated by bit encoding based on information to be encoded; wherein, The information to be encoded includes at least: vertex information of a three-dimensional mesh and prediction residual values ​​of edges of the three-dimensional mesh.

39. An encoder comprising a first determining unit and a first predicting unit; wherein: The first determining unit is configured to determine vertex information of the three-dimensional mesh; The first prediction unit is configured to perform connectivity prediction based on vertex information of the three-dimensional mesh and determine a prediction weight value of an edge of the three-dimensional mesh; The first determining unit is further configured to determine a true weight value of the edge of the three-dimensional grid, and determine a prediction residual value of the edge of the three-dimensional grid according to the true weight value of the edge of the three-dimensional grid and the predicted weight value.

40. An encoder comprising a first memory and a first processor; wherein: The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 18 to 37 when running the computer program.

41. A decoder comprising a second determining unit, a second predicting unit, and a decoding unit; wherein: The second determining unit is configured to determine vertex information of the three-dimensional mesh; The second prediction unit is configured to perform connectivity prediction based on vertex information of the three-dimensional mesh and determine a prediction weight value of an edge of the three-dimensional mesh; The decoding unit is configured to parse the bitstream and determine the prediction residual value of the edge of the three-dimensional grid; The second determining unit is further configured to determine a reconstruction weight value of an edge of the three-dimensional grid according to the prediction weight value and the prediction residual value.

42. A decoder comprising a second memory and a second processor; wherein: The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 1 to 17 when running the computer program.

43. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the method according to any one of claims 1 to 17 or the method according to any one of claims 18 to 37 is implemented.

Images